Big Data and the Social Sector: Where does it come from? How can we use it?

In Profiles & Interviews by Srik Gopalakrishnan2 Comments

theese are the brakesAny fast-moving phenomenon breeds a requisite and parallel set of caveats. Srik Gopalakrishnan of FSG gives a sharp look into both the big data phenomenon and the places where we might want to apply the thought brakes.

The massiveness of data (in volume and complexity) is only part of the big story. The other, more critical aspect is our ability to process all this data to generate “business intelligence”. Advanced business analytics are now available to help pore through quantitative AND qualitative data in seconds, generating insights and trends that would formerly have taken days to gather.

Can the social sector benefit from big data? One handicap that is familiar to all of us in the social sector is that data systems and information markets are, to put it mildly, under-developed. Schools struggle to properly keep track of the movement of teachers and students as they enter and leave the school system, leave alone data about their performance. Hospitals have difficulty capturing indicators such as re-admission rates that they are now required to report. Nonprofits may collect data to understand the characteristics and needs of the people they reach, but often lack capacity to collect information related to long-term outcomes. In this context, the notion of big data and predictive analytics may seem like a faraway dream.

Before we close the door on the possibility, let’s take a moment to examine where exactly big data comes from, and how potentially those could be used in the social sector.

  • For example, big data is generated as formerly non-digital processes become digitized. Think everything from credit card payments to downloading mp3s to online shopping. As clicks replace bricks, they also generate data that can be aggregated and sorted at the touch of a button. Now think of the possibilities, say in a classroom, as blended learning becomes the norm and every click that that child makes in her journey toward mastery of a content area is recorded and available for analysis.
  • Big data is also generated as devices become more integrated into our everyday lives. We are almost immune to the devices such as sensors, cameras and GPS’ that play a role in our everyday lives. These devices are capturing information about us that can reveal key trends. Cut to the social sector and we see some of this technology being incorporated into healthcare. We now see virtual communities abound engaging in conversations about their health, driven by data from devices such as sleep pattern sensors.
  • Finally, we generate lots of big data as we willingly share information with the world through social media. Consider the billions of tweets that are sent out every day. Or check-ins on foursquare. Or updates on facebook. Everywhere we go on social media, we leave digital breadcrumbs that point to collective trends. Just as businesses are starting to mine these breadcrumbs, so are some social sector organizations. Ushahidi is an innovative organization that uses aggregated trends from text messages and social media feeds to coordinate immediate response to natural disasters.

However, taking one step back from potential use and the vision of data as the social sector’s messiah, we should keep in mind that this phenomenon is still brand new and we need to address several issues around privacy, accuracy, reliability, and use. Lest we forget, the same data that is used to target services can also be used to target and discriminate people of certain backgrounds. Creating some agreements around use and preventing misuse is critical.

So what does this all mean? Are we poised as a sector to “leapfrog” over our clunky, outdated data infrastructure straight into the league of sophisticated analyses that would help us target our services to the right people in the right places at the right time, and while doing that, also measure our impact with precision?


  1. I find collecting data to be a hit or miss situation. It is easiest collect data on a client because it is apart of the admissions process at the homeless shelter I work for. But, getting data from newsletter subscribers, customers in the thrift store, outside individuals, it just fizzles out. And that’s just the initial hoop to jump.

    With online data tracking, what programs can a nonprofit use for little to no cost when tracking clicks on the website, our social media, and newsletters? Any recommendations from M4G or other commentors in the social sector?

    On a yet another tangent, what should one be tracking when online fundraising? With mailers, I look at ROI on cost to conduct the mailer and the donors generated. But what about online fundraising? I saw a spike in simple layout construction of our website, but are there more techniques I could use besides placing a donate button on the top of every page?

    Thank you for your assistance.

  2. Big data without “big intelligence” is meaningless. In fact, it is our ability to transform the bytes of data into actionable information that converts the by-products of service delivery (education and medical care, in this case) into a resource that can be leveraged to provide additional services, improve targeting or even scale social enterprises.

    As each organization strives to efficiently and effectively fulfill its obligation, it collects information on its perceived KPI. Those KPI may be defined internally by administration or imposed from the outside in the form of regulation. However, anything “beyond the scope” of such KPI becomes a burden for organization, which aims to acquire the minimum information required from its clients or suppliers. Requesting too much incurs additional processing costs on the organization and risks alienating the constituent. Asking too little inhibits the organization’s long-term reactive capacity to comply with new legislation or to implement improved processes. Ultimately, data technologists must aim for what Antoine de Saint Exupery defines as perfection, which occurs, “not when there is nothing more to add, but when there is nothing left to take away.”
    At Pullapproach, we are working at the forefront of these challenges. We lower the barriers for data acquisition, making the process easier for both the organization and constituent alike. (See the original post from my colleague Nikon Rasumov.) To accomplish this, we started by designing the complex information infrastructure behind interactions in order to be able to extract, compare, and relate information. In fact, we are currently in the process of finding the perfect balance of information in Japan by creating an open innovation platform to define and evaluate business solutions to local challenges.

    Best wishes and thanks for using this forum to share your insights!

Leave a Comment