Big data roundup

Update, March 4: Here's an article in the NY Times about an exciting use of integrated big data, a series of programs IBM designed for Rio de Janeiro.

There has been a series of articles in the past couple of years about the era of big data we've arrived in, including today's NY Times article about Facebook's efforts to manage its stream ("firehose" is the term the NY Times used) of user data in the face of privacy concerns. McKinsey recently compiled a chart showing the potential some sectors of the US economy have for using big data:

Source: McKinsey and Company

Not-for-profits that provide social services, including mental health, health, or other social services have been collecting large amounts of data for years, but many have had trouble unlocking its potential for any number of reasons. Yet analysis of large amounts of data can allow providers to test interventions and make better management decisions, improving productivity and services.

The McKinsey Global Institute analyzed data in five domains, including health care in the US and the public sector in Europe, and concluded that big data can generate value in each. These conclusions seem easily extendable to the not-for-profit sector in the US, and I recommend reading McKinsey's full report or at least the executive summary, both available free, after registration here. A shorter version of the executive summary, focusing on strategy, is available as an article here. The basic points are:

• the era of big data is here, allowing providers to collect data across units, integrate information, and analyze the information;
• big data can change the way you do business, making processes and information transparent;
• you can use data to experiment and test hypotheses;
• privacy is already a concern;
• managers will need to understand big data, and will need specialists who can provide analytic support.

 These are big questions. But it's not too early to start thinking about them.