There is a growing movement in government and public policy towards transparency. Data associated with trade, traffic, education and crime is being made available to anyone and everyone who wants to use it. The problem is, the data is exactly that: data. Spreadsheets, databases, flat files of numbers and many other data formats are being pushed into the public view as a mechanism to support transparency.

Of course, unless you are amongst the data savvy, making sense of large data files comes at a great cost in time and resources. Having access to all of the traffic and accident data for the country doesn’t help you understand how safe your neighborhood is for drivers. Getting the download of all of the census data from 2010 doesn’t tell you how your city is changing. And, downloading all of the data about schools and the opportunities that they provide doesn’t give you a lot of help in understanding the schools in your district, nor does it offer the bigger picture trends that speak to the condition of education nationally.

Data in its undigested form is rarely useful; we need the stories and insight the data provides. Simply put, we need to go beyond data transparency and get to insight transparency.

Open government data initiatives are to be applauded for embracing a spirit of transparency and accountability. However, without the ability to quickly and accurately report the findings contained within those datasets, these initiatives are of little value.

ProPublica has been addressing this problem by taking existing public policy and government data and transforming it into compelling stories and applications. Using the data to produce national stories based on analysis of the data and specific data inspection applications, ProPublica has been working to make the data that you care about more accessible when you need it.

ProPublica is one of the few organizations of its kind that cares about both storytelling and data interpretation. With this in mind, we partnered with ProPublica to transform some of their data into stories. In particular, we have been working with data related to high schools, the educational opportunities that each provides and the overall economic state of their student body. Using ProPublica’s core data, Quill is producing stories that focus on the educational opportunities at a school, where there are gaps and the conditions that are correlated with them, and how those opportunities compare within the school’s district and state.

Producing tens of the thousands of stories, Quill has written an entire nation of school stories that are relevant to students, teachers, parents and policymakers, no matter what school or district is of interest to them. Directed by both the data and the editorial and analytic guidance of ProPublica, Quill has automatically generated clear, concise narrative descriptions of the educational opportunities (and gaps in those opportunities) for every high school in the country.

A parent interested in the schools available to their own children can now read about those schools and see how they stack up to others in the area, others in the state, or others across the country. These stories go beyond the data to a level of insight that makes them both highly informative and strikingly relevant.

We see this work as a great expression of how automated storytelling can transform the often-difficult world of data into a plainspoken world of insight. We are looking forward to putting Quill to work on other public data sources to achieve the goal of information transparency that we believe is long overdue.

 

Kris Hammond is the Chief Technology Officer of Narrative Science.  Connect with Kris on and Twitter.