So reporters and editors at The Lens — a local, reader-supported online newsroom — decided to create The Vault, a comprehensive database of city contracts, showing exactly how the city government spends its money.
With the help of a grant from Knight Foundation, our newsroom downloaded over 5,400 government PDFs and uploaded them to DocumentCloud -- another project supported by Knight -- which turned them into searchable text detailing the nuts and bolts of city business.
It’s a really big -- and boring -- haystack.
But we’re going to use it in many interesting ways.
Most obviously, if a reader, reporter, neighborhood leader or activist has a hunch or tip, our contracts portal will give them a starting point for their investigation. Searching all New Orleans city contracts for a particular word or phrase is now as easy as searching Google.
This is a huge improvement over the city’s website, where these documents are posted once they’re signed. The city’s site does not let you search through the text of these PDFs — so you can’t just type in a name or address to see what comes up.
You have to know what you’re looking for — a huge obstacle for most people trying to find the answer to a question.
Filing a public records request won’t help much. Like in other states, you can request public records matching certain specific criteria, but you can’t ask public agencies to do your investigating for you.
Perhaps this service will ferret out corruption and graft. Maybe it will help someone figure out what’s going on with long-delayed construction on the neighborhood park. Perhaps government officials will use it to learn how to operate more efficiently.
But we’re also really excited about another use for all these records.
Ben Welsh from the Los Angeles Times has spoken about how reporters often think about data journalism as a rifle. An investigative reporter starts with an idea of what he’s aiming at and uses data to hunt it down.
Instead, Welsh suggests that reporters use computers like those robotic spiders from “Minority Report.” In other words, journalists should program computers to go out and find interesting bits of potential news. Once computers identify a lead, humans can report out the tip.
To that end, in the months to come we plan to pair our contracts database with a database of campaign finance and business records — plus other paper records from other city agencies that we’ll have to gather by hand.
From there, we’ll use use a computer to look for patterns and make connections to help identify any cronyism or unfair bidding.
In computer-speak, it will look something like:for each donor for each elected official for each city contract if something looks fishy, tell a human...
We’re excited to dig into this stack of 5,400 tips — and to help neighborhood leaders, citizen activists and other news outlets start their own investigations.
So if you’re interested in how the city of New Orleans is spending tax dollars, head over to The Vault and see what we’ve done.
If you have ideas on how to improve this service, let me know.