Multimedia is one of three things that make online journalism different from offline journalism, but the other two things — interactivity and user-control — depend largely on journalists understanding data driven journalism. This isn’t about numbers, but about structured data. Here’s a bootcamp that’s intended to introduce journalists to the tools and concepts of structured data and data driven journalism.

As with all classes, this one has some required reading before you start:

• Journalism and the Scientific Tradition (Chapter 1 of “Precision Journalism” by Phil Meyer)
• A fundamental way newspaper sites need to change (Adrian Holovaty)
• What Does a ‘Data Delivery Editor’ Do? (Ken Sands)

I start with Phil Meyer’s “Precision Journalism” book both to illustrate that a lot of what we are talking about is not new. A lot of it is just a way of making computer assisted reporting more visual and more transparent. But the other reason I like to use it is because he does a nice job of explaining the mindset you need to have going in to any data driven story. And mindset is infinitely more important than technical skill.

Phil Meyer writes argues that journalists and scientists share a lot of similar values.

1. Skepticism
2. Openness – Transparency – Replicability
3. Operationalization (what can be observed and described)
4. Tentativeness of “truth” – new facts can be discovered that might change our understanding
5. Parsimony – given then choice between two explanations, favor the simpler.

Data driven journalism is not about creating a data dump on your audience — simply digitizing and organizing a lot of atomic-level facts. Good data driven journalism helps readers understand the data. This means that journalists need to always have a hypothesis in their mind — and it must be one that is relevant to the audience. The data is used to test that hypothesis.

This kind of thinking leads to a nice view of objectivity that works well in the pursuit of accountability journalism amid the cacophony of the Web: Journalists need to be objective going in to their experiment, but conclusive coming out of it. And by remaining transparent during the reporting process, journalists can retain their humility by holding up their reporting to public scrutiny.

What can you do with data driven journalism?

Holovaty argues that a lot of journalism is better done with structured data – or, “information with attributes that are consistent across a domain,” as he defines it — than with narrative stories. Let’s take look at some of the examples that Holovaty and Sands discuss in their pieces. What is the structured data in each? What are the categories of information and what are some of the values found in those categories?

Data can be anything. Addresses, times, crime reports, sports scores, drink specials, votes cast. In short, data IS who, what, when, and where. Sometimes even how and why.

Now, let’s look at some examples of narrative stories in The Daily Tar Heel.

• Campaign Signs Getting Bigger
• Easy to Spread Diseases in Dorm
• Leaking Roofs Plague Chapel Hill Schools

While each of those stories include compelling anecdotes, they also lack the data needed to support the assertions made in their headlines. By practicing data driven journalism, reporters can make their stories more transparent and more relevant to their audience. That ultimately makes the journalists themselves more credible and more valuable. And those are two things that can differentiate your site from every other on the Web.

Meyer outlines six steps that journalists need to take when dealing with data.

1. Collect it – This can be done through observation or acquisition of public records.
2. Store it – Data is often a snapshot in time. But it can be used over and over again.
3. Retreive it – Listing, searching, etc.
4. Analyze it – Are there trends over time? Are there predictors of outcomes?
5. Reduce it – You may have to use statistics, but don’t tell anyone that.
6. Communicate it – More and more often, this means data visualization. Charts, graphs, maps, tag clouds, etc.

I’d like to add a seventh step for online data publishing:

7. Empower the reader to do their own steps 3-5 – make it searchable and sortable so readers can test their own hypothesis. Then, give them a space where they can report back and discuss their findings.

Spreadsheets

Every journalists should know how to use a spreadsheet. The most common spreadsheet is Excel, but you can also use something like Google Docs for free.

The Reporter’s Cookbook provides an excellent four-part Excel training program, complete with sample data files. Microsoft provides some free online tutorials for Excel 2007. Lynda.com also has visual tutorials for its subscribers.

Excel Vocabulary You Should Know Before Moving On

• Worksheet
• Rows (records)
• Columns (fields)
• Cell

Basic Excel Tasks You Should Master Before Moving On

• Enter data (by hand, acquire, scrape)
• Delete data/Clear data
• Change column width
• Formatting cells
• Insert row
• Insert column
• Sorting
• Filtering
• Using functions
• Using Formulas

Now that you’re familiar with Excel, let’s try our hands at doing some basic journalism with a real dataset. Step 1, Meyer says, is to collect data. So let’s go get the 2008 election returns from the N.C. State Board of Elections.

You’ll want to download summary.csv (which actually will arrive in a compressed zip format. More on decompressing files here.) CSV stands for “comma separated values.”

Now that we have this open, let’s see if we can use some basic spreadsheet functions to answer questions that people might have about this data. For example:

• Did more people vote in the U.S. Senate race or the gubernatorial race? (Use the SUM function)
• Double check the state’s data. (Use Multiply and Divide to check the raw vote totals and vote percentages)
• What was the average presidential vote per county? (Divide total votes by number of counties)
• Which House candidate received the most votes? Who received the least? (video on sorting in Excel)
• Show just the Republican candidates. (Using filter.)

You should also be sure you understand how to repeat a formula and the difference between absolute and relative cell references in formulas.

Putting Data on the Web

Now that you are beginning to see some of the cool things you can do when you break journalism down in to structured data, it’s time to start thinking about getting that data up on the Web for others to use and see.

Probably the simplest way is to use Google Docs. With the spreadsheets you can easily publish the data as a Web page, or you can give people access to sort and filter the information and even edit it if you’d like.

One of the coolest things you can do with Google Docs is create an online form (tutorial video) that allows your audience to add information to the spreadsheet. These forms can be embedded in any Web page even if you don’t know anything about HTML or programming. Obviously, you wouldn’t want readers changing the number of votes in your election results spreadsheet, but maybe you could allow high school coaches password-protected access to update their game scores. Or restaurant owners could update their weekly specials or seasonal hours.

To see Google Docs (including a form) in action, visit the bottom of Letsbuyanewspaper.com.

Tabelizer is also another simple tool for creating HTML tables out of spreadsheet information. It’s pretty self-explanatory.

Data Visualization

Another cool tool for beginning to visualize data is Swivel. As the site itself says, it’s still a little rough around the edges, but check out some examples of what you can do with it.

Speaking of data visualization, everyone loves an interactive map. The Knight Digital Media Center has a good set of tutorials on how to get your spreadsheet information displayed on a Google map. (Andrew Dunn also has a nice seven-step tutorial for simple use of Google maps.)

Dippity is a fairly simple tool for creating very nice looking time lines.

If you’re feeling really frisky and want to play with a little (very little) Javascript, check out the Simile project to do time lines, maps and other data presentations.

Finally, if you’ve decided at this point that you’re still a word person, you can always satisfy your data visualization needs with a little tag cloud. Shoot, The New York Times does it…

Databases

But Professor, you say, you’re just talking about spreadsheets. I want to do databases. Sure thing, the first thing you need to understand the primary difference between a database and a spreadsheet.

Probably the easiest way to think about the difference is to think about a database as a collection of spreadsheets. These spread sheets are all related to each other, so they’re called relational databases. J-Lab has a nice explanation of relational databases.

But to do very cool things with databases, you need to know some flavor of SQL and a scripting language. Or at least learn Access.

There is one non-programming option that has become popular among journalists who want to deploy online databases. Caspio is an online service that allows you to upload your databases or spreadsheets to their servers and create some very simple Web forms. The lowest cost for the service, though, is $480 a year. Non-profits and students who are willing to pre-pay a year can get the price down to $336 a year. Not cheap, but cheaper than hiring a programmer.

Let me end with a final word about why I’m a strong believer in the value of data driven journalism.

First, it adheres to a notion of objectivity that goes beyond “balance.” Its transparency holds journalists accountable and the accuracy it demands should improve our credibility.

Second, online data driven journalism makes our work more relevant to the audience. It helps get the right information to the right people at the right time with a high level of efficiency.

Third, it is the foundation of something I’m calling sustainable journalism — the idea that you can unearth information once and reuse it many times. Journalists who store and present their reporting as structured data can build deep resources of commodity information. On top of those resources, they can spend more time digging up new information and doing the kind of analysis and trend-spotting that will differentiate professionals from amateurs. Probably the two most prominent examples of this are The New York Times Topics and The Washington Post’s Votes Database that those news organizations incorporate via links in to almost every story.

Looking for More?

• Statistics Every Writer Should Know (I know, I know. I’m not a numbers guy either…)
• The Reporters Cookbook (Good list of introductory lessons to spreadsheets, databases and other tools of computer assisted reporting)
• Syllabus of SMPA 130 at The George Washington University (Makes me wonder why I’m re-inventing the wheel here when The New York Times’s Derek Willis so kindly posts his excellent class right online.)
• Computer Based Training classes (UNC faculty and staff only)
  • Introduction to Programming
  • Basics of XML Programming
  • SAMS Teach Yourself PHP, MySQL and Apache All in One

Ms. Clark died on Saturday. But the Triangle’s newspapers should ask not for whom the bell tolls. It tolls for thee.

But it also tolls for online community sites if papers begin to kick the bucket.

The News & Observer broke the story, which piqued my curiosity about an apparently fascinating woman I didn’t know. I Tweeted that I wanted to know more about her. And Ruby Sinriech Tweeted right back that a thread of comments had been posted on the front page of her site, OrangePolitics.org.

And, sure enough, eight people — including a former town council member, the news director at the local radio station and the editor of the up-and-coming Carrboro Citizen — posted memories there.

Meanwhile, here’s a rundown of reader memories on local media sites:

News & Observer – Chapel Hill News editor Mark Schultz made a plea for memories on his blog. The N&O got one.

The Herald-Sun – A story, but no comments.

WCHL – A story, but no ability to comment on stories. Which explains why Joe Schwartz went to OrangePolitics.org to leave his.

Chapel Hill News – A story, but no ability to comment on stories.

Carrboro Citizen – The ability to comment… but no story. Although editor Kirk Ross also posted on OrangePolitics.org.

I can’t rightly explain what’s happening here. Structurally, there’s not much difference between the comments on OrangePolitics and the other sites. OrangePolitics is the only site that allows anonymous comments, but those are held for moderation. The registration processes for all the sites are about the same, although the MSM sites pry for a bit more personal contact information. All sites post the comments of registered users immediately (including my single-vowel post on the N&O that I can’t seem to remove.) So it’s not a technical/structural issue.

Clearly the leaders of local media “get it.” Three leaders of local news organizations were online adding and/or seeking comment. So it’s not a philosophical issue.

So what is it? Why did the story get reported in one place, but discussed in another? Is it that OrangePolitics.org is almost totally user-generated — that conversation is woven in to that community’s information gathering process?

What would the community have lost without the newspaper reports? Finding a vacuum, would the news have broken elsewhere and still elicited comments online?

Also, it’s not as if the newspaper stories didn’t include community memories of Ms. Clark — the N&O had two sources and the Herald-Sun had five, including two of the state’s most prominent politicians. But, oddly, none of the voices on OrangePolitics.org were included in the news stories and none of the voices in the news stories were found on OrangePolitics.org.

Is this part of some new media symbiosis that brings new voices to the table while retaining the old? That merges a reporter’s initiative with a community’s contributions?

I’m not convinced. I think that kind of symbiosis would require more interaction between the two. And right now they don’t touch or acknowledge each other in any meaningful way. It is, to steal from some of my newly acquired parenting jargon, “co-playing.”

But, honestly, I’m stumped here. And I’m supposed to be giving a lecture on this kind of thing in a few weeks… I hope Dan Gillmor’s right…y’all want to help me out here?

P.S. — Oh yeah. One of the best pieces of online journalism I’ve seen recently came from Andrew Dunn, the University editor at The Daily Tar Heel. He posted it to OrangePolitics.org…

1. Copyeditors. As Andria points out, automation is great for the low-hanging fruit of stenographic journalism. But that automation needs verification. Copyeditors need to invent new workflows for quality assurance on vast amounts of data. When dealing with major metro crime reports or national election returns, copyeditors will no longer be able to dot every i and cross every t. They will have to do QA by sampling. They will need to understand algorithmic thinking and content management system idiosyncrasies in order to do this well.

2. Pattern Analyzers. Yes, Andrew, campaign contributions to N.C. Gov.-elect Bev Perdue appear to be more geographically scattered than those to Charlotte Mayor Pat McCrory. But so what? Much of this data is freely available to every citizen. Professional journalists earn their paychecks by giving data meaning.

3. Just plain annoying. Journalists also earn their keep by asking for information that isn’t offered, by making public important information that powerful people want to hide. Journalists are the professional guardians of open government that free societies need so everyone else can spend their time more productively. Call me a bad citizen or a good capitalist, but I want journalists to watch government so I don’t have to.

And as annoying as computers can be, they will never be as annoying as humans.

They say: “We plan to highlight innovation, chronicle setbacks and analyze the business side of college newspapers. We also plan to look at how college journalists are lending their voices to the big stories of the day.”

A good idea, methinks. Congrats, Sara and Andrew. You and your blog subjects are the Future of News.

And I’d be remiss if I didn’t also give a shout out to a great new multimedia site just launched by Tar Heel students. Andaman Rising is a collection of 15 student-produced multimedia stories about life in southern Thailand after the 2004 Asian tsunami. This kind of excellent multimedia documentary work is becoming so reliable here in Chapel Hill that I take it for granted. Congrats students. You, too, are the Future of News.