Under what conditions would you volunteer to help a project like this in your community? News organization — how much would you pay for a service like this? What characteristics would it need to have to make it worth your money? What else do you see here that needs further clarification?

(And a big hat-tip here to Penny Abernathy, the Knight Chair in Digital Media Economics here at the UNC-Chapel Hill School of Journalism and Mass Communication. She got this project kicked off with a grant from the McCormick Foundation and who is my co-pilot on this application.)

Here’s our draft pitch:

Crowdsourcing Data to Bring OpenBlock to Rural America

This project would create a co-op to develop and deploy public records databases at news organizations, especially those serving communities of fewer than 75,000 people, preparing those records for presentation and integration in an OpenBlock format.

These rural news organizations are struggling to move to the digital age in part because their staffs are so small they don’t have the capacity to identify, digitize, re-aggregate and map all the various public records available at the state and local level into databases that can be accessed intelligently by both reporters and the reading public.

The project would tackle the lack of capacity at rural papers from two directions. It would create a centralized repository of state, county and city schemas and datafeeds that could be easily used in OpenBlock. This a job well-suited for a small group of experts. In addition, the project will create a statewide corps of amateur data-checkers and records requesters. Data quality assurance and data gathering are jobs well-suited for a crowd of many people, each working on a small piece of the puzzle.

These volunteer citizen-journalists would actually be member-owners of a co-op business. Each task they perform would earn them additional shares in the company’s annual profits. We would generate revenue by charging rural newspapers a fee. The more records and the better their accuracy, the more news organizations would sign on for the service.

In some cases, volunteers would pick up CDs of data from county offices. In others volunteers would scan and upload PDFs of hand-written police incident reports. In still other cases, people would key into a database the information on those PDFs. This job is so big that no single small news organization could do it. But with a corps of member-owners working together, we could create a model for gathering valuable public records from rural America. To individual communities, these records are necessary to foster an informed civic dialog and healthy economy. But in aggregate, these records may also be able to shed light on trends in rural America that would otherwise go unreported.

Improving Delivery of News and Information to Geographic Communities

In small towns and rural America, the local newspaper is more than just a source of information and an engine of commerce.  More importantly, it fosters and builds geographic community and sets the agenda for public policy debate.  This project will foster civic and community engagement — first, by forming a network of knowledgeable volunteer citizen-journalists, and also, by making public records readily available and organized to support decision-making and accountability at all levels of government.

Unmet Needs

In many cases, data that is readily available in GeoRSS or at least CSV format from big cities (such as this example from San Francisco) is simply not available even in print from rural governments. For example, journalism students at the University of North Carolina working last semester to gather and organize public records in two rural counties for an OpenBlock application met with a number of obstacles (which they describe in their blogs) – ranging from significant photocopying fees to inappropriate redactions and denial of access to public information.

Even when acquisition of public datasets is relatively simple – for example, public health restaurant inspections — someone must request that data from a specific county be exported in fielded data format. It is inefficient for each rural news organization to make separate requests for this data in each of North Carolina’s 100  counties. In these cases, our public records coop would outline an initial request for the data for each county.

What’s New?

Currently there is no tool or service that can efficiently gather, format and publish public records on rural news organizations’ sites. In part, this is a technology problem that may soon be overcome with the alpha rollout of OpenBlock later in 2011. But a much bigger piece of the problem is the data itself – neither OpenBlock nor any other technology has the ability to obtain public records as fielded digital data and create a newsworthy user interface for all the various types of records a news organization might need.

Without a project like this there is no indication that OpenBlock will be a viable option for papers in rural communities.

What Will Change?

By the end of the project, we will have

•          at least one member-owner in each county in North Carolina

•          at least 12 news organizations subscribing to the service

•          at least one type of schema for which we’ve collected data from each county

Most importantly, we will have raised public awareness of open government and we will start seeing rural counties and towns publish public data in standardized, machine-readable formats on the Web.

What tasks/benchmarks need to be accomplished to develop your project and by when will you complete them?

How will you measure progress?

Do you see any risk in the development of your project?

How will people learn about what you are doing?

Is this a one-time experiment or do you think it will continue after the grant?

But I also envied Drescher. He was also the only one at the table who had ever dropped $200,000 of his company’s money on an investigation of a state agency. And the only one who knew what it was like to spend four years pinging the government for public records before he had a story solid enough to sell to his subscribers and advertisers.

One other thing made Drescher an enviable character in the Triangle’s media ecosystem. Despite their valid criticisms of increasing gaps in The News & Observer’s coverage of our communities many noted without irony in their voices, the small, independent and non-profit news operations had the most impact on public policy when they got the attention of Drescher’s paper or one of the local television stations.

And that made me realize that if our state is going to retain its generation-long reputation as a home for journalism that gives voice to the voiceless and holds powerful people accountable, then we must find a way to foster dozens of new and diverse tributaries of news and information that flow into the big, slow-moving mainstream media. Without the tributaries, the MSM seems likely to evaporate entirely. Without a larger channel into which they can empty, the tributaries seem likely to overwhelm us with a flood of disconnected datapoints.

Drescher and the rest of us at the meeting had been brought together by Fiona Morgan, a Duke graduate student, former reporter for The Independent and author of an excellent new study about media in the Triangle. The report is one of several local media ecology studies produced by the New America Foundation‘s Media Policy Initiative. The meeting was hosted by the Triangle Community Foundation. Two quotes in the report struck me as bookends that support my belief that North Carolina needs both professional and amateur journalists in order to build a sustainable news ecosystem in the age of digital, networked media.

For a traditional journalist like me, Ruby Sinreich is the model blogger. Her site, which deals with Orange County politics, is unabashedly left-leaning but also deeply respectful of the facts and of diverse views. Sinreich breaks news and fosters conversation that far surpasses anything found elsewhere in the media glut of Chapel Hill and Carrboro. As Morgan’s report chronicles, Sinreich maintains her blog as a hobby. The site costs $240 a year to publish, but the labor she puts into it is probably worth $25,000 to $40,000 a year. Considering Sinreich also has a day job and a small kid, that’s an amazing contribution to the community. But it’s also a precarious contribution. Sinreich’s blog is not a sustainable business and she told Morgan that she wasn’t sure she wanted it to be.

“It’s a perennial question whether to take advertising,” Sinreich is quoted as saying. “If I did, I feel I would have to be more accountable to the readers of the site and do more research on my posts, rather than writing about whatever interests me. If there were an important meeting coming up, I’d feel like, ‘I don’t want to go, but I have to go cover it.’ … I have a job, and it pays much more than BlogAds.”

So how much, exactly, can you make selling ads on a local blog? Well, Durham blog Bull City Rising brings in $50 to $400 a month. Cary Politics aims to raise $100 a month in donations.  And the entertainment-heavy New Raleigh blog has a self-reported annual revenue of “the low six figures.”

The revenue for these new media startups is dwarfed by even the most troubled mainstream media outlets in the region. Compare the blogs’ revenue to the Triangle’s most established “alternative” newspaper, The Independent. In a good year, The Independent brings in $3 million, its president, Steve Schewel, said last week.

Incumbent media outlets also dwarf the websites’ in terms of audience. According to Morgan’s report, the blogs get between 2,000 and 40,000 people to their sites each month. The N&O and WRAL draw a few million to their sites.

So we shouldn’t be surprised when even the most altruistic amateur journalists are driven more by their political interests than their financial interests. Money can keep a reporter on the beat even when there’s something else she’d rather be doing. Money keeps the watchdog on the lookout even while the rest of us enjoy our daily lives.

But, of course, money’s influencing power can also color the media’s reporting of the facts. As Morgan notes, UNC-TV gets more than half of its money from the state legislature. Recently, that’s created a conflict and public debate about whether the state’s open records law trumps the state’s “shield law” that is intended to keep reporter’s notes private and from being co-opted as a compulsory investigative force for the government. As it played out, UNC-TV management opted to act as a state agency.

In the media business, as in politics, many small streams of revenue seem more likely to dilute the ability for one large stream to exercise undue influence. And also just as in politics, incumbency seems to have its advantages in the media business.

The similarities between politics and the news media don’t end with those two examples. It was another quote in Morgan’s report – this one from Jock Lauterer – my colleague at the UNC School of Journalism and Mass Communication – that shows just how much the MSM needs the blogosphere as well. Lauterer is director of the Carolina Community Media Project, a co-producer of the Northeast Central Durham Community VOICE newspaper.

“The whole community organizing dynamic is what newspaper publishers and editors have not been good at, or haven’t had to do before. But if we’re not partnering, we’re dead. We have to be constantly seeking out new partners to be sustainable,” he told Morgan for her report.

Now I don’t know how many newspaper editors and publishers you know, but most of the ones I’ve met are allergic to the idea of organizing or being organized. It runs against the fierce sense of independence that allows the best of their investigations to be unmoved by popular sentiment. The mainstream news editors I know have grown accustomed to being harangued by often loyal but sometimes deeply disturbed members of their audience. Inviting them any closer into the journalistic process seems a threat not just to their professional integrity but their personal wellbeing.

So that’s another way that mainstream journalists are much like political candidates. The difference is that any political operation worth its salt knows that you never turn away any willing supporter – even the crazy ones. Not every volunteer is going to get access to the strategy meetings, but good campaigns finds some small way that any potential friend can contribute to the cause.

Professional reporters, whose ranks are being mowed down by an unsustainable business model that relies on corralling an increasingly dispersed and disaggregated audience, need to organize their audience in order to lower the cost of reporting.

The Triangle doesn’t necessarily need more professional journalists. What it needs is more people who think journalistically. We need more people who can not just describe what is seen, but who are curious about what we might not be seeing. We need more people who are less interested in what they can make people think and more interested in showing them how we know what we know. Curiosity and verification are the core tenants of journalistic thinking that I teach my newswriting students, and we need to find a way to hone those instincts among all North Carolinians. At the very least, they’ll become better consumers of news and there’s a chance that some might even become better producers of news.

But organizing a community takes more than just will. It takes a plan, and there are a few actions we can take right now if we want to strengthen journalistic thinking in the state and lower the barriers to entry for new media entrepreneurs:

1. Every public record that is produced after today at every level of government should be made available quickly online in a format that can be easily digested by computer programs. These programs can lower the cost of detecting early trends or newsworthy oddities by automatically combing fields of data.
2. Every incumbent news organization should hire at least one person to actively cultivate online community. Like broken windows in an abandoned urban neighborhood, the uninformed anger and irrelevant rants of article comments on news websites have given visitors the impression that it’s OK to behave badly in those places. Site publishers have to be active and engaged hosts who reward good behavior and punish bad. Similarly, blogs and social media have too long been on “the wrong side of the tracks” from the places where traditional reporters look for stories. The mainstream media needs to help anyone in their community that is already blogging on current events learn how to dig deeper into a story by requesting public records, identifying larger trends, and verifying everything they see and hear.
3. Community foundations should establish and fund a volunteer program similar to AmeriCorps that financially supports recent college graduates who want to spend two years reporting news from communities that have no professional reporters dedicated to them.
4. Journalistic thinking and digital publishing should become a part of any civic leadership or volunteer training effort. Morgan’s report notes the importance of “neighborhood colleges” and public access television in helping to develop community leaders and media producers. Each of those programs should add journalistic thinking and digital publishing to their agendas.
5. Leaders of the high-tech industry in the Triangle should organize public-interest “code camps” during which computer programmers spend intensive weekends focused on developing free and open-source digital tools that can be used by professional and amateur journalists.
6. Media literacy should be added to the state middle and high school curricula. Journalistic thinking can help North Carolina’s students learn about writing, math, and the scientific method. Teaching them to produce digital media can increase the fluency with information technology that will help them find jobs and develop the state’s rural and urban economies.

Morgan’s report and her follow-up conference last week make it absolutely clear that the Triangle has an unusual diversity of journalistic species. But what’s lacking is a symbiotic ecosystem that all of them need to survive. Whatever you think about the MSM or about pajama-clad bloggers, you can’t help but realize when reading Morgan’s report that in our current Darwinian media landscape, the creature most likely to become extinct is a free, fair and factual public discourse that holds powerful people accountable, shines light in dark places and gives voice to the voiceless.

But you know what happens to you and me when you assume things, so I checked it out. Turns out I was mostly right. I’m going to put together a summary of information that got reported about this race, but it got me wondering about this question: How much information – and what kind – of information do North Carolinians need about downballot statewide primary races? Are they getting? From where? Or why not?

After all, if journalism’s worth saving it’s only because of the impact it has on public life. I’ve long been curious about the connection between information and citizen participation. The presumption – not always right, as Samuel Popkin and Michael Schudson might tell you – is that the more information voters have the “better decisions” they will make.

A little more than 700,000 people voted in those races. Some of them might have wanted more information than others? How many had enough? How many would have changed their votes if they had had different information?

And, if we can figure out who needs this information – and what information they need – is there any business model that gets it to them? Do we need independent reporting on downballot races like this or is informing voters the job of the State Board of Elections and the candidates themselves?

Facebook groups are ripe for the harvesting

Reading up on the issue, it seemed that people tackled the issue in one of two ways — either as a trailing indicator with roots in pre-kindergarten or as a leading indicator of difficulties that a person will have throughout his or her life staying out of jail, holding down a job, and maintaining a family.

So we run a real danger of trying to wrap our arms around a topic that seems to be correlated to lifelong problems that begin at birth persist throughout life.

On Monday, we’re hosting our newspaper partners in Chapel Hill. We’ll find out then how they see the issue playing out in their communities. But as I educate myself on the topic and have been discussing it this week with students, here are some of the questions I have.

My question to you: What would you like to know about North Carolina’s diploma dilemma? How would you like to see us cover the issue. I welcome your comments.

• There are 1.2 million North Carolinians who don’t have a high school diploma. Who are they? Where are they? What do their lives look like?
• 23,550 North Carolinian kids dropped out of school last year. Who are they? Where are they? Why did they drop out? Where will they end up?
• The districts with the highest dropout rate tend to be concetrated in the rural mountains or in the rural east. What effect does a rising dropout rate have on most North Carolians – the 90 percent who do have a diploma? Is this a problem of public policy or individual achievement and opportunity? Is there a correlation between
  - teacher salaries and dropout rates in districts?
  - tax rates?
  - per student spending?
  - income level of parents?
  - education attainment of parents?
• Starting in 2003, there was a huge jump in the number of dropouts that were attributed to students seeking further education at community colleges. Why the sudden increase? Was there a policy decision that prompted it? Are these students ultimately more successful in life than students who drop out because of work, family, drug or discipline reasons?
• About 5 percent of the state’s high school students dropout every year. For Native American’s it’s 7.7 percent. For Hispanics, it’s 7.6 percent. For blacks, it’s 6.16 percent and rising. How do social and cultural issues surrounding race and ethnicity effect the causes and potential solutions?
• The state legislature is giving away $7 million to 60 efforts across the state aimed at reducing the dropout rate. The state Department of Public Instruction has launched a PR campaign to raise awareness of the issue. What’s working? What’s not? How do you measure? Where is the best return on these investment?
  

But this challenge isn’t unique to political leaders, it’s also one that journalists must meet and a gesture on which they are following through even less.

Hooked on the promise of the free advertising inventory generated by online comments, more and more newspaper Web sites are deploying  some type of online discussion technology.  What they aren’t deploying is the kind of human  resources that are needed to foster and develop online conversations. Why do most comments on news articles follow Godwin’s Law? Because there is little or no authentic conversational leaders. There is no human being making connections between people and ideas and, um, fact.

Just look at this recent survey of online journalists in North Carolina — online community management ranked as the skill that these editorial staffers said was least important to their jobs.

Here are my quick thoughts on how news organizations should begin to approach online comments.

But what’s he talking about? Page views? Unique visitors?

What parts of the site are busiest? Fundraising? Issue briefings?

How are people finding his site? Google “earned search”? Online ads? Media coverage?