The New 
Precision Journalism

by Philip Meyer

 
Download Chapter 1
Word
 

Index button

Chapter 1 - Journalism and the scientific tradition
Chapter 2 - Some elements of data analysis
 
 
 

Chapter 3 - Harnessing the power of statistics
 
 
 
 

Chapter 4 - Computers

Chapter 5 - Surveys
 
 
 
 

Chapter 6 - More about data analysis
 
 
 
 

Chapter 7 - Field experiments
 
 

Chapter 8 - Databases


 
 
Chapter 9 - How to do an election survey
 
 
 

Chapter 10 - The politics of precision journalism


Journalism and the Scientific Tradition

If you are a journalist, or thinking of becoming one, you may have already noticed this: They are raising the ante on what it takes to be a journalist.

There was a time when all it took was a dedication to truth, plenty of energy, and some talent for writing. You still need those things, but they are no longer sufficient. The world has become so complicated, the growth of available information so explosive, that the journalist needs to be a filter as well as a transmitter, an organizer and interpreter as well as one who gathers and delivers facts. In addition to knowing how to get information into print or on the air, he or she also must know how to get it into the receiver's head. In short, a journalist has to be a database manager, a data processor, and a data analyst.

It takes special training. In the good old days, there was serious doubt in professional circles about whether journalism, as a discipline, included any body of knowledge at all. Journalism, in this view, is all procedure, not substance.1When James Bryant Conant had to deal with a windfall bequest to Harvard to "improve the standards of journalism," he chose substantive training for mid-career journalists. "Mr. Conant felt that there was not a sufficient knowledge base to justify a journalism school . . . this conclusion led to a remarkably successful program which we have no desire to alter," Harvard president Derek Bok recalled fifty years later.2

In a simpler world, journalism programs that ignored journalism might have been justified. In the information society, the needs are more complex. Read any of the popular journals of media criticism and you will find a long litany of repeated complaints about modern journalism. It misses important stories, is too dependent on press releases, is easily manipulated by politicians and special interests, and does not communicate what it does know in an effective manner. All of these complaints are justified. Their cause is not so much a lack of energy or talent or dedication to truth, as the critics sometimes imply, but a simple lag in the application of information science -- a body of knowledge -- to the daunting problems of reporting the news in a time of information overload.

Harvard's Nieman program, which gives selected mid-career journalists an academic year to repair whatever educational gaps they perceive, is used by some to broaden their scope as generalists. But more and more are using it to adapt to the new demands by becoming more specialized. In a world where the amount of information is doubling every five years,3it takes a specialist to understand, let alone communicate, very much of it. The journalistic body of knowledge, therefore, must include these elements:

1. How to find information;

2. How to evaluate and analyze it;

3. How to communicate it in a way that will pierce the babble of information overload and reach the people who need and want it.

To the extent that journalists learn how to do these things, they are meeting one of the elements of a profession: access to an esoteric body of knowledge. We are learning, and journalism is becoming more professionalized, but, as with any sweeping change, it is proceeding unsteadily and at different rates in different places.

 

Where craft meets theory

In journalism schools, the concept of precision journalism -- the application of social and behavioral science research methods to the practice of journalism -- found a ready market. The ready acceptance of this concept in academe was due in part to its contribution to the healing of the breach between the green eyeshade and chi-square factions. It demonstrated the applicability of social science research methods to the very real problems of newsgathering in an increasingly complex society. It produced work that both the researchers and the craft people could appreciate. The tools of sampling, computer analysis, and statistical inference increased the traditional power of the reporter without changing the nature of his or her mission -- to find the facts, to understand them, and to explain them without wasting time.

In the profession, however, the barriers were greater. Precision journalism threatened the twin traditions of journalistic passivity and journalistic innocence. The former tradition holds that media should report news, not make news. Media involvement in public opinion polling has been criticized on the ground that the media should not do polls but should wait passively until other people do them and then report on them.4Media polls also violate the innocence rule. A reporter should be a person who casts a fresh eye on everything, something that he or she cannot do if burdened by too much specialized knowledge. A journalist, Vermont Royster told the 1967 class of Nieman Fellows, should be "a professional amateur." The extreme case is a foreign correspondent I once knew who laughed when I asked him if he was learning the language of the country to which he was assigned. In his view it was not necessary, might even be a hindrance. His readers did not know the language, and his job was merely to observe what they would observe if they were there and report on what he saw. If he learned a foreign language he might start to think like a foreigner and lose touch with those readers.

The trouble with being a passive and innocent journalist is that, like any passive and innocent person, one can be too easily taken advantage of. The underlying theme in most modern criticism of journalism is that the media are too easily dominated by powerful politicians and their skillful "spin doctors" whose desires too easily determine what is defined as news and what is not. To defend against being manipulated, the media need more self-confidence, and the best route to self-confidence is through knowledge. Media polls proliferated in the 1980s precisely because the editors no longer trusted the polls that politicians tried to give them and armed themselves with their own data-collection operations out of self-defense. Thus polling became not so much a way to make news as an enhanced tool of the newsgathering process itself -- provided, of course, that journalists are in charge of the entire process, from conception, through research design, to analysis and interpretation. The precision journalist does not hire a pollster to create a news event; the journalist becomes the pollster. Jim Norman of USA Today, Rich Morin of the WashingtonPost, and I. A. (Bud) Lewis of the Los Angeles Times were among the prototypes.
 

Beyond objectivity

The model of the journalist as passive innocent had at least one virtue: it provided a kind of discipline. It was consistent with the tradition of objectivity, a tradition that keeps the journalist from imposing personal viewpoints on the readers. But the objectivity model was designed for a simpler world, one where unadorned facts can speak for themselves. Frustration over the unmet ideal of objectivity led some of the media in the 1960s to embrace something called the "new journalism," which freed journalists from the constraints of objectivity by granting them artistic license to become storytellers. Jimmy Breslin and Tom Wolfe were among the first successful practitioners, and their methods worked until they reached a point where they stopped being journalism. The literary tools of fiction, including exquisite detail, interior monologue (what a newsworthy person is thinking as well as his or her overt behavior), and short-story structure, with a character, a problem, and a resolution in a short span of words, can yield journalistic products that are a joy to read. Unfortunately, the data-collection process is extremely difficult. To make life appear to be a short story, one has to be extremely selective, and that requires gathering a very large number of facts from which to select. The constraints of daily journalism will not support that level of effort day after day. Some practitioners of the "new journalism" took to making up their facts in order to keep up with the deadline pressures. Others stopped short of making things up, but combined facts from different cases to write composite portrayals of reality that they passed off as real cases.5

Despite the problems, the new nonfiction remains an interesting effort at coping with information complexity and finding a way to communicate essential truth. It pushes journalism toward art. Its problem is that journalism requires discipline, and the discipline of art may not be the most appropriate kind. A better solution is to push journalism toward science, incorporating both the powerful data-gathering and analysis tools of science and its disciplined search for verifiable truth.

This is not a new idea. Walter Lippmann noted seventy years ago that journalism depends on the availability of objectifiable fact. "The more points, then, at which any happening can be fixed, objectified, measured, named, the more points there are at which news can occur."6  Scientific method offers a way to make happenings objectified, measured, and named.
 

Journalism as science

You might think that scientists would object to such an effort as an absurd pretension on the part of journalists. Not so. The first publication of Precision Journalism in 1973 was made possible by a foundation devoted to the support of the social sciences. In 1989, physicist Lawrence Cranberg argued that "journalism itself is a science, and . . . a properly qualified, responsible journalist is a practicing scientist." Both scientists and journalists, he said, "march to the same orders and serve the common need of mankind for shared knowledge and understanding."7

If journalists would only admit that shared responsibility, Cranberg said, it would clear the way for better training in investigative methods, less tolerance for superstition (astrology was a dead issue until newspapers revived it after World War I), and a commitment to a more rational and peaceful social order. Journalists are so steeped in the ideal of objectivity and open-mindedness that any kind of statement of social goals is likely to make us uneasy, even when the goals are as benign as reducing superstition and building a more rational society. But the plea for more powerful tools of investigation resonates well with us. Fact-finding ability is, after all, the source of what power we possess.

Yet another benefit of a scientific approach to journalism, not mentioned by Cranberg, ought to sit well with most of us. The information sciences are now sufficiently developed so that we can give some systematic attention to the forms of the messages we prepare and the best ways to compose and send them so that they will be received and understood. This body of knowledge enables us to find out how to get messages into the heads of the audience, not just their hands. Courses in the processes and effects of mass communication had become standard in the better journalism schools by the 1960s. And by the 1980s, leaders in industry could see the need for getting down to such basics. James K. Batten, chief executive officer of Knight-Ridder Inc. told the story of an otherwise promising young journalist who failed to get hired at the prestigious newspaper group after he disdainfully declared in the job interview that he wrote to please himself, not the readers.

"Over the years, we've all hired people like him -- and made our newspapers less reader-centered in the process," Batten said. "The time has come to stop. And our allies in the colleges and universities who educate young journalists need to understand that truth -- and send us people with an eagerness to reach out to readers."8

The new precision journalism is scientific journalism. (In France, the term "precision journalism" has been translated as "le journalisme scientifique.") It means treating journalism as if it were a science, adopting scientific method, scientific objectivity, and scientific ideals to the entire process of mass communication. If that sounds absurdly pretentious, remember that science itself is restrained about its achievements and its possibilities and has its own sanctions against pretension. "There are always hidden facts, and truths are elusive in every domain of human inquiry," Cranberg the physicist wrote. "What is needed to discover hidden facts and arrive at elusive truths is a skilled determination to get at them, and that is a determination that is as appropriate to a properly trained journalist as to a properly trained physicist."

Starting in the 1970s, journalism began moving toward a more scientific stance along two separate paths. The increasing availability of computers made large bodies of data available to journalists in a way that was not possible before. And in the business office, the failure of newspaper circulation to keep up with the growth in number of households made publishers pay more systematic attention to the marketplace and the factors that motivated readers to spend time and money with the publishers' products. The notion that a newspaper is a product and that a reader is a rational creature who makes a choice about whether to pay the cost of using the product became respectable. And so market forces were pushing journalism as a whole, not just a few isolated players in the field, to a more scientific stance.

Progress always has its detractors and nonbelievers. As recently as 1989, at a summit conference of newspaper people bent on discovering the causes of readership loss, an editor of a metropolitan newspaper still publicly denounced the main finding of two decades of readership surveys: that the most frequent reason for not reading the paper is lack of time. "I don't buy it," he said, arguing that if newspapers would just do a better job of providing their fundamental services, like getting the paper delivered on time, things would be better.9 His was not an isolated view, and a lot of research reports have been written in the effort to explain what "no time to read" really means. Christine Urban, speaking to the American Newspaper Publishers Association in 1986, had the best answer. What the readers are telling us, she said, is "Watch my lips -- I don't have the time to read the paper every day."10  In the hustle and babble of the information age, the cost in time of extracting information from a journalist's report has to be one of the considerations in the preparation of that report. USA Today's contribution in the 1980s was that it showed how a newspaper could be positioned as a saver of time. It did this by editing and formatting the newspaper with such care and precision that it could meet the reader's need for surveillance -- scanning the world for personal dangers and opportunities -- with a minimum commitment of time. It delivered data that had been extensively -- and expensively -- processed.
 

What to do with data

Knowing what to do with data is the essence of the new precision journalism. The problem may be thought of as having two phases: the input phase, where data are collected and analyzed, and the output phase, where the data are prepared for entry into the reader's mind. This book is mostly about the first phase, but the two are so intertwined that it will also deal to some degree with the output side.

The main goal of what follows will be to tell you how to do these things with data:
 

1. Collect it. Whether or not you ever try to emulate scientists in their data-collection methods, you can profit from knowing some of their tricks. It is always worth remembering, as Professor H. Douglas Price told me at Harvard in the spring of 1967, that "data do not come from the stork."

2. Store it. Old-time journalists store data on stacks of paper on their desks, in corners of their offices, and, if they are really well organized, in clip-files. Computers are better.

3. Retrieve it. The tools of precision journalism can help you retrieve data that you collected and stored yourself, data that someone else stored with a user like you in mind, or data that someone else stored for reasons completely unrelated to your interest, perhaps with no earthly idea that a journalist or public user would ever be interested in it.

4. Analyze it. Journalistic analysis often consists of merely sorting to find and list the interesting deviances. But it can also involve searches for implied causation, for patterns that suggest that different phenomena vary together for interesting reasons.

5. Reduce it. Data reduction has become as important a skill in journalism as data collection. A good news story is defined by what it leaves out as well as what it includes.

6. Communicate it. A report unread or not understood is a report wasted. You can make a philosophical case that, like the sound of a tree falling in the forest, it does not exist at all.


Theoretical models

On a spring morning, running along a misty Chapel Hill street, I topped a low hill and saw in the distance a crouched yellow figure about 18 inches high. It appeared tense, ready to spring, fangs bared, aiming for a point along my path ahead. As I got closer and my eyes were able to resolve the figure more clearly, I saw it for what it really was: an ordinary fireplug. The image of the dog had been the creation of my own brain, imposing an order and a pattern of its own on the ambiguous data relayed by the eyes.

Raw data alone can never be enough. To be useful, to be understood, data have to be processed, abstracted, fit into some kind of structure. You have to put the material into a mental framework that aids in interpretation and understanding. This truism applies equally to the data of everyday perception and to strings of numbers in computers. To think about them at all, you need a perceptual framework. If you don't provide one consciously, then your unconscious mind, perhaps driven by anxiety like that of a jogger in a neighborhood of dog owners, may provide the wrong one for you.

Different writers in different fields have given different names to these perceptual structures.  Psychologists sometimes call them "schema." They are also known as constructs, hypotheses, expectations, organizing principles, frames, scripts, plans, prototypes, or even (this one from psychology) "implicational molecules."11 Walter Lippmann called them "stereotypes."12  In general, he said, "the way we see things is a combination of what is there and what we expected to find. The heavens are not the same to an astronomer as to a pair of lovers . . . the Tahitian belle is a better looking person to her Tahitian suitor than to the readers of the National Geographic Magazine."13  And whether you see a dog or a fireplug depends on what you expect to see.

In its most sophisticated and conscious form, the schema, construct, or stereotype becomes a theoretical model. A formal model describes the essential parts of a process, natural or man-made, in a way that allows conclusions drawn from the model to be tested by experiment or by observation or both. But in most cases, our use of models or constructs is not that sophisticated and not that careful. We use them simply because we need them to think at all. The data of everyday life, left unfitted to any model, take too much effort to process and interpret in the raw. Our senses take in too much of it.
 

Testing the model

The problem with theoretical models --  both the everyday kind and the formal ones of science -- is that we tend to get stuck with them. Two national studies of newspaper credibility published in 1985 and 1986 offer an example. One study (we'll call it Study Y for now) showed that 53 percent of the public believed that news organizations favor one side when presenting news dealing with political and social issues while only 34 percent thought the media are fair to all sides. And by 73 to 21 percent, most thought news organizations often invade peoples' privacy.

The other study (Study X) asked about the same issues with a five-point scale, and it showed that by 52 to 10 percent, more people called newspapers fair than unfair. And the public was about evenly split on the privacy issue: 32 percent thought newspapers invade privacy and 30 percent thought they respected privacy. The two studies were conducted only six months apart, between December 1984 and May 1985.

As you might expect, one was presented as a "good news" study full of optimism about public attitudes toward the media. The other resulted in a report full of gloom and foreboding. Now here's the surprise: the sunny report was written by the people who produced Study Y, the one with the unfavorable attitudes cited above. And the gloomy report was based on the numbers in Study X, the one that found that most people trust newspapers.

Did the wires in the computer get crossed? No. The two research groups were working from different schema. Study X was sponsored by the credibility committee of the American Society of Newspaper Editors. Its chairman, David Lawrence, wanted to use the study as shock therapy to awaken editors to a problem and stir them to action.14  Study Y was sponsored by Times Mirror, which wanted some corporate visibility and a chance to make some news. Publishing findings "which run counter to the conventional wisdom" is a good way to do that.15

Both groups of researchers meant to tell the truth, but whether they saw a dog or a fireplug depended on the schema with which they approached the inquiry. To the journalist trained in objectivity, it may seem that the problem was in starting with any structural framework at all. The journalistic ideal is to be open-minded, to enter an investigation with a clean slate, free of any prejudgment. Having a hypothesis seems a lot like being prejudiced. That view, while well-intentioned, is not practical. You can't begin to think about a problem without some kind of theoretical framework. And you will have one, whether you are conscious of it or not. The process of hypothesis formulation in scientific method forces the framework to the conscious level where it can be coldly and objectively evaluated.

Now it is true that there is some danger that the evaluation will not be cold and objective enough. The two newspaper studies cited above are a good illustration. The cure for that problem is not to abandon the process of hypothesis-making but to state your hypothesis and evaluate it in full public view so that other investigators can check on your work. Both the Times Mirror and the ASNE researchers did just that. No long-term harm was done, because both left a paper trail that described their method, their findings, and the route to their conclusions. Even better, they made their data available for secondary analysis by other scholars. Their publications stimulated much analysis and discussion about the next step in understanding the peculiar relationship between a news medium and its readers, listeners, or viewers. One of the characteristics of science is that it is always subject to such checking and rechecking. That is one of the lessons from the case of the two credibility studies. The other is that it is extremely important to give careful thought to the theoretical structure with which one approaches a problem and to appreciate the consequences of the choice of a schematic model.
 

Reality testing

Once you choose a model, you may be stuck with it for a long time. A model that works well enough for the purpose at hand soon becomes comfortable, like an old shoe, and is given up reluctantly. Inappropriate models, clung to after their usefulness is gone, are the source of superstition, conventional wisdom, and closed minds. Herbert Butterfield, the historian of modern science, has argued that the development of new models was more important in the creation of modern physics than the collection of new data. A successful scientist needs "the art of handling the same bundle of data as before, but placing them in a new system of relations with one another by giving them a different framework, all of which virtually means putting on a different kind of thinking-cap for the moment."16  Modern scientific method provides an incentive to put on that different thinking cap by prompting a continual testing of the old models and a perpetual search for better ones. Aristotle, whose model for the mechanics of motion held that a moving body continued on a given path only so long as something was pushing it, was wrong. And yet his model was the dominant one for centuries, partly because science was not yet conditioned to challenge and to experiment. (One problem encumbering Aristotle and his Greek contemporaries was the sharp distinction between philosophers and working people. To actually conduct an experiment would involve getting one's hands dirty. This disinclination to do more than think persisted even to the time of Galileo, who preferred "thought experiments" to the real thing.)

Modern scientific method provides for aggressive reality testing. Journalists are interested in testing reality, too. The main difference -- beyond the obvious one that journalists are in more of a hurry -- is that journalists are more passive about it. Instead of testing reality directly with their own observations, deductions, and experiments, they are normally content to do their cross-checking by consulting different authorities with different viewpoints and different interests. The flaw in this methodology is that the journalist may not have any good basis for evaluating the conflicting sources and may be forced into the traditional objectivist stance which demands the unlikely assumption that all voices have an equal claim to the truth. The journalist who adapts the tools of scientific method to his or her own trade can be in a position to make useful evaluations with the more powerful objectivity of science.

Journalists already share some of the characteristics of scientists, often without knowing it. Among them:
 

1. Skepticism. "If your mother says she loves you, check it out," is an aphorism from the culture of journalism, not science, but it fits both equally well. Neither journalists nor scientists are content to rest with what popular opinion or authority claims is true. Truth is always tentative, always has room for sharpening and improvement.

2. Openness. The key word is "replicability." A good investigative reporter documents his search for truth, making a paper trail that other investigators can follow to reach the same conclusions.

3. An instinct for operationalization. To test a model, a scientist thinks about the processes that the model represents and where they lead. Then he or she looks for a place in the observable world where aspects of that process can be measured in a way that will confirm or refute the model. That process of finding the observable and testable piece is called operationalization. Both scientists and investigative journalists depend on it. The confirmation of a theory is its power to predict the results of an operational measurement.

4. A sense of the tentativeness of truth. In the ancient argument between absolutism and relativism, science is more comfortable with relativism. The truths that it discovers are welcomed when they improve our understanding or our technology, but with the recognition that they might be replaced by stronger truths in the future. This concept is not an easy one for journalists, whose quest for simplicity and certainty makes absolutism appealing.

5. Parsimony. Given a choice between rival theories, we generally prefer the simpler one. The best theory explains the most with the least. The Copernican theory of the universe prevailed over the older system of Ptolemy because it was simpler. In order to account for the motion of the planets, Ptolemy proposed a system of "epicycles" in which each planet moved in orbits within orbits around the earth. As instrumentation got better, astronomers detected movements that the theory could not explain unless more epicycles within epicycles were postulated. Putting the sun at the center of the system eliminated the need for epicycles.


The importance of humility

It is ironic that journalistic defense of the First Amendment today is often argued from an absolutist position. The eighteenth century political thought underlying the constitution of the United States, particularly the First Amendment, is based on a rejection of absolutism. In the dawn of the scientific era, the futility of compelling any particular belief was clear because new data or a new interpretation of old data was always arising to challenge the old beliefs. And so it made sense to be tolerant of unorthodox beliefs and to protect them under the basic law of the land. Today's heresy could be tomorrow's wisdom.

So a little bit of humility is good for everyone, but especially for scientists and journalists. Justice Oliver Wendell Holmes noted that what we think of as truths might better be called "can't helps":

When I say that a thing is true, I mean that I cannot help believing it. . . . But . . . I do not venture to assume that my inabilities in the way of thought are inabilities of the universe. I therefore define truth as the system of my limitations, and leave absolute truth for those who are better equipped. . . . Certitude is not the test of certainty. We have been cock-sure of many things that were not so.17
This unpretentiousness can give the scientist and the journalist a certain freedom. You do not have to wait to verify that you have found certain, absolute, unassailable truth before sharing the finding. If you have made a mistake, and been open about the process that led you to it, someone else will discover it and publish, and the cause of truth will have been advanced another notch. Democratic pluralism, which lets truth emerge from the combat of many voices, is a good environment for both scientist and journalist.

One of the advantages of adopting a theoretical model for journalistic use is that it keeps you and your readers focused on the relevant. Much information in public affairs is arcane and complicated. The journalist who becomes an expert in the field being covered can quickly lose the way in irrelevant detail.

My first major investigative story as a newspaper reporter was about wasteful purchasing of fire and windstorm insurance on school buildings in Dade County, Florida. Under the plan then in effect, the insurance program was controlled by an agent whose compensation was a share of the commissions. The more insurance cost, the more money he made. Debates at school board meetings on the subject were obscure in their detail and impossible to follow. In frustration, the superintendent of schools asked his board for a $10,000 study to resolve the technical complexities. The request was tabled and eventually died.

What was needed to understand the situation was a theoretical model that could be operationalized. Mine was a basic log-rolling or mutual back-scratching model. It led to the following hypotheses:

1. The insurance servicing agent's self-dealing would make insurance more costly than for similar bodies without such a conflict of interest.

2. If the board goes along with this cost, its members must be receiving something of value: for example, the power to reward their friends in the insurance business.

3. If they are rewarding their friends, board members must again get something in return.

Each of those propositions is simple. You don't need a detailed understanding of the insurance business or its regulation to grasp any of them. And each could be operationalized.

The first proposition was tested by comparing property insurance costs of the Dade County schools with those of the Metropolitan Government of Dade County, a separate political unit with the same geographic definition. The Metropolitan Government costs, per dollar of building value, were a third those of the Dade County Schools. Moreover, the loss ratio for school insurance was quite low by industry standards. Over the long history of the program, only 15 percent of the premium money had gone back to the school system in claims.

Direct questioning of the school board members and the benefiting insurance agents verified that the system had the earmarks of political patronage. To keep the other insurance agents in the community happy with the system, the school system's servicing agent shared a portion of his commissions with them. School board members decided how this sharing would be allocated. Now all that was needed was to close the logical loop by showing that board members got something in return.

Florida pioneered in election reporting laws, and each member's campaign contributions were on file at the courthouse. Each entry showed a donor's name, the candidate's name, and a dollar amount. Confirmation of the third hypothesis required a link between the donors and the insurance business. There were 181 agencies participating in the school insurance program. From an industry directory I got the names of their officers and made a card index, one name per card, and alphabetized it. Then I checked each name on the campaign contributor lists against the card index. Bingo! Insurance people were important contributors to all five of the school board members who had run in the previous election. The chairman of the board received 65 percent of his total contributions from insurance people. Two others got more than half from the same sources. The resulting three-part series led the local page.18 The school board quickly reformed the system and eventually gave teachers a raise with the money that was saved.

A reporter working on such a story today would, of course, use a computer instead of three-by-five index cards to check out the names. But the mechanics are less important than the concept. A model that points you to the relevant facts and suggests an operationalization is far more important in increasing your power as a reporter than the machine that does the clerical part. The machines do make things possible that could not be done before, but to get the most out of them, we need to think the way that scientists think, building the models that enable us to use the new computing power.

Journalists tend to be practical people, and as such we are sometimes too quick to scorn people who deal in what seems to us to be empty theory lacking in any real-world application. But without theory, we have nothing but unordered raw data, and we suffocate in it. The computer can alphabetize it for us, it can give us lists ordered by any dimension found in the data, but we still have to have a theory to give the list a point.
 

Sources of theories

Where do theories come from? The range of sources is broad, from folklore to the products of the most creative and sophisticated minds in science. A good theory is one that has consequences that can be tested and used as building blocks for more discoveries and more theories. Science is as dependent on openness as journalism is.  Michael Polyani, a chemist who worked at the dawn of the nuclear age, compared the scientific process to a group of workers assembling a giant jigsaw puzzle. A group can't do it efficiently unless each member knows what the other members are doing. "Let them work on putting the puzzle together in the sight of the others, so that every time a piece of it is fitted in by one, all the others will immediately watch out for the next step that becomes possible in consequence," he said.19 The methodology of journalists covering public affairs is exactly the same, even if the process is less self-conscious. Reporters, like scientists, are in the business of reality testing, examining the existing theories, thinking through their consequences, developing related hypotheses that can be operationalized (i.e., tested), and putting them to the test.
 

The case of the Detroit riot

Periods of social upheaval offer especially good opportunities for testing theories. When the race riots of the 1960s began, first in the Watts area of Los Angeles, then in Newark and Detroit, there were several popular theories to explain the cause. One theory, popular with editorial writers, was that the rioters were the most frustrated and hopeless cases at the bottom of the economic ladder who rioted because they had no other means of advancement or expression. This theory can be tested with survey research. If you can identify the rioters and compare them with the non-rioters, you will find, if the theory is true, that the less-educated are more likely to be riot participants. The DetroitFreePress did perform such a survey in 1967, and found that people who had attended college were just as likely to participate in the riot as those who had failed to finish high school. The theory was not supported by the data.20 

Education
 
Dropouts
High School
College
Rioters
18%
15%
18%
Non-rioters
82%
85%
82%
Total
100%
100%
100%
Another popular theory of the time was that the root cause of the riots was the difficulty that southern blacks had in being assimilated into northern culture. Forced by southern repression and the effects of slavery into a passive role, they found an outlet for long-suppressed aggression once they left the South. This theory, too, is easily operationalized. If true, rioting should be a more frequent behavior for immigrants from the South than for those blacks raised in the North. The Free Press survey produced a different result. Another theory subjected to reality testing and found wanting!

  
  Where were you brought up as a child?
 
South
North
Rioters
8%
25%
Non-rioters
92%
75%
Total
100%
100%
Another opportunity came after the assassination of Martin Luther King in 1968. The immediately popular theory was that his nonviolent movement died with him and that blacks would turn for leadership to the advocates of violence. The MiamiHerald, which had done an attitude survey among its black population before the assassination, went back to the same respondents and found that King's ideals were stronger than ever.21

Race relations has proved a fertile field for journalistic reality testing more recently. Pulitzer prizes were won for stories published in the Dallas Morning News in 1985 and the Atlanta Constitution in 1988 that showed how federal government programs were being used to perpetuate racial discrimination. The Texas case involved segregation in public housing. The Atlanta story documented discrimination in federally insured mortgage lending. Both projects relied on computer analysis of large government databases. Both started with a theoretical framework: a government with a commitment to racial equality should produce measurable results that would reflect that equality. Measurement and analysis showed that, in reality, the results of government programs did not reflect equality.22 To decide what to measure, the reporters had to have a theory and an operational hypothesis. In other words, they were thinking like scientists.

Here is what Julia D. Wallace, USA Today managing editor for special projects, had to say after her first few months of wrestling with precision journalism: "I guess the bottom line here is it is going to take more to be a journalist in the future. It's not enough to have writing ability, good sources, and lots of energy. Reporters and editors are going to need to know something about statistics and software.

"It's not easy. But in the end, it sure is fun."23
 


Notes

1. For example, Ron Lovell, The Quill, October 1987, pp. 22-24: "There is no substantive body of knowledge for the field."  return to text

2. Derek Bok, personal communication, February 27, 1990. return to text

3. "Soon it will be doubling every four."  Richard Saul Wurman, Information Anxiety (New York: Doubleday, 1989), p. 32.  return to text

4. See, for example, Nicholas Von Hoffman, "Public Opinion Polls: Newspapers Making Their Own News?" Public Opinion Quarterly, 44:4 (Winter 1980), 572. return to text

5. For some examples, see the discussion of "the new nonfiction" in Philip Meyer, Ethical Journalism: A Guide for Students, Practitioners, and Consumers (New York: Longman, 1987).  return to text

6. Walter Lippmann, Public Opinion (New York: The Free Press, Paperback Edition, 1965; first publication, 1922), p. 216.  return to text

7. Lawrence Cranberg, "Plea for Recognition of Scientific Character of Journalism," Journalism Educator, Winter 1989, pp. 46-49.   return to text

8. James K. Batten, Press-Enterprise Lecture, Riverside, California.  April 3, 1989.  Miami: Knight-Ridder, Inc.   return to text

9. Michael J. Davies, quoted in Keys to Success: Strategies for Newspaper Marketing in the '90s (Reston, Va.: American Newspaper Publishers Association, 1989), p. 5.  return to text

10. Christine D. Urban, "Reader Expectations -- What They Think of Us," address to American Newspaper Publishers Association, San Francisco, April 1986.   return to text

11. Reid Hastie, "Schematic Principles in Human Memory," in N. Cantor and J. F. Kihlstrom (eds.), Personality, Cognition and Social Interactions (Hillsdale, N.J.: Erlbaum, 1981), pp. 39-40.   return to text

12. Lippman, Public Opinion, pp. 53-68.   return to text

13. Ibid., p. 76.   return to text

14. "Newspaper Credibility: Building Reader Trust" (Reston, Va.: American Society of Newspaper Editors, 1985).   return to text

15. "The People & The Press: A Times Mirror Investigation of Public Attitudes Toward the News Media Conducted by the Gallup Organization" (Los Angeles: Times Mirror, 1986), p. 4.   return to text

16. Herbert Butterfield, The Origins of Modern Science, Revised Edition (New York: The Free Press, Paperback Edition, 1965; first publication, 1957), p. 13.   return to text

17. Oliver Wendell Holmes.  Quoted by Arthur Schlesinger, Jr., "The Opening of the American Mind," New York Times Book Review; July 23, 1959, p. 27.   return to text

18. "Freeloading Insurors Get Your $59,000," Miami Herald, September 27, 1959, p. 1B.   return to text

19. Quoted in Richard Rhodes, The Making of the Atomic Bomb (New York: Simon & Schuster, 1986), p. 34.   return to text

20. Philip Meyer, "The People Beyond 12th Street: A Survey of Attitudes of  Detroit Negroes After the Riot of 1967," Detroit Free Press reprint, 1967.   return to text

21. Philip Meyer, "The Aftermath of Martyrdom: Negro Militancy and the Death of Martin Luther King," Public Opinion Quarterly, TK.   return to text

22. Craig Flournoy and George Rodrigue, "Separate and Unequal: Illegal segregation pervades nation's subsidized housing," Dallas Morning News, February 10, 1985, p. 1. 
 return to text

23. Julia D. Wallace, remarks to First Amendment Congress, Washington, D.C., October 27, 1989.     return to text

Download Chapter 1
Word
 

Index button

Chapter 1 - Journalism and the scientific tradition
Chapter 2 - Some elements of data analysis
 
 
 

Chapter 3 - Harnessing the power of statistics
 
 
 
 

Chapter 4 - Computers

Chapter 5 - Surveys
 
 
 
 

Chapter 6 - More about data analysis
 
 
 
 

Chapter 7 - Field experiments
 
 

Chapter 8 - Databases


 
 
Chapter 9 - How to do an election survey
 
 
 

Chapter 10 - The politics of precision journalism