Chapter 11 – Non-Reactive Research Methods (pp. 269-295)
Overall teaching objective: To introduce undergraduate criminal justice research methods
students to various non-reactive research methods and to demonstrate their applications.
Note to instructors: This chapter is presented in two sections. The first section provides an
overview of the research method. The second section uses a research report to demonstrate how
previous researchers applied this method to a project relevant to criminal justice practice. In
both sections the material is organized by the generic research process that was presented in
Chapter 2.







In Chapter 9 (Experimental Design Research Methods), you were introduced to a concept
called reactivity.
Reactivity occurs when research subjects change their behavior because they become
aware that they are being watched or measured.
To overcome this common threat to external validity, researchers may want to observe
and measure the behavior of their research subjects in a natural setting. ‘Natural setting’
does not mean in the woods. It means observing how people behave normally in social
situations.
To capture human behavior in natural settings, researchers have developed numerous
non-reactive research methodologies.
Non-reactive research is a collection of research methods that unobtrusively gather
information from research subjects that is, without their knowledge.
Because the research subjects are unaware that they are being observed, they are less
likely to change their behavior.
Of course there are certain ethical considerations in this research method, but when done
carefully, non-reactive research methods can be instructive to the body of knowledge.
Making Research Real 11.1 – Who Buys Gasoline Here?(p. 269)
 This research project begins when a researcher observes that the buttons on the
gasoline pumps are worn out in different places from one chain convenience store
to the next.
 In the lower socio-economic section of town the button over “Pay Inside Cash” is
worn while in the higher socio-economic section of town the button over “Pay
Outside Credit” is worn.
 The instructor uses these erosion measures to make conclusions about the
differences in the patrons in these two stores.
Non-Reactive Research Methods Basics (p. 271)
 Non-reactive research methodologies have in common their ability to collect
information from and about human beings without their knowledge.
 These methods can include;
o evaluating the things people possess (i.e., accretion measures),
o studying how things are used (i.e., erosion measures),
o observing how individuals or groups behave (i.e., unobtrusive
observation), and
o analyzing information collected and made available by someone else (i.e.,
archival data)..
Types of Non-Reactive Methods (p. 272)
 There are three major types of non-reactive methods: field research, secondary
analysis, and content analysis.
o In field research, the researcher observes behavior(s) in a natural setting.
o Another unobtrusive method is secondary analysis. Here, the researcher
analyzes previously collected or archival data. Sometimes the data are
collected for the purpose of another research project. Sometimes the data
are collected and made public for scholars and practitioners to analyze as
they see fit. In either case, a researcher would obtain the data set and
analyze all or parts of it.
o A third and final type of non-reactive method is content analysis. In a
content analysis, the researcher analyzes existing textual information to
study human behavior or conditions.
Making Research Real 11.2 – Field Research in Criminal Justice (p. 272)
 A police captain responds to a neighborhood association’s complaint that
somebody is stealing their morning papers.
 There appears to be no pattern to these thefts.
 The captain assigns an officer in an to (covertly) observe the neighborhood to
catch the culprit.
 The culprit turns out to be a dog.
Making Research Real 11.3 – Secondary Analysis in Criminal Justice (p. 272)
 A warden is not supportive of a recent proposal to triple bunk prison cells.
 The warden conducts a review of injury reports involving staff in the prison for
the past 20 years.
 The warden finds that as the population of the prison increases the frequency and
severity of staff injuries also increases.
Making Research Real 11.4 – Content Analysis in Criminal Justice (p. 274)
 An English professor is interested in how police officers communicate in writing.
 She evaluates police reports and identifies common words and phrases.
 Using this information she develops a training program to help the officers write
more precisely.
The Benefits and Limitations of Non-Reactive Research Methods (p. 274)
Table 11.1 - The uses of non-reactive research. (p. 275)
:

When research subjects are likely to change their behavior after learning that they are


being observed
When the researcher wants to observe behavior in its natural setting and/or as it naturally
occurs
When the data has been previously collected by a different individual, group or
organization
Table 11.2 - The limitations of non-reactive research. (p. 276)




Ineffective for studying the internal motivations, beliefs and attitudes that underlie some
behavior
Potentially unethical if it involves breaching research subjects’ right to or expectation of
privacy
Impossible in cases where secondary or archival data do not already exist or are
unavailable
Non-reactive studies are often hampered by internal and external validity problems
Note to instructors: In this section the research method is discussed within the context of a
relevant research project. The material is organized by the generic research process that was
presented in Chapter 2. The story (i.e. the research project that provides the context) is presented
in a series of set out boxes called Developing the Method. These are repeated here to allow
instructors to use the research to teach the concepts, tools and techniques related to this research
method.
The Non-Reactive Research Process (p. 276)
 As in the previous two chapters, we will take a look at the actual research process
to understand non-reactive methods in greater depth.
 Following the steps of the research process outlined in Chapter 2, we will review
the process by which a researcher might conduct field research, secondary
analysis and content analysis.
 To illustrate the process further, I will highlight one research study that involved a
content analysis. We begin with a general introduction of this case study.
Developing the Method 11.1 - A Case Study in Non-Reactive Research (p. 276)
Several years ago, when my children were young, my wife announced, “From now on, we are
not going to watch the evening news! It’s too violent.” She was right. We were living in a large
city at the time and it seemed like the evening news was filled with stories about murder, rape,
assault, robbery and other violent acts. “If it bleeds, it leads” is a common saying in news rooms.
In short, crime ‘sells’.
The tendency of media outlets to focus on crime and violence has the unfortunate effect of
creating false perceptions regarding crime. After a few weeks of watching the evening news,
one might think that crime is both common and getting worse. But the statistics indicate that
crime is both uncommon and decreasing. In fact, since 2001, violent crimes have decreased 13.2
percent.
To better understand media portrayals of crime, Melissa Hickman Barlow, David E. Barlow and
Theodore G. Chiricos set out to conduct a content analysis of media content on crime. They
wanted to know whether economic conditions had an effect on how the media portrayed crime.
In 1995, they published their research in the scholarly journal Crime and Delinquency under the
title “Economic Conditions and Ideologies of Crime in the Media: A Content Analysis of Crime
News”. Their research is an excellent example of content analysis and non-reactive methods in
general.
Ask a Research Question (p. 277)
 Most non-reactive research is exploratory or descriptive nature. Because it is
often difficult to measure the underlying cause of behavior using a non-reactive
research method, non-reactive methods are less often used in explanatory
research.
Developing the Method 11.2 - Asking a Research Question in Non-Reactive Research (p.
277)
Barlow et al. (1995) state that their study is exploratory, which would be appropriate for a
content analysis. However, the authors propose that political and economic conditions might
help explain how the media portrays crime. Thus, it appears that there are pieces of this study
that are explanatory in nature. In their own words, they wanted to know “whether media
accounts misrepresent crime in ways that support dominant class interest and whether
misrepresentation changes in relation to conditions in the political economy” (Barlow et al. 1995:
6). In other words, they are asking whether the media reports on the crimes that most of us want
to read about because of what is happening at the time both politically or economically.
Conduct a Literature Review (p. 278)
 Reviewing the research methods and mistakes of previous researchers who
used non-reactive research techniques may help define concepts, access
certain populations, and avoid detection in non-reactive research.
Developing the Method 11.3 - Conducting a Literature Review in Non-Reactive Research
(p. 278)
In their review of the literature, Barlow et al. (1995) discovered a rather robust research history.
They found numerous sources that confirmed what we all suspected: “crime news distorts and/or
frames crime and crime control in ways that support institutions of power and authority” (Barlow
et al. 1995: 3). In short, there are political and economic interests involved in how crime is
portrayed in society. Though the literature suggested that media reports on crime painted a
distorted picture of crime, previous research had not determined how these distortions changed
with changing political and economic conditions. So, the researchers decided to proceed with
their study.
Refine the Research Question (p. 279)
 In non-reactive research that is exploratory or descriptive in nature, researchers
may simply pose research questions and/or general statements about what they
expect to find in the study.
 In non-reactive research that is explanatory in nature, researchers will develop
more formal hypotheses.
Developing the Method 11.4 - Refining the Research Question in Non-Reactive Research (p.
279)
Barlow et al. (1995) predicted that newspaper coverage of violent crime would not only overstate
the problem of violent crime, but that coverage would be most intense during times of
unemployment and economic stagnation. Thus, they presented two sets of hypotheses at the
outset of their article. The first set of hypotheses focuses on the depiction of violent crime in
news articles:
Hypothesis 1: The proportion of news articles that are about crimes of violence is
significantly larger than the percentage of violent crimes known to the police.
Hypothesis 2: The proportion of news articles on crimes of violence is greater
during high-unemployment years than during low-unemployment years.
Hypothesis 3: The proportion of news articles on crimes of violence is greater
during periods of economic stagnation than during periods of economic
expansion.
The researchers’ second set of hypotheses focused on the “characteristics and images of
offenders within crime news articles” (Barlow et al. 1995: 10):
Hypothesis 4: A significantly larger proportion of non-White offenders are
depicted in the news articles than the proportion of non-White offenders who are
actually arrested.
Hypothesis 5: When the social class of the offender is mentioned, the largest
proportion of social class descriptions is in the lower class category.
Hypothesis 6: The proportion of negative images of offenders is greater during
high-unemployment years than during low-unemployment years.
Hypothesis 7: The proportion of negative images of offenders is greater during
periods of economic stagnation than during periods of economic expansion.
In general, the researchers predicted that newspaper coverage would highlight the non-White and
lower class characteristics of criminal offenders during times of unemployment and economic
stagnation. That is, the media would focus disproportionately on non-White, lower class
criminal offenders during economically difficult times.
Define the Concepts and Create the Measures (p. 280)
 For the most part, the actual process of conceptualization and operationalization is
the same in non-reactive research as it is in other research methods.
Developing the Method 11.5 - Defining Concepts and Creating Measures in Non-Reactive
Research (p. 281)
Barlow et al. (1995) propose that two elements of crime reporting will vary by economic
conditions: (1) the type of crimes that is reported and (2) the characteristics of offenders that are
highlighted. In essence, they define the concept of ‘media coverage of crime’ in two parts.
To measure the type of crime reported by the media, the researchers differentiated between two
types of crime: violent and non-violent. Thus, when the researchers analyzed media content,
they coded the content as focusing on violent or non-violent crimes. To measure the
characteristics of offenders that are highlighted, the researchers included the following variables:
age (under 30 or 30+), race (White or Non-White), Gender (Male or Female), social status
(Lower, Middle, or Upper), Employment (Unemployed, Blue-collar, or White-collar), Marital
status (Married or Not married), Family history (Positive or Negative), Education (High school +
or Less than high school), Friends (Has friends or Is isolated), and Religion (Religious or Not
religious). It is likely that the researchers selected these variables because their review of the
literature suggested that factors like age, race and gender influence whether and how criminals
are depicted in the media.
The authors conceptualize ‘economic conditions’ in two ways: (1) the level of unemployment
and (2) the level of economic stagnation. The authors do not explicitly define what they mean
by ‘high’ or ‘low’ unemployment. Instead, they decide to analyze media coverage during the
year 1953, when there was a relatively low unemployment rate (2.9%), and 1958, when there
was a comparatively high unemployment rate (6.8%). Likewise, the authors do not define what
they mean by ‘economic stagnation’ and ‘economic expansion’. Again, they decide to analyze
media content during years that are popularly known to be periods of economic stagnation.
Those years were 1975, 1979, and 1982, during which the unemployment rates were 8.5%, 5.8%,
and 9.7%, respectively (Barlow et al. 1995: 6).
Design a Method (p. 282)
 Non-reactive field research involves covert observation of and, in some cases,
covert participation in the behavior that a researcher is interested in studying. In
this method, it is important to prepare for all possible contingencies that may arise
in the field and all possible threats to researchers’ safety.
 The use of secondary data is appealing to most researchers because of its
availability and low cost. However, the secondary data must be responsive to the
researcher’s data needs and research question(s).
 The most important step in designing a content analysis is to create strong
conceptual definitions and operational measures for the phenomena that are being
studied. In some cases, this may involve training research assistants.
Developing the Method 11.6 - Designing a Method in Non-Reactive Research (p. 284)
To study media coverage of crime at different periods of time, Barlow et al. (1995) decided to
study articles appearing in Time magazine. The researchers chose Time because it was a widely
circulated news magazine and “the best available representative of mainstream media as an
influence on and reflection of popular consciousness concerning contemporary social issues”
(Barlow et al. 1995: 6). In short, the authors suspected that the magazine might reflect changing
public opinion and cultural mores.
An additional consideration for selecting Time magazine was that the magazine had a long
publication history, making it a suitable source of longitudinal information on changing
portrayals of crime. Their selection of this publication enabled the authors to examine “the
content of crime news articles…at different points in the postwar period” and to analyze “media
representation of crime in relation to changes and developments in the political economy”
(Barlow et al. 1995: 6).
Collect the Data (p. 284)
 Coding sheets provide researchers with an effective and efficient mechanism for
recording information in content analyses and field observations.
 Recording observations is a central part of field research. Video- and audiotaping observations is the most accurate recording method, but these are also the
most reactive and, in some cases, they are also unethical. Recording observations
on paper or a laptop are the least accurate, but also the least reactive. Researchers
must find the appropriate balance between accuracy and obtrusiveness.
Developing the Method 11.7 - Collecting Data in Non-Reactive Research (p. 287)
During the years on which their analysis focused (1953, 1958, 1975, 1979, and 1982), Barlow et
al. (1995) chose articles that were “completely or substantially about crime, criminals, or
criminal justice.” Only articles that were at least one column in length were chosen. A total of
175 articles met these criteria. Technically speaking, this would be a purposive sample. Only
the articles on crime appearing during the years they chose were included in their analysis. The
researchers in this case did not use a random sample presumably because no sampling frame
existed. In this case, the researchers would have had to identify every article on crime ever
published in a written and widely circulated American news source during the selected years.
Though this was arguably possible, it was not feasible or efficient. Instead, the researchers chose
a representative news source to study media portrayals of crime. Though their sampling strategy
was not random, it provided important insight into human behavior.
Once the articles were identified, the researchers turned their attention to conducting the actual
content analysis. To do this, they read each article and collected information related to the
variables in which they were interested. First, the researcher differentiated between articles
about violent and non-violent crime, which was relatively straightforward (Barlow et al. 1995).
Collecting information about the characteristics of the offenders was a bit more problematic.
Few articles included information on each of the ten variables in which the researchers were
interested. For example, very few of the articles included information about the offenders’
education, relationships with friends, religion or marital status. In short, their analysis did not
yield information about every variable related to the characteristics of the offender.
Although they did not report this detail in their article, it is likely that Barlow et al. (1995) used
coding sheets as they read each of the articles. These coding sheets probably contained a list of
each variable along with their attributes. As they read the article, the researchers likely made
notes on these coding sheets to facilitate subsequent data entry and further analysis. For
example, if the article contained a reference to the offender’s college degree, the researcher
would place a check mark beside the attribute ‘High school +’ under the education variable.
It is also likely that the researchers read through a few of the articles at first to see if their data
collection procedure would work, though, again, they do not report this. For example, if none of
the articles contained information about the offender’s religion, it would be futile to include this
variable in the analysis. If, on the other hand, most of the articles contained information about
the offender that they had previously not considered important (e.g., history of drug use), they
might have considered adding this variable to their analysis.
Analyze the Data (p. 287)
 The analysis of secondary data, field observations and content material can be
qualitative or quantitative.
 The type of analysis depends on the research question and the type of data that are
available.
Developing the Method 11.8 - Analyzing Data in Non-Reactive Research (p. 288)
Barlow et al. (1995) did not include information in their article about how they prepared the
information for analysis or how they actually analyzed the content of the articles. It is likely that
they entered the information from the coding sheets into a computer program like Excel or SPSS
for subsequent analysis. This assumption is based on their use of tabular data within the text of
their article.
In terms of their analysis strategy, the authors relied on a statistical technique called Chi-Square
analysis, in which they compared percentages. For example, they compared the percentage of
articles that focused on violent crime during periods of high unemployment to the percentage of
articles that focused on violent crime during periods of low unemployment.
Interpret the Results (p. 288)
 The data or information produced by many non-reactive research techniques often
can be interpreted in multiple ways. As such, researchers using these techniques
should entertain various interpretations. They should also be up front about the
limitations of their study.
Developing the Method 11.9 - Interpreting the Results in Non-Reactive Research (p. 289)
Overall, Barlow et al. (1995) found what we already suspected: crime news is not at all
representative of actual crime trends. In other words, the media does not do a very good job of
creating an accurate or representative picture of crime. These researchers were somewhat
successful in developing a connection between economic conditions and the types of crime
stories reported in the media. However, the causal connection between economic conditions and
crime reporting is really not known. For example, there is really no way to determine whether
the economic conditions existing at the time actually influenced the editorial decision to report
on a particular type of crime. These types of causal connections are often difficult in nonreactive research.
Though insightful, this particular study was not without its flaws. Some of these were pointed
out by the researchers. First, one could argue that the sample of articles is not representative of
the overall media. Other news outlets reported on crime during this period and may have taken a
different position than the editors at Time. Second, the articles included in the analysis were
from five years within a 29 year time frame. Time’s editorial philosophy and idea of what was
newsworthy may have changed numerous times during this time frame. Finally, though the
authors show some relationship between economic conditions and distorted depictions of crime,
their analysis did not allow them to substantiate why this might be the case.
Communicate the Findings (p. 289)
 Researchers may use tables, charts and graphs to communicate the results of their
non-reactive research if the analysis is quantitative in nature. Alternatively, if the
analysis is qualitative in nature, they may tell a ‘story’ using quotes, examples and
descriptions.
Developing the Method 11.10 - Communicating the Results of Non-Reactive Research
Barlow et al. (1995) chose to publish their results in a peer reviewed academic journal. They are
university scholars and as such benefit from the publication of their results in this medium.
Publishing their research in an academic journal also enhances the scientific value of their major
finding that the media provides a distorted portrayal of crime. There has been much debate over
media bias between traditional media outlets like ABC, NBC and CBS and cable news outlets
like Fox News, CNN and MSNBC. Presenting research on this subject in a respected academic
journal removes it from the usual muckraking that happens between media providers and
contributes actual evidence to the debate around media bias.
Getting to the Point (Chapter Summary) (p. 291)
 Non-reactive research is a collection of research methods that gather information
from research subjects without their knowledge. Because the research subjects
are unaware that they are being observed, they are less likely to change their
behaviors. These techniques are effective for observing behavior in a natural
setting.

Non-reactive research methodologies have in common their ability to collect
information from and about human beings without their knowledge. These
methods can include evaluating the things people possess (i.e., accretion
measures), studying how things are used (i.e., erosion measures), observing how
individuals or groups behave (i.e., unobtrusive observation), and analyzing
information collected and made available by someone else (i.e., archival data).

Non-reactive research techniques are most effective when research subjects are
likely to change their behavior when they know they are being observed, when the
researcher wants to observe behavior in its natural setting, and/or when the data
the researcher needs is already available.

Non-reactive research methods are not particularly effective when a researcher
needs to understand underlying motivations and belief systems, when research
subjects have a right to or expectation of privacy, and when secondary or archival
data do not exist.

Most non-reactive research is exploratory or descriptive nature. Because it is
often difficult to measure the underlying cause of behavior using a non-reactive
research method, non-reactive methods are less often used in explanatory
research.

Reviewing the research methods and mistakes of previous researchers who used
non-reactive research techniques may help define concepts, access certain
populations, and avoid detection in non-reactive research.

In non-reactive research that is exploratory or descriptive in nature, researchers
may simply pose research questions and/or general statements about what they
expect to find in the study. In non-reactive research that is explanatory in nature,
researchers will develop more formal hypotheses.

For the most part, the actual process of conceptualization and operationalization is
the same in non-reactive research as it is in other research methods.

Non-reactive field research involves covert observation of and, in some cases,
covert participation in the behavior that a researcher is interested in studying. In
this method, it is important to prepare for all possible contingencies that may arise
in the field and all possible threats to researchers’ safety.

The use of secondary data is appealing to most researchers because of its
availability and low cost. However, the secondary data must be responsive to the
researcher’s data needs and research question(s).

The most important step in designing a content analysis is to create strong
conceptual definitions and operational measures for the phenomena that are being
studied. In some cases, this may involve training research assistants.

Coding sheets provide researchers with an effective and efficient mechanism for
recording information in content analyses and field observations.

Recording observations is a central part of field research. Video- and audiotaping observations is the most accurate recording method, but these are also the
most reactive and, in some cases, they are also unethical. Recording observations
on paper or a laptop are the least accurate, but also the least reactive. Researchers
must find the appropriate balance between accuracy and obtrusiveness.

The analysis of secondary data, field observations and content material can be
qualitative or quantitative. The type of analysis depends on the research question
and the type of data that are available.

The data or information produced by many non-reactive research techniques often
can be interpreted in multiple ways. As such, researchers using these techniques
should entertain various interpretations. They should also be up front about the
limitations of their study.

Researchers may use tables, charts and graphs to communicate the results of their
non-reactive research if the analysis is quantitative in nature. Alternatively, if the
analysis is qualitative in nature, they may tell a ‘story’ using quotes, examples and
descriptions.