Effectiveness of
Mass-Media Health Campaigns
Mass media health campaigns have been an important strategy for health promotion and
disease prevention since the 1940’s. Yet considerable debate surrounds the effectiveness of these
campaigns. Mass media health communication campaigns frequently take the form of a series of
television and radio public service announcements (PSAs) with collateral print materials such as
posters, booklets, and brochures. Other organizations are often involved as intermediaries to help
disseminate the messages of the campaigns. Mass media campaigns have been conducted on
topics ranging from general health issues to specific diseases, including the following:
cardiovascular health; smoking; alcohol and drug abuse; nutrition; safety; family planning;
cancer control; immunization; screening; mental health; lung disease; hypertension; SIDS; Reyes
syndrome; and AIDS (Flay, 1987; Hornik, 1993; Joseph, 1988; Lau, Kane, Berry, Ware & Roy,
1980; Solomon, 1982; Reardon, 1988).
The purpose of this paper is to briefly describe the literature on the effects of these
campaigns focusing on a study I conducted that systematically reviews the empirical evidence of
the effects of campaigns on awareness, knowledge, attitude formation, and behavior (Freimuth &
Taylor, 1995). The paper will then identify three key issues, which need to be addressed by those
evaluating these campaigns. .
REVIEWS OF MASS MEDIATED CAMPAIGNS
Rogers and Storey (1987) summarized the prevailing views on campaign effectiveness
over the years, describing three eras of effects. The first, the era of minimal effects, reached its
peak in the 1940s and 1950’s and was characterized by studies that refuted the idea that mass
media could directly and consistently affect the behavior of individuals.
The second era, campaigns can succeed, followed in the 1960s and 1970s and brought
the realization that campaigns could succeed if they were designed and conducted in more
strategic ways. These strategies included the use of formative evaluation, audience segmentation,
interpersonal communication networks, and the setting of more reasonable campaign goals.
Effectiveness of Health Campaigns
2
Contemporary campaign research has reached a more balanced view of effects as
evidenced in the title Rogers and Storey (1987) give to the third era, moderate effects. Increasing
attention is given to small but overt behavior change and distal consequences of such changes.
An Australian review of mass media health promotion campaigns in two areas, cardiovascular
risk behavior and safety restraints (Redman, Spencer, and Sanson-Fisher, 1990) illustrates these
moderate effects. The authors began with 24 studies but determined that only nine met their
criteria for adequate evaluation methodologies. These nine were further divided into two models
of media effects: media only and media as agenda-setting plus community programming. Not
surprisingly, they concluded that media only campaigns had discouraging results but that most
studies of media plus intensive community interventions reported significant changes in
behavior. The authors, however, challenged these positive results by questioning how important
the media component was to the success of such combined programs.
Hornik (1996) offered the most provocative review of the evidence about the effects of
health communication campaigns. He asserted that the large community trials such as
Minnesota, Stanford and COMMIT, have shown either minimal or no overall effects yet there is
substantial evidence from observational studies such as the National High Blood Pressure
campaign evaluation of major health effects and plausible associations between the campaign
and the effects. His explanation for this contradiction is the lack of adequate exposure in most of
the controlled trials. Hornik suggests that if programs work, they likely work because they
activate a complex process of change in social norms rather than because they transfer
knowledge.
It is probably time to consider a fourth era and that one is characterized by the use of the
internet and by paid media rather than relying on public service time. It is too early to have
much data from this fourth era but the White Houses’ Office of the Drug Czar’s anti-drug
campaign shows some promising results as do many of the state anti-smoking campaigns.
What has been missing from these previous reviews is a systematic analysis of the size of
effects achieved for different types of objectives, e.g., awareness, knowledge, attitudes, and
behaviors. I addressed this gap by identifying and reviewing the extant empirical data from
evaluations of mass mediated health campaigns (Freimuth & Taylor, 1995).
Effectiveness of Health Campaigns
3
PROCEDURES USED IN CONDUCTING THE REVIEW
Because campaigns may have multiple objectives ranging from awareness to
behavior change, I categorized the evaluations according to McGuire’s hierarchy of effects
model. McGuire (1989) offered a comprehensive information—processing model in the form of
an input/output matrix. The output or dependent variables are response steps mediating
persuasion and include exposure, attending, comprehending, yielding, behaving, and retaining.
The independent communication variables include source, message, channel, receiver, and
destination characteristics. His input/output matrix provides a framework for organizing much of
the research conducted on campaigns. Certain assumptions had to be made in order to conduct
such a review. The first assumption is that health campaigns conducted since 1980 have reached
a point of standardization. Most health campaigns incorporate some aspects of planning and
strategy selection, selecting channels and materials, developing materials and pretesting,
implementation, assessing effectiveness, and feedback to refine the program. Therefore, what is
important in this review is not discriminating among campaigns as to quality but instead,
assuming some basic level of competency, and focusing on the size of effects the campaign
generated.
The second assumption is that the evaluation methods used to produce empirical results
are adequate. Some evaluations are clearly more rigorous than others and in the longer
description of this analysis. I pointed out these differences, even to the extent of categorizing
results according to the evaluation methods used.
An attempt was made to identify all quantitative evaluations of U.S. health campaigns
published between 1980 and 1997. Standard databases were used to locate these evaluations
such as Medline, Psych Abstracts, Social Science Citation Index, and Eric. Calls were also made
to key organizations and agencies such as NIH, CDC, and voluntary health organizations, e.g.,
the American Cancer Society to ask if there were unpublished campaign evaluations available.
Even with these methods, it is quite likely that I have missed some.
Each evaluation identified was examined carefully to determine what campaign
objectives were measured and the size of effects achieved. If a campaign evaluation measured
Effectiveness of Health Campaigns
4
multiple objectives, it was included in each of the relevant tables.
RESULTS
Presents the summary data for four campaign objectives: awareness, knowledge, attitude, and
behavior. In the following sections, the results for each of these objectives are discussed.
Table 1.
Average changes achieved after mass-mediated health campaigns
Campaign Objective
Average Size of Change %
Awareness (N=16) ............................................................................. 56
Knowledge change (N=15) ................................................................ 22
Attitude change (N-21) ..................................................................... 8
Behavior change (N=29) .................................................................... 13
AWARENESS
Theoretically, the mass media are supposed to be most effective in achieving awareness.
This review supports that expectation. When measuring awareness as simple recognition of the
message, up to 83% levels of awareness have been reported, with a median of 48%. Although,
without a pre message measure, some of this (perhaps up to 9%) may be measurement error, e.g.,
a desire to please the interviewer.
Ceiling effects must also be considered. If awareness is moderately high before the
campaign, there are ceilings on the increases possible and probably these increases are harder to
achieve. If both pre and post levels of awareness are available, increases can be calculated based
on the percent of audience possible to change. For example, if awareness of the seriousness of
colon cancer was 11% prior to a campaign and 40% after it, the increase, instead of being 29%
Effectiveness of Health Campaigns
5
would be 29% of the possible change of 89% which is 33%.
KNOWLEDGE
Knowledge gain is clearly achievable using mass mediated health campaigns. When
exposure is guaranteed, dramatic increases in knowledge (as large as 60%) have been observed.
When exposure is not guaranteed but the campaign can saturate a community as in the Stanford
Three Community Study (Maccoby, et al., 1977) knowledge gains around 25% seem feasible.
The size of these knowledge gains decrease when the campaigns are national in scope and must
compete with numerous other stimuli. Still, most of the campaigns were successful in achieving
some knowledge gain, although around 10% appears to be a more achievable increase. Multi—
channel campaigns appear to be much more successful than single channel, especially print only
campaigns.
ATTITUDES
All but four of the 21 evaluations of these health communication campaigns showed
significant attitude change. The actual amount of change varied considerably. These results
suggest that if exposure is insured, considerable attitude change is possible. The greatest amount
of change (+38% for an AIDS video shown in waiting rooms of STD clinics (Solomon &
DeJong, 1986) a case of forced exposure. The ARTA campaign (Woods, Davis, & Stover, 1991)
also demonstrated considerable attitude change, an average of 20% across five attitude items,
however, it must be remembered that the ARTA camp has received unusually high exposure for
a PSA campaign, and was only part of extensive media coverage of HIV/AIDS. Therefore, it is
impossible to know how much of that change is attributable to the campaign itself. Some of the
evaluations clearly suffered from ceiling effects and the results are difficult to interpret. The
surveys measuring outcomes of the Cancer Prevention Awareness campaign (USHHS, 1986), for
Effectiveness of Health Campaigns
6
example, found pre—campaign levels of 90+% on some of the items leaving little room to
measure change. In spite of more control over airing than the typical PSA, the single channel
campaigns did not achieve as much attitude change. The AD Council (1991) with its one PSA
reported an average of 4.5% change on two items and the Partnership for a Drug Free America
(1990) reported six percent change.
BEHAVIORS
Although behavior is normally considered one of the most difficult objectives to achieve
in mediated health campaigns, the campaigns reviewed here were quite successful. Only six of
the 29 behavioral change campaigns identified failed to achieve some level of change. The
average change reported was 13% should be noted that these results may be biased by the
tendency toward not publishing non—significant findings.
DISCUSSION
Even though mass media health campaigns are used extensively, considerable debate
continues over their effectiveness. This review differed from previous ones in that it included
only those campaign evaluations that collected quantitative evidence of impact and it organized
these data according to campaign objectives. In general, the results confirm Rogers and Storey’s
(1987) description of the era of moderate effects. As McGuire’s (1989) hierarchy of effects
model would predict, the size of the effects were greater at the earlier steps, i.e., awareness, and
knowledge than the later stages of attitude change, and behavior change.
KEY EVALUATION ISSUES
STANDARDS OF EVIDENCE
These results certainly support the use of mass mediated campaigns in health promotion
Effectiveness of Health Campaigns
7
and disease prevention programs. They also provide estimates for campaign planners to use in
predicting the impact of their efforts. Reviewing these campaigns as well as examining the
conclusions reached by others suggests some key evaluation issues that need to be addressed by
this field. Perhaps the most significant one concerns the standards for acceptable evidence
demonstrating effectiveness. There are many groups involved in efforts to systematically review
research results and apply criteria for success leading to best practice guidelines. Examples
include the Centers for Disease Control and Prevention’s Guide to Community Prevention and
the Substance Abuse and Mental Health Agency’s PEP series. Generally, these groups set
criteria where “strong” evidence equals randomized clinical trial designs. For example, consider
the following three criteria of study design for assessing effectiveness for the Guide to
Community Preventive Services:

Greatest -- Concurrent comparison groups and prospective measurement of
exposure and outcome

Moderate -- All retrospective designs or multiple pre or post measurements but
not concurrent comparison group

Least -- Single pre and post measurements and no concurrent comparison group
or exposure and outcome measured in a single group at the same point in time
Given the nature of mass media, it is difficult to establish true control groups. Generally, a
quasi-experimental alternative is to match communities and designate one as a treatment and one
as a control. Hornik (1996) argues that such designs are inappropriate for these efforts. He even
suggests that the better designed the evaluation is, the worse the evidence for important effects.
Contrasting the Stanford Five Community Project with the High Blood Pressure Education
Effectiveness of Health Campaigns
8
Program illustrates this point. The Stanford study was a tightly controlled quasi-experiment.
Theoretically, the two treatment communities received extensive education and mediated
messages about cardiovascular risk reduction over a five-year period contrasted to the control
communities who were not exposed to these materials but who, undoubtedly received similar
messages from other sources. The project estimated that the individuals in the control
communities received about five hours of education each year, divided among several major
behavioral objectives. The results showed only small treatment city advantages. Similar
patterns have been reported for other large matched community trials such as The Minnesota
Heart Health Program, the Pawtucket Heart Health Program, and the Community Intervention
Trial for Smoking Cessation (COMMIT). In all of these cases, there was a strong secular trend
in risk reduction resulting in significant change in the control communities. Hornik contrasts
these carefully controlled designs to the High Blood Pressure Program begun in 1972, which he
characterizes as a messy social diffusion effort that was highly successful. Underlying these
arguments is the need for a clear understanding of how one expects a mass-mediated health
communication campaign to work. If we see these campaigns as a discreet treatment such as a
pill that can be delivered or withheld, then the controlled randomized design might be
appropriate, but if the campaign is seen as a social process which may include deliberate
communication messages which, in turn, may diffuse and effect other media and a range of
institutions, then other designs are probably more appropriate. We need to turn out attention to
some alternative approaches such as natural experiments and correlated time series rather than
experimental and quasi-experimental designs.
Effectiveness of Health Campaigns
9
MEASUREMENT
Another issue that offers many challenges when evaluating health communication
campaigns is measurement. Assuming one begins with a sound logic model of how the
campaign is supposed to work, what should be measured and what is the unit of measurement?
Most evaluations measure individuals in some way, one’s attention, knowledge increase, attitude
change, or behavioral change. Yet Hornik’s concept of a messy diffusion effort would lead one
to measure many other forms of social change in addition to individual change. One could
examine the impact on the media’s agenda, on social and political policies and institutions
Because there is so little precedent for these kinds of measures, it might be helpful to begin with
a study of a successful effort such as the National High Blood Pressure Education Program and
document its impact on different parts of the social system.
Cost effectiveness is one attribute of a mass mediated campaign that is seldom measured
yet would be very useful to campaign planners. For example, Hu and colleagues (1995)
calculated that in California a $20,000,000 media campaign produced a decline of about
232,000,000 packs of cigarettes smoked over a two year period, or about $1 per 11 packs not
smoked. Over time, their estimates suggest that each 10% increase in media time purchased
would produce an additional decline of 0.5% in cigarette sales. As more of these campaigns
begin to use paid media, these kinds of measures will become even more important.
Another aspect of measurement in any kind of behavior change effort is whether to
depend on self-reported behavior or to actually observe behavior in some way. In general, we
have probably depended on self-reported behavior too much. There are some behaviors that
cannot ethically be observed such as sexual behavior but there are sometimes creative ways to
Effectiveness of Health Campaigns
10
assess the validity of the self-report data. For example, comparing self-reported use of condoms
with actual condom sales. In the smoking area, a subset of self-report data are often validated
with physiological data or reports from significant others. We need to continue to find ways to
establish the credibility of the measures we use to track health status and effectiveness of our
interventions.
RESOURCES FOR EVALUATION
A third and very pragmatic issue around evaluation of mass mediated health campaigns is
when to evaluate and what proportion of the overall budget to spend on evaluation. Program
planners are often reluctant to conduct outcome or impact evaluations because they want to use
all of their resources to reach as many people as possible with the intervention. Others decide or
are forced to spend a portion of their budgets on evaluation but then must decide how to divide
these resources between formative and summative evaluation. It would be very helpful to have
some guidance on the recommended portion of program funds to spend on evaluation and how to
distribute those funds between formative and summative evaluation.
CONCLUSION
Mass media health campaigns clearly can be an effective tool for health promotion
whether the effort is on a national or local scale. We should stop arguing whether they are more
or less effective than other strategies or whether one channel is better than another. Instead we
should carefully formulate our conceptual model of how we expect an intervention to work and
then evaluate it accordingly. Health promotion interventions are not like pills – they are much
more complex and indirect in the way they work. Therefore our evaluation designs may be very
different allowing us to track a social influence process and document its effects on social and
Effectiveness of Health Campaigns
political institutions as well as on individuals.
11
Effectiveness of Health Campaigns
12
REFERENCES
Flay, B. R. (1987). Selling the smokeless society: Fifty-six evaluated mass media programs and
campaigns worldwide. Washington, D. C.: American Public Health Association.
Freimuth, V. S. & Taylor, M. (1995). Are mass mediated health campaigns effective? A review
of the empirical evidence. Paper prepared for the National Heart, Lung and Blood
Institute.
Hornik, R. (1993). Public health education and communication as policy instruments for
bringing about changes in behavior. Paper prepared for meeting on behavioral and social
factors in disease prevention.
Hornik, R. (1996). Public health communication: Making sense of contradictory evidence.
Paper prepared for meeting on evaluation in public health communication.
Hu, T. W., Sung, H.Y, & Keeler, T. E. (1995). Reducing cigarette consumption in California:
tobacco taxes as an anti-smoking media campaign. American Journal of Public Health,
85, 1218-22.
Joseph, J. G. (1988). HIV infection and the effectiveness of education for the “General
Population”. Paper prepared for the Office of Technology Assessment.
Lau, R., Kane, R., Berry, S., Ware, J., & Roy, D. (1980). Channeling health: A review of the
evaluation of televised health campaigns. Health Education Quarterly, 7, 56-89.
Maccoby, N., Farquhar, J. W., Wood, P. D., & Alexander, J. (1977). Reducing the risk of
cardiovascular disease: Effects of a community based campaign on knowledge and
behavior. Journal of Community Health, 3, 100-114.
McGuire, W. J. (1989). Theoretical Foundations of Campaigns. In R.E. Rice & C. K. Atkin
(Eds.), Public Communication Campaigns (2nd ed., pp. 43-65). Newbury Park, CA:
Sage Publications.
Partnership for Drug-Free America. (1990). What we’ve learned about advertising. American
Association of Advertising Agencies.
Reardon, K. K. (1988). The role of persuasion in health promotion and disease prevention:
Review and commentary. In J. A. Anderson (Ed.) Communication yearbook 11. (pp.
277-297), Newbury Park, CA: Sage.
Effectiveness of Health Campaigns
13
Redman, S., Spencer, E. A., & Sanson-Fisher, R. W. (1990). The role of mass media in
changing health-related behavior: a critical appraisal of two models. Journal of Health
Promotion of Australia 7(2), 91-99.
Rogers, E. M. & Storey, J. D. (1987). Communication campaigns. In C. Berger and S. Chaffee
(Eds.) Handbook of communication science. Newbury Park, CA: Sage.
Solomon, D. S. (1982). Health campaigns on television. In D. Pearl, L. Bouthilet, and J. Lazar
(Eds.) Television and behavior. Washington, D. C.: NIMH Technical Reviews.
U. S. Department of Health and Human Services. (1986). Technical Report Cancer Prevention
Awareness Survey Wave II. Bethesda, MD: National Institutes of Health.
Woods, D.R., Davis, D., & Westover, B.J. (1991). “American Responds to AIDS”: Its content,
development process, and outcome. Public Health Reports, 106(6), 616-622.