IOWA STATE UNIVERSITY
OF SCIENCE AND TECHNOLOGY
Statistical Laboratory and
Department of Statistics
Snedecor Hall
Ames, Iowa 50011-1210
515 294-3440
FAX 515 294-4040
March 8, 2009
Re: Research Basis
I am writing to inform you of my knowledge of evidence-based research regarding the Rock In
Prevention program, and to update you on the National Registry of Evidence Based Programs and
Practices (NREPP) application process for Rock In Prevention. Regarding NREPP, a Federal Register
Notice is anticipated in June, 2009, announcing the reopening of submissions for NREPP and listing any
new requirements. When applications reopen (October 1, 2009 through February 1,2010), the Iowa State
University's Institute for Social and Behavioral Research will submit an application for Rock In
Prevention.
The frequencies and regression analysis identify that both Rock In Prevention PLUS and Rock In
Prevention LNE are effective in reducing substance use and increasing the perception of harm, in
comparison to the control group.
Documented scientific evidence shows that both Rock In Prevention LNE and Rock In Prevention Plus
are effective in reducing the use of the four targeted substances: alcohol, tobacco, marijuana, and
inhalants. Iowa State University, Institute For Social and Behavioral Research will continue to study
Rock In Prevention outcomes.
I am a Full Professor currently in the Departments of Statistics and Political Science, and Director of the
Public Policy and Administration Program, at Iowa State University (ISU) and hold the titled rank of
University Professor. In addition, I am affiliated with ISU's Institute for Social and Behavioral Research
(http://www .isbr. iastate .edul).
From 2003-2007, I served as Director of the Research Institute for Studies in Education (where I also was
Coordinator of Research from 1999-2003), and from 1999-2007, was a Professor in the Department of
Educational Leadership and Policy Studies, all at Iowa State University. I have served as co-editor of the
Policy Studies Journal (1993-2002), was a member of the Editorial Advisory Board for TESOL Quarterly
(2003-05), and currently am an Associate Editor of the Journal of Information Technology & Politics .
.
My 30 years of research, external funding, and teaching focus on statistical methods and their applications
to public policy and program evaluation, with emphasis on evaluation of education programs.
I have received funding from numerous federal agencies, state agencies, local government, and other
organizations, including the:
• National Science Foundation
•
•
•
•
United States Department of Education
Urban Mass Transportation Administration
Iowa Department of Education
Iowa Department of Public Health
•
•
•
Des Moines Independent Community School District
Iowa Department of Public Health
Center for Substance Abuse Prevention
•
•
•
•
•
•
Administration for Children and Families of the U.S. Department of Health and Human Services
Iowa Department of Economic Development
City of Des Moines
Iowa Board of Regents
Pew Foundation
Iowa Association of School Boards
•
American Judicature Society
Publications include 10 books (several in multiple editions), 19 book chapters, 88 journal articles and
refereed proceedings, and over 200 other publications. I am lead editor of the recently-published National
Science Foundation-funded book, Gold Standard(s) of Quality Research in Science Literacy: Science
Education, Reading, Statistics, and Other Adventures in Science-Based Research (New York: Springer).
I have presented and engaged in other roles (including panel chair, organizer, and discussant) at over 300
professional meetings. I serve regularly as a statistical consultant for researchers, administrators, program
staff, and students, and have received awards for research, teaching, and professional practice.
My full vita is available, together with other relevant information, at my homepage for Statistics
(http://www .stat. iastate.edu/ directory/personal. php ?id=mshelley) and Political Science
(http://www .pols.iastate.edu/shellev .shtml).
Introduction
Researchers at Iowa State University are continuing to study Rock In Prevention outcomes, will work on
future journal articles based on those results, and are completing the NREPP process. There are two
styles of Rock In Prevention, both of which were studied through a grant from the Substance Abuse and
Mental Health Services Administration (SAMHSA) and the Center for Substance Abuse Prevention
(CSAP), among the leading federal authorities for scientific, evidence-based, prevention programs.
I will share the research design and scientific evidence that documents both Rock In Prevention LNE
and Rock In Prevention PLUS as evidence-based and effective programs. Frequencies and data analysis
using regression methods identify that both Rock LNE and Rock PLUS are effective in reducing
substance use and increasing the perception of harm, compared to the control group. Documented
scientific evidence shows that both Rock LNE and Rock PLUS are effective in reducing the use of the
four targeted substances, alcohol, tobacco, marijuana, and inhalants.
Rock has two components, both of which utilize outcomes-driven, evidence-based practices and music
as a teaching method-which
is what compels kids to listen and learn. Rock includes visual, auditory,
and kinesthetic teaching techniques.
Rock LIVE has served more than 1.2 million students with empirically-based research and best
practices. These include a live, in-school or on-site, educational workshop that provides
educational messaging, social dramas, and interaction among young students, their teachers, and
trained high school mentors. Education materials include five-week teacher classroom follow-up
guides, and family follow-up guide CDs for teachers and families to reinforce the positive
messaging and empowerment tools. Also used are evaluations for high school mentors and peer
helpers and for the participating elementary and middle school students.
2
Rock PLUS typically launches with the Rock LIVE workshop and its materials. Trained high
school mentors also assist. Rock PLUS includes an evidence-based 12-week elementary
classroom curriculum for teachers and/or counselors that comprehensively address issues vital
to developing healthy choices. The curriculum is based on the national standards and guidelines
for effective practice established by the Collaborative for Academic, Social, and Emotional
Learning (CASEL).
The curriculum meets educational goals in several disciplines:
• Personal and social skills
• Academic achievement
• Parental involvement
•
•
•
•
•
•
Career development
Self-awareness
Health and wellness
Alcohol
Tobacco, and drug awareness
Character education
•
•
•
•
•
Media literacy
Anger management
Problem solving
Violence prevention
Positive school climate
Rock PLUS curriculum is implemented at level I in Grades 3-4, level II in Grades 4-5, and
level III in Grades 5-6. There is an Instructor Manual Kit and Student Workbook Kit for each
level. Each kit contains a CD of the songs that are incorporated as teaching tools. The Rock
PLUS curriculum contains the lesson objective, rationale, materials, discussion questions,
song lyrics, and in-class activities.
A school-home link integrates the weekly lesson, with parents and children working together.
Parents sign and kids return the completed school-home link. This provides data regarding
parental involvement in their children's education. Educational research documents that
parental involvement does help to improve academic achievement. These assignments give
families the tools to begin vital conversations about: anger control, drug awareness, media
influences, making good choices, responsibility, honesty, empathy, self-esteem, problemsolving, and stress management. Rock PLUS also conducts student, parent, and teacher
written evaluations.
The ISU Research Proiect
The purpose of the research project was to measure the effectiveness of Rock In Prevention LIVE and
Rock in Prevention Plus, in comparison with a Control Group (with no treatment). The specific objectives
accomplished by the research plan are to:
• Determine the effectiveness (compared to a no-treatment control condition) of the two treatments
in the prevention, in delaying the onset and reducing the level of substance use by elementary
school students and the high school students who helped deliver the program;
• Measure and document reductions in substance use and other negative outcomes related to
substance use among participants; and
• Determine the effectiveness of the two treatments in decreasing risk factors for substance use and
increasing protective factors that delay or prevent substance use.
3
The study included randomly selected school sites; pretest, posttest, and six-month follow-up substance
use and perception of harm survey instruments (Government Performance Results Act [GPRA] youth
surveys from the U.S. Department of Health and Human Services); and control schools. Each student
completed a pretest, posttest, and follow-up survey (six months after the end of the program).
The data demonstrate that the control group consistently showed increases in use across all four
substances and reduced levels of perception of harm. The students who received either Rock LNE or
Rock PLUS had lower levels of 30-day use and higher levels of perception of harm than the control
group. This helps to demonstrate that Rock interventions are effective.
The inclusion of a control (no treatment) group is required to conduct scientifically-based research, as
opposed to a simpler evaluation. Only scientific-based and approved research will allow programs
(including Rock PLUS and Rock LNE) to apply for model status or promising program status with the
U.S. Department of Health and Human Services, the U.S. Department of Education, or the U.S.
Department of Justice, which are the major federal funding agencies for substance abuse prevention.
The frequencies and regression analysis identify that both Rock LNE and Rock PLUS are effective in
reducing substance use and increasing perception of harm, in comparison to the control (no treatment)
group. These research findings offer evidence that will be elaborated when applications for Rock PLUS
and Rock LNE are submitted to the U.S. Department of Health and Human Services for inclusion as a
model program under the NREPP process.
Furthermore, the analysis of the control group provides evidence that school sites that do not utilize either
Rock LNE or Rock In Prevention PLUS have higher levels substance use, do not delay or prevent
substance use, and have decreased perception of harm, compared to schools that utilize either Rock
program. It is imperative to provide substance abuse prevention programs among this age group (9 to 13),
to delay substance use and increase perception of harm. The 2002 Iowa Youth Survey identified that
increased use of alcohol, tobacco, inhalants, and drugs, and decreased levels of harm perception, occur
among Iowa's youth from the 6th to the 12th grade.
Rock in Prevention LIVE/Rock in Prevention PLUS/Control
sth
grade* comparisons.
What is disturbing is that substance use begins to accelerate as Iowa's youth enter middle and high
school. The 2002 Iowa Youth Survey found that alcohol and tobacco use reported in the past 30 days
increased for alcohol from 5% in 6th grade to 15% in 8th grade and for cigarettes from 2% in 5th grade to
8% in 8th grade. And the rise in use accelerates in later grades; 43% of 11th graders reported alcohol use
and 23% reporting cigarette use in the last 30 days.
Prevention programs that have an impact on restricting the initiation of substance use, reducing the level
of substance use, and increasing the perception of harm are especially necessary for this age group, due to
increased availability, use by older peers, and middle school social pressure. The following data identify
that the control group (who did not receive Rock LNE or Rock PLUS, but may have been in another
program prior to the research) consistently showed increased use across all four substances and had
reduced levels of perception of harm. The students who received either Rock LNE or Rock PLUS had
lower levels of 30-day use and higher levels of perception of harm than the control group. These results
are based on a report released March 28, 2006. Most of the students were in the 6th grade when they
completed the follow up survey.
4
Graph A, for example, shows that among 5th graders, the percent who report drinking alcohol in
the past 30 days, increases from pretest to posttest to follow-up, but remains at a lower level for RIP
(Rock LIVE-the purple bars) and especially for RIPP (Rock PLUS-the
blue bar) participants than for
the control group.
Graph A: Percent drinking alcohol in past 30 days for
5th
graders
12
4610
028
Pretest
EJRIPP 5th grade
-RIP 5th grade
oControl 5th grade
o20021YS 6th grade
Posttest
Followup
Note: The Iowa Youth Survey IYS data are listed only at pretest as a comparison
to this project's
sample.
Graph B shows that 5th grade students' reported use of marijuana essentially disappears at posttest
and at followup for both RIPP (in purple) and RIP participants (in blue), but increases substantially for
students in the control group.
Graph B: Percent using marijuana in past 30 days for
stb
graders
2.5
Pretest
0.5 012
1.5
eRIPP 5th grade
_RIP 5th grade
oControl 5th grade
o 20021YS 6th grade
Posttest
Followup
Note: The Iowa Youth Survey IYS data are listed only at pretest as a comparison
5
to this project's
sample.
Also, Graph C shows a dramatic decrease from pretest to posttest to followup in the percentage of
RIP participants (the purple bars) who perceive no risk from smoking marijuana once or twice, as well as
a much lower level at followup for students in RIPP (the blue bars), compared to a steady increase from
pretest to posttest to followup for control group students.
Graph C: Percent with perception that there is no risk to smoking marijuana once or twice for
51h
graders.
9
8
7
6
5
IJ
RIPP 5th grade
• RIP 5th grade
4
3
o Control 5th grade
2
1
o
Pretest
Posttest
Followup
Note: The Iowa Youth Survey questions did not match the wording of this question; so no comparisons
were made for this question.
In summary, the frequencies and regression analysis identify that both Rock In Prevention PLUS and
Rock In Prevention LIVE are effective in reducing substance use and increasing the perception of harm,
in comparison to the control group.
Documented scientific evidence shows that both Rock In Prevention LIVE and Rock In Prevention Plus
are effective in reducing the use of the four targeted substances: alcohol, tobacco, marijuana, and
inhalants.
Research continues on these and other aspects of the impact of Rock In Prevention PLUS and Rock In
Prevention LIVE.
Thank you. I am happy to respond to any questions you may have.
k:~(
SW-Lle
Mack C. Shelley, II
University
Studies
Professor
d
of Statistics,
Political
.
Science,
Director of the Public Policy and Administration
Department of Political Science
and Educational
Program,
6
Leadership
and Director
and Policy
of Graduate
Education,