CORE VIPP EVALUATION:
EEP MEETING FEB 2012, DOCUMENT CONTENTS
This document contains 3 sections: (1) A table that includes the CORE BIC Performance Measures, possible indicators to assess if states are
reaching those measures, and questions from the current State of the States Questionnaire that link to the measures; (2) Section 22 of the States
of the States Questionnaire, and (3) an Evaluation Question Bank of State of the States questions that are not in Section 22 but are in other
sections of the questionnaire.
GOALS:
1) To generate new questions and/or modify existing questions on the State of the State Questionnaire to measure the state
health departments’ injury and violence evaluation capacity. The question items must:
1) Improve data collection;
2) Improve feedback and reporting; and/or
3) Align with the BIC performance measure indicators.
2) Where necessary, develop additional BIC indicators.
CORE Basic Integration Component (BIC) Performance Measure Indicators
Page 2
 This section shows the BIC Performance Measures that are listed in Funding Opportunity Announcement (FOA) and the
Program Consultants brainstormed indicators that elaborate on the performance measures. Safe State/SAVIR Evaluation
Team is using these Performance Measure Indicators as a guide in generating or strengthening questions that show capacity
building overtime. It is important that the 2011 State of the States Questionnaire captures appropriate BIC Performance
Measures Indicators. Please note that not all indicators will be best captured in the survey, but may be capture in other data
collection tools like progress reports, annual reports, etc.
We invite the EEP to brainstorm new indicators and evaluation capacity building questionnaire items that will align
with the listed performance measures and that will useful in the BIC evaluation and reporting. Blank spaces are
provided for new entries for both BIC Performance Measures Indicators and Questionnaire Items.
State of the States Questionnaire – Section 22: Prevention Programs and Evaluation
Page 4
 This section is designed to capture the health departments’ programmatic and evaluation activities for each injury and violence
focus area. The questions presented are updated items from 2009 State of the States. We would like for the EEP to comment
and recommend changes to the question items that will strengthen and improve the data collection and reporting of the
question items and (if appropriate) capture important indicators that are related to the BIC performance measure indicators
(page 6).
Note: Items in blue font were asked in previous State of the States Questionnaires and reported in aggregate to
national state health departments.
Evaluation Question Bank
Page 7
 This section shows two question items that we would like to introduce into Section 22: Prevention Programs and Evaluation.
We would like for the EEP to recommend new question items that will 1) strengthen the data collection and reporting within this
section; and 2) capture important indicators that are related to the BIC grantees performance measure indicators (page 6).
1
BIC PERFORMANCE MEASURES INDICATORS & BRAINSTORMED QUESTIONNAIRE ITEMS
[Evaluation Capacity Area]
Goal /
Performance Measure
BIC Indicator
Time
3. Support and Evaluate Program and Policy Interventions / 5. Program Evaluation
YEAR 1
3.y1.1
Used surveillance findings and literature
Development of recommendations on most
review to inform decision making about the
appropriate best practice or evidence based
use of best practices for injury and violence
strategy
prevention
Evidence of substantial support for evaluation of 4
identified strategies
5.a1
Used surveillance findings to guide injury
prevention and control activities
3.y1.2
Through the ICPG: Selected four strategies
(two program interventions and two policy
strategies) that are best practice or evidencebased and align with priorities identified by the
ICPG
Determination of 4 best practice strategies (at least
two policy strategies)
3.y1.3
Through the ICPG: Developed a thorough and
specific plan for implementation for at least
two topic specific program interventions and
two policy strategies
Creation of a specific implementation plan for
selected strategies, including measureable goals
and objectives with timeframes
Through the ICPG: Developed a thorough and
specific evaluation plan
Evaluation plan
Health Impact measures
Development of objective/quantitative measures of
effectiveness
Specific strategies identified by plan
ICPG partner roles and responsibilities defined
SHD roles and responsibilities defined (contribution)
3.y1.4
Brainstormed SOTS Questionnaire Items Related
to Evaluation Capacity
*22.3 <weak>
2
BIC Performance Measures (continued)…
Goal /
Time
Performance Measure
BIC Indicator
3. Support and Evaluate Program and Policy Interventions / 5. Program Evaluation
YEARS 2- 5
3.[y2-y5].1
Collected and analyzed data to evaluate the
Health Impact measures
effectiveness of intervention programs and
Impact Assessment
policy strategies
Focus of SHD Injury Program evaluations
Data used to support evaluation; Baseline HI data
reported in Year 1; Annual HI data reported in
Years 2-5
3.[y2-y5].2
Used existing CDC evaluation framework
and other tools
Used existing CDC evaluation framework and other
tools
Brainstormed SOTS Questionnaire Items Related
to Evaluation Capacity
*22.4
*22.5
*22.4
Evaluation plan
5.y1.1
Used existing CDC evaluation framework
and other tools in the development of a logic
model. Links:
3.[y2-y5].3
5.a2
Used evaluation findings to make changes in
programs, evaluations, and policies
ANNUALLY
3.a1
Worked with CDC-appointed technical
5.a4
advisor and CDC-named evaluation
contractor
 Was a logic model developed as part of your
program evaluation? [Yes/No/Don’t Know]
 If yes, did you use the CDC Evaluation Framework
to develop the logic model? [Yes/No/Don’t Know]
Use of SHD Injury Program evaluation results
Updated action plans; Submission of Annual Data
report
 Are you using the results of the evaluation findings
to make changes to your program and strategies?
Technical assistance reports
3
2011 STATE OF THE STATES QUESTIONNAIRE
22. PREVENTION PROGRAMS AND EVALUATION
This section primarily focuses on the areas of injury/violence addressed by IVPP programs in 2011.
Questions marked with an asterisk (*) require an answer.
22. PREVENTION PROGRAMS
*22.1. This question attempts to assess the level of effort the state IVPP gives to each injury/violence area, whether or not it is lead by or located
within the state IVPP.
Please select from the drop down menu to indicate the level of program focus during 2011 in each injury/violence area that the state IVPP
addresses through its programs and activities.
Choices include primary focus, secondary focus, minimal focus, or did not focus on in 2011. You can choose multiple areas of primary or
secondary focus.
For ease of data entry, this question continues with additional Prevention Programs in Question #2 below.
Part 1
All Terrain Vehicle (ATV) injury
Program Focus
 Primary Area of Focus
 Secondary Area of Focus
 Minimal Focus on this Topic
 Did not address in 2011
Child abuse/neglect
Child passenger safety
Domestic/intimate partner violence
Elder Abuse
Fall injuries
Fire and burns injury
Firearm injury
Homicide
Mass trauma/disaster-related
Motor vehicle injury
Motorcycle/motorized scooter injury
Occupational injury
4
2011 STATE OF THE STATES QUESTIONNAIRE
*22.2. CONTINED FROM QUESTION #1: This question attempts to assess the level of effort the state IVPP gives to each injury/violence area,
whether or not it is lead by or located within the state IVPP.
Please select from the drop down menu to indicate the level of program focus during 2011 in each injury/violence area that the state IVPP
addresses through its programs and activities.
Choices include primary focus, secondary focus, minimal focus, or did not focus on in 2011.You can choose multiple areas of primary or
secondary focus.
Part 2
Pedestrian injury
Program Focus
 Primary Area of Focus
 Secondary Area of Focus
 Minimal Focus on this Topic
 Did not address in 2011
Poisoning
Rural/agricultural injury
School-based injury
Sexual assault/rape
Spinal cord injury (SCI)
Submersion injuries/drowning
Suicide/self-inflicted
Suicide attempts
Teen Dating Violence
Traumatic brain injury (TBI)
Injuries to children
Injuries to adolescents
Injuries to elderly
Injuries to a racial/ethnic group
Injuries to gender-specific group
Other #1
Other #2
Other #3
Other #4
Other #5
5
2011 STATE OF THE STATES QUESTIONNAIRE
*22.3. For the primary areas of focus you indicated in the previous questions above (#1 and #2), please select how the IVPP determined that those
areas of focus were primary. Choose all that apply.
 Data
 Funding directives
 Needs assessment
 Political influence
 State mandates
 Other (please specify)
 2011 State of the States Questionnaire
*22.4. In questions #1 and #2 above, you indicated the areas of primary focus for your state IVPP in 2011. Please tell us more about how you
evaluated activities in these primary focus areas. For the areas indicated as primary focus in 2011, for how many of them did you complete the
following types of evaluation? The total number entered for each type of evaluation should not exceed the total number of primary focus areas.
For example, if you had four (4) areas of primary focus in 2011, the total number of formative evaluations completed should not be more than four
(4).
o
o
o
o
o
Definitions are provided below for each type of evaluation:
Formative Evaluation – testing program plans, messages, materials, strategies, or modifications for weaknesses and strengths before they
are put into effect
Process Evaluation – testing whether the program’s procedures for reaching the target population are working as planned
Impact Evaluation – assessing the program’s progress toward its goals (i.e., measuring the immediate changes brought about by the
program in the target population such as changes in knowledge, attitudes and beliefs that may lead to changes in health behavior)
Outcome Evaluation – measuring whether your program met its ultimate goal of reducing morbidity and mortality due to injury
Citation: Thompson NJ, McClintock HO. Demonstrating Your Program’s Worth: A Primer on Evaluation for Programs To Prevent Unintentional
Injury. Atlanta: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, 1998.




Formative
Process
Impact
Outcome
*22.5. Did your state IVPP use the CDC "Framework for Program Evaluation in Public Health" to plan and implement program evaluations in 2011?
 Yes
 No
*22.6. Does your IVPP have access to a program evaluator?
 Yes, within IVPP program
 Yes, within state health department
 Yes, by consultant
 No
 Other (please specify)
6
EVALUATION QUESTION BANK
(These questions appear on the SOTS in other sections, and we can consider adding them to the evaluation section.)
*25.2. How were these programs or activities evaluated in 2011? Check all that apply.
 Number of participants enrolled
 Pre- and post-surveys of participants
 Emergency department or hospital admissions
 Media reports
 Policy changes
 Program costs
 Other (please specify)
*28.2. What outcomes did you measure to evaluate these programs and activities in 2011? Please check all that apply.
 Process indicators (e.g. number of participants, number of materials distributed, etc.)
 Anecdotes/informal measures (e.g., word of mouth; interviews)
 Institutional/policy change
 Media reports/presence
 Likeability/acceptability
 Knowledge gain/retention/refresh
 Behavior change/maintenance
 Long term outcomes (health impact measures like injuries, hospitalizations, fatalities, etc.)
 Policy change
 Environment change (e.g. new crosswalk, new program, home modifications, etc.)
 None
 Other (please specify)
7
8