Changes to the Core Measures

advertisement
Many grantees generously provided input to the Drug Free Communities (DFC) Support Program
National Evaluation team concerning how COMET might be made less burdensome and more user
friendly. In December, 2011, substantial changes were approved for COMET and for the DFC Core
Measures. These changes are intended to acquire useful data for evaluating outcomes of the program as
well as to facilitate the ease with which grantees enter data. This email introduces you broadly to those
changes, provides a timeline overview and lets you know about next steps. The goal is to assist all DFC
grantees in making this transition as smoothly as possible. Grantee commitment to a timely transition to
these changes is necessary in order to successfully meet all grant requirements. While grantees must
address changes related to the DFC National Evaluation, you are encouraged to continue to collect any
additional data that your coalition thinks appropriate to best evaluate work at the local level.
Changes to the Core Measures
Attached is a sample survey for collecting the DFC Core Measures. Many DFC core measures remain the
same, but several have been modified somewhat. Following is a summary of these changes.

Added prescription drugs as a core substance: The primary change to core measures reporting
is the addition of prescription drugs as a core substance. Although this is not a change in the
core measures, per se, it does necessitate the addition of core measures questions to the
survey. The focus for this core substance is on the use of prescription drugs not prescribed to
the youth.

Perception of Risk Core Outcome
o
Changed perception of risk for alcohol to a measure of binge drinking: This core
measure now focuses on perception of the risks of binge drinking (i.e., having five or
more drinks of an alcoholic beverage once or twice a week). Because the perception of
risk of regular alcohol use (i.e., nearly every day) is a key legacy metric for STOP Act
grantees, STOP Act grantees will continue to be required to input this data into COMET.
o
Changed perception of risk for marijuana use to define regular use: The DFC core
measure now defines regular use of marijuana as at least 1-2 times per week.

Added core measure on perception of peer disapproval and, for alcohol, an item on attitude
toward peer use. By including perceptions regarding disapproval of substance use in peers, the
DFC evaluation will be better prepared to explore relationships between use and community
context. Peer disapproval will be framed in the following way: “How wrong would your friends
feel it would be for …”. For alcohol only, STOP ACT grantees are also required to ask, “How do
you feel about someone your age having one or two drinks of an alcoholic beverage nearly every
day?”

Dropped age of first use questions. Age of first use is no longer a DFC core measure and is also
no longer a GPRA measure for STOP Act grantees.

Submit Grantee survey for review and approval. Again, many of the core measures remain the
same. For example, grantees should continue to measure past 30-day use as they have done in
the past, with the exception of adding an additional item regarding prescription drugs. Attached
to this message is a sample of the minimum survey needed to meet the new DFC core measures
requirements. Grantees should also begin to do a self-review of their current survey to
determine compliance relative to the new core measures. ICF will be providing a revised survey
1
review guide to grantees to assist in this process. Many of you may use national or state surveys
that already have items that meet the revised core measures. For example, many existing
surveys (e.g., Pride, CTC, Monitoring the Future, ADAS) already have questions on 30-day use of
prescription drugs. However, in many cases, it is likely that you will need to adjust your data
collections to comply with all core measures requirements.
To ensure data quality, all grantees are required to submit to ICF (DFC_Evaluators@icfi.com)
the survey they will be using to report on the DFC core measures in order to have the survey
approved. Grantees are encouraged to submit their survey as soon as they have completed any
needed changes. ICF is committed to providing feedback on surveys quickly. Grantees should be
prepared to wait for as long as three weeks for this feedback if the survey is submitted during a
high volume time. The sooner your coalition begins this process, the less likely delays will be
faced in any planned data collections.
Changes to COMET
Several changes to COMET were made in order to respond to grantee needs, as follows. Once changes
are complete, grantees will be sent a mock-up of the new system so data collection can begin.

Survey modifications: Subtle changes to several of the surveys in order to shorten their length
(e.g., questions that appeared to be duplicated were eliminated). While this is not a change to
COMET per se, grantees may want to be aware of this.

COMET format: COMET has been shortened and simplified to ease burden while improving
data clarity and precision. The format for submitting activities data has been substantially
revised to ease use. By removing the linkages between goals, objectives, strategies, activities,
and outcomes, grantees are expected to see a significant savings in time while maintaining the
rigor of the evaluation. Formats for reporting have also been reworked.
Timeline
ONDCP requires that all grantees be in compliance with the new core measures by no later than the
February 2013 reporting period. Some grantees may be prepared to make a transition to the new core
measures as early as the May 2012 or August 2012 data submission, and are encouraged to do so
whenever possible. For grantees who will be submitting data in May or August 2012, and who have
already collected that data, you will be able to indicate whether you used the old core measures or the
new ones. If you have not yet collected data, you are strongly encouraged to collect data that reflects
the new core measures. New grantees will enter their first report in COMET in May 2012, and continuing
grantees will enter their next report in COMET in August 2012. Following the August 2012 reporting
period, all grantees will move to a February/August reporting schedule, with the February COMET report
doubling as the continuation application.
In the new COMET system, grantees will continue to have access to their previous COMET data through
their submitted progress reports and a data export tool. During the transition to the new system,
access to the current COMET system will be removed for all grantees. KIT Solutions and ICF are
developing a training to update grantees on changes. Those trainings will be held in spring 2012 (starting
with new grantees), with additional trainings occurring over summer 2012. A schedule of trainings will
be sent to grantees as soon as possible. Once the new system is launched and the grantee system users
participate in the training, COMET access will be re-granted. If temporary COMET access is needed
2
during the interim period, grantees should contact DITIC Support. In addition to trainings, the revised
COMET will have more embedded. detailed guidance on how to report coalition activities. KIT Solutions
obtained preliminary feedback on the new system during the New Grantees Meeting and is making
additional changes based on that feedback. Once complete, grantees will be sent a mock-up of the new
system so data collection can begin with the new measures.
Next Steps and Final Thoughts: What Should Grantees Do?
Grantees should review this information and communicate any questions to DFC_Evaluators@icfi.com
as soon as possible. Hopefully, grantees will be pleased to see adjustments that have been made in both
the core measures and in COMET. Still, we know that even when change is wanted, it can be challenging.
Please do not hesitate to ask if you have any questions – we are here to help! ICF will be providing
additional information as quickly as possible. Thank you in advance for your help in making these
changes, and we look forward to working with you through this challenging but exciting process.
You can reach ICF International, the DFC National Evaluation team at DFC_Evaluators@icfi.com or at
1-877-854-0731 (Monday-Friday).
You can reach KIT Solutions (DITIC Support) at DITICSupport@kitsolutions.net or at 1-888-DITIC-4-U
(1-888-348-4248).
3
Download