Making it Last…
The Marriage of Social Marketing and Evaluation
Social Marketing and Evaluation Webinar
for the 2006 Newly Funded Communities
July 10, 2007
2006-07 System of Care Start-up Webinar Series
Social Marketers Are
From Mars
Evaluators Are
From Venus
2006-07 System of Care Start-up Webinar Series
Social Marketers Are From
Mars…





Like simplicity
Comfortable with generalities
Goal-oriented
Like to see quick results
Prefer short, straightforward
presentations
 Risk-takers
Evaluators Are From Venus




Like complexity
Deal in absolutes
Process-oriented
Like to take time, explore
all angles
 Prefer lengthy, abstract reports
 Risk-averse
Mars and Venus Are Both Planets in the
Same Universe
Want to help children:
 Stay in school
 Stay at home
 Become more productive members of
society at a cost that helps sustain
programs in communities
2006-07 System of Care Start-up Webinar Series
Challenges and Barriers to a Happy,
Productive Relationship
Social Marketers
 Want interim
data to report
 Want easy to
communicate
info
 Have difficulty
interpreting/
identifying
alone
Evaluators
 Want to focus on
long-term
outcomes
 Want data
communicated
accurately
 Uncomfortable
passing along
early results
2006-07 System of Care Start-up Webinar Series
Why Cooperate At All?










Presentation of outcomes is key to sustainability
Increase the public’s knowledge and awareness of an issue,
problem or solution
Influence perceptions, beliefs and attitudes that may
change social norms
Increase your organization’s visibility as a resource on
children’s mental health
Enhance your organization’s leadership position
Strengthen relationships
Form a united front for projecting and reinforcing messages
Create policy change
Increase demand or support for community-based,
youth/family-driven, and culturally competent mental health
services
Truly make a difference in the lives of children and families in
your community
2006-07 System of Care Start-up Webinar Series
Why Cooperate At All?

Evaluators have the information:

Descriptive/Outcome Data

Local Wraparound Fidelity Index

Service Experience Data

System of Care Assessment

Data Profile Reports

Continuous Quality Improvement Reports
2006-07 System of Care Start-up Webinar Series
Why Cooperate At All?

Social marketers know how to present it:

Audience Research

Communication Strategies and Tactics

Packaging Data
2006-07 System of Care Start-up Webinar Series
Can This Marriage Be Saved?
2006-07 System of Care Start-up Webinar Series
Understanding Is the Key

What are roles?

What is the language?

What are the desired outcomes?
2006-07 System of Care Start-up Webinar Series
Role of the Evaluator








Understand program goals
Analyze ongoing/post-program needs
Design evaluation that captures pertinent
data
Conduct evaluation; refine where
necessary
Monitor other data sources; compare and
contrast
Communicate regularly with staff across
program, particularly social marketers
Report interim and final results
Empower families and youth to be
persuasive advocates
2006-07 System of Care Start-up Webinar Series
Role of the Social Marketer







Understand program goals
Devise strategic communications plan
Identify and profile key audiences
Monitor changes and trends in attitudes
and behavior
Share challenges and successes with
audiences
Revisit and refine plan
Communicate regularly with staff across
program, particularly evaluators
2006-07 System of Care Start-up Webinar Series
Role of Family and Youth







Distinguish if the data reflects their
community
Help social marketers package the data,
making it accessible and relevant
Inform message development
Complement data products with
descriptive testimonials
Prevent dissemination of potentially
misleading data
Provide the bridge from numbers to lives
Promote retention and recruitment for
studies
2006-07 System of Care Start-up Webinar Series
What Can You Offer Each Other?
Social Marketers
 Understanding of
audience needs
 Suggestions for
simpler language
 Clearer
communication of
statistical info
Evaluators
 Accurate, timely
data on varied
topics
 Interpretation of
results
 Monitoring and
analysis of data
from outside
sources
2006-07 System of Care Start-up Webinar Series
If We’re From Different Planets, Do We
Need an Interpreter?
Social Marketers
 Audience
segmentation
 Editorial board
 Actuality
Evaluators
 Informed consent
 Program theory
model
 Longitudinal
statistical
analysis
2006-07 System of Care Start-up Webinar Series
Breaking Down the Language Barrier

Eliminate jargon where possible

Define terms

Ask questions
2006-07 System of Care Start-up Webinar Series
Making Beautiful Music
Together
2006-07 System of Care Start-up Webinar Series
National Example

Collaboration between the National
Evaluation Team and the Children’s
Campaign has resulted in many products
including:
 Fact sheets

PowerPoint presentations

Awareness Day data
2006-07 System of Care Start-up Webinar Series
Challenges

To balance audience-friendly language
with accurate data
2006-07 System of Care Start-up Webinar Series
Lessons Learned

Dialogue can start from either team

Begin working together at the very beginning of development

Collaborate on the development of a prototype

Create a timetable which includes time for multiple revisions

Learn about the other team's development process

Pose evaluation questions several different ways for better
understanding

Keep the dialogue going!
2006-07 System of Care Start-up Webinar Series
Questions?
Please press *7 on your phone
to unmute your line. When you
are finished, please press *6 to
remute.
2006-07 System of Care Start-up Webinar Series
Local Example:
Family Voices Network of Erie County
We are in our 3rd Year – grant funded in
October 2004.
 We have an informal process for
communicating with evaluation– we’re
located in the same building and maintain
regular contact with the families. Sometimes
this is a dual role between social marketing
and evaluation staff.
 We share data with our parents and
youth.

2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Social Marketing & Evaluation Team



Many times the social marketer and the evaluator
wear dual hats.
We review data with our parents and youth and
listen to their suggestions on how to make data
family-friendly.
We inform the policy-makers of our findings
based on the data collected bi-weekly at our
Management Team meetings.
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Project Samples
Community Assessment for Family
Involvement:

Knowledge & Effectiveness Survey
Children’s Mental Health Awareness Day:

NCMH Awareness Survey
Wraparound Bulletin:

Our quarterly newsletter
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Project #1: Knowledge and Effectiveness Survey
The 3 purposes of our Community Survey:

The purpose of this survey is to identify community knowledge and attitudes
and to provide marketing assessment and direction including stigma
evaluation.

The social marketing team presented the concept to the evaluation team and
they worked together to develop a survey that would serve as a baseline of
where our community stands today. By doing so, we have a measuring point
to see improvements over time.

Between the evaluation team and the social marketing team, we identified
our audience to be surveyed and worked together on identifying what we
wanted to measure.
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Project #1: Knowledge and Effectiveness Survey
Category
(as defined by survey code)
Total N sent
N Returned
Rate of
Return
Care Coordinator
38
23
61%
Child Psychiatrist
2
1
50%
Family Medicine MD / DO
153
16
10%
Pediatrician MD / DO
152
29
19%
Judge
5
0
0%
Juvenile Justice
3
1
33%
Psychologist
81
18
22%
Supervisor CC
7
5
71%
Vendor
466
82
18%
Psychiatrist
19
1
5%
TOTAL
926
176
19%
Family Voices Network of Erie County:
Project #1: Knowledge and Effectiveness Survey
We then rolled-up the data into four
categories:
1. Families
2. Care Coordinators
3. Supervisors
4. Service Providers (vendor network)
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Project #1: Knowledge and Effectiveness Survey
We asked our families to tell us what was important to
them from the survey categories. They identified:
1. Wraparound process/care coordination
2. The family role
3. Moving from system to natural supports
4. Making a referral
5. Child & Family Team (CFT) process
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Project #1: Knowledge and Effectiveness Survey
Next Steps:
• We will work with the evaluation team to create
communications plans that address the needs of
the family in the priority order they have
identified.
• The evaluation team will provide us with other
data from care management and other survey to
assist in our plan development.
You can view the survey form on our Web site:
http://www.familyvoicesnetwork.org//en/files/
Forms/Knowledge-Effectiveness-Survey.pdf
2006-07 System of Care Start-up Webinar Series
Family Voices Network of Erie County:
Project #1: Knowledge and Effectiveness Survey
Local Example:
Monterey County’s System of Care

Mostly agricultural county, in many areas
60-75% Hispanic

Lead Agency: Monterey County Health
Department, Children’s Behavioral Health
(about 125 staff)

Social marketing and evaluation are led by
outside consultants
2006-07 System of Care Start-up Webinar Series
To achieve optimal outcomes
from social marketing & evaluation efforts
both need to be
And
recognized as integral
ensures that
parts of all other
programs have
SOC activities.
This allows
the social
marketer to
influence data
collection from
a creative and
audience
perspective;
Ensures that
social marketing
activities
have a built-in
evaluation
component;
achievable outcome
measures & a clear
identity &
communication
plan.
Program staff,
social marketing,
evaluation, families, youth
& collaborating agencies work
together from the very beginning –
starting with the development of the
social marketing plan, program
logic models and design of
outcome measures.
Monterey County’s System of Care:
Strategy One: Bring Everyone to the Table
The Local Evaluation Team
Family & Youth
Cultural Competency
Behavioral Health
Probation
Members of
Data Management
the Local
Evaluation Team
bring
diversity & expertise
Social Marketing
to the monthly and
“as needed”
work sessions.
Social Services
Evaluation
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:
Strategy Two: Develop a Local Evaluation Framework





13 local evaluation measures
Developed local survey with
families and youth
State and national data
Program specific outcomes
Data from partnering agencies
How do we bring
partners together
to discuss shared
outcomes?
Monterey County utilizes a Comparison of Results Approach.
Seeks to answer the questions:
How are we doing?
Have outcomes for
families and youth
improved this year in
comparison to last year?
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:
Strategy Three: Develop a “Content Matrix” to Share
Evaluation Findings
Target
Characteristics
Audience
of Content
Information of
Interest
Format
Families
Bilingual, user-friendly.
Personal stories and
information. Avoid
acronyms and terms
that need further
explaining (e.g., 200%
poverty level).
Not interested in studies
and data collection
details.
Data relevant to their
child, e.g. “Last year in
Monterey County, 80% of
children diagnosed with a
behavioral disturbance
received a high school
diploma.”
Fact sheet or
newsletter with
limited text, simple
visuals, key facts
without details,
contact for more
information.
Policy
Makers,
Dept.
Directors
Limited time, prefer
“bottom line” facts and
visuals.
Outcomes, data sources,
analysis “What does this
mean to me and my
constituents?” What is
working, what isn’t?
Presentation,
briefing sheet,
Score Card.
Monterey County’s System of Care:
Example: Bilingual Newsletter Reaching Families
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:
Strategy Four: Develop a Distribution Plan and
Implement it!
Target
Audience
Format
Distribution
Who is
responsible?
Families
Newsletter
Mailed to 350 families; 25
copies each dropped off at 8
clinic locations.
Family Partnership
Program
Policy
Maker,
Dept
Directors,
Media,
Public
Briefing
Briefing scheduled for May 8th
as part of Children’s Council
meeting; Press release
Project Director
Staff
Staff Memo
Mailing (50 copies)
Social Marketer
Score Card
Presentation at all staff
meeting.
Article in staff
newsletter
Staff mailboxes (135 copies)
and email.
Scheduled for March staff
meeting.
Staff mailboxes and email
(article will go into February
issue).
Admin
Project Director
Social Marketer
Monterey County’s System of Care:
Example: Score Card
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:
Score Card 2005- At a Glance

Purpose: To highlight system-wide (shared) accomplishments
of Monterey County’s System of Care in a user-friendly way. To
show why systems of care are important to our community and
why it is important to continue to fund them.

Primary Target Audience: Policy-makers, Board of
Supervisors, Children’s Council, Governance Council, Community
Stakeholders and Staff (Language: Eng)

Distribution: Briefing to Children’s Council and media on
Children’s Mental Health Summit Day; Mailing
2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:
Summary of Strategies

Strategy One: Bring everyone to the table

Strategy Two: Develop a local evaluation framework
Strategy Three: Develop a content matrix to share
evaluation findings

Strategy Four: Develop a distribution plan to share
findings and implement it!

Strategy Five: Ask for feedback and actively
involve new and existing partners in the process

2006-07 System of Care Start-up Webinar Series
Monterey County’s System of Care:
Lessons Learned & Accomplishments
Scorecard is a visually appealing and useful tool to explain SOC values
and principles.

Highlighting of shared programs and accomplishments generated
goodwill – and momentum (many new partners this year!) Also- triggered
more collaboration in other areas!

Ongoing struggle – who is our target audience? Translate into Spanish?
Weighing cost against level of distribution.

Need to make the process youth- and family friendly – training. Goal in
Monterey County: Bring more youth to the table.

Key lesson learned to new communities (from a
social marketing perspective): Have the social
marketer involved early on in the development stages
of the evaluation framework and survey tool. The social
marketer can provide input into how questions and
outcomes can be translated into meaningful data to
varies target audiences.
2006-07 System of Care Start-up Webinar Series
Making It Last
 Have early conversations about
goals/objectives
 Inform each other of
progress/challenges
 Be a resource for trends, new
developments, community news
 Present information together for clarity
and accuracy
Resources




Caring for Every Child's Mental Health Campaign
www.mentalhealth.samhsa.gov/child/default.asp
(Features an order form for Children's Campaign products
such as Children’s Coloring Books, Annual Reports to
Congress, etc.)
Subscribe to the listserve 4-THE-CHILD
subscribe@yahoogroups.com
Family Leadership in Systems Evaluation Tip Sheets
 Engaging New Families in Evaluation & Language
Qualifying Family Preference in Hiring
Federation of Families for Children’s Mental Health
 The World of Evaluation: Three Short Courses
2006-07 System of Care Start-up Webinar Series
Questions?
Please press *7 on your phone
to unmute your line. When you
are finished, please press *6 to
remute.
2006-07 System of Care Start-up Webinar Series
Presenter email addresses:
Deborah Porter,
deborahaporter@wobblee.com
 Lisa Rubenstein,
lisa.rubenstein@samhsa.hhs.gov
 Jana Sczersputowski,
jana@yoursocialmarketer.com
 Tisha Tucker,
alyce.l.tucker@macrointernational.com

2006-07 System of Care Start-up Webinar Series