Concept Mapping
A Next Generation Strategy
For Planning and Evaluating
Systems Change
Presenters
Lenore B. Behar, Director
Child & Family Program Strategies
William M. Hydaker, Director
Hydaker Community Consulting
Andrea Aderton, Project Director
Circle of Hope Project
St. Joseph, MO
Mona Gauthier, Principal Investigator
Cathy Baxter, Evaluator
commUNITY cares
Hattisburg, MS
John Reiss, Consultant
Florida Institute for Family
Involvement
Associate Professor of Pediatrics
Institute for Child Health Policy
University of Florida
Gainesville, FL
Purpose: Enhance, facilitate, and support
family and consumer involvement in the
development of responsive, family-centered,
and community based systems of care for
children, youth and young adults and their
families.
Integrating
Families, Communities, and Providers
(IFCAP)
Funding:
Federal Maternal and Child Health Bureau’s Division of
Services for CSHCN
Funding Initiative:
Implementation of HP 2010 goal of statewide access to
family-centered care through medical homes
Eligible applicants:
State Title V CSHCN Programs or Program’s designee
IFCAP Partnerships
FIFI:
Invited by Children’s Medical Services (Florida Title V) to
apply, as the lead organization, for 3-year grant
Lead Partners:
Children’s Medical Services
Florida Pediatric Society (Florida AAP Chapter)
Eligible applicants:
State Title V CSHCN Programs or Program’s designee
IFCAP
Vision and Approach
Vision:
Build and sustain a family-centered, community-based,
culturally competent system of care for all children and youth
with special health care needs (CYSHCN) and their families in
Florida through community partnerships.
Approach:
Family – Enhance capacity to be full partners
Providers – Enhance capacity to implement medical home approach and
partner with families
System – Enhance capacity to facilitate, fund and sustain quality care
and partnerships for CYSHCN
Community – Support existing partnerships to build local systems
Supporting Communities
Build capacity of communities to respond to the needs of CYSHCN
Assist communities in understanding needs of CYSHCN and their
families
Prepare communities to serve as the primary site of service
delivery for their CYSHCN
Help communities build a network or services and supports for
CYSHCN and their families
Ensure communities have the necessary resources to maintain
medical home efforts
Use Existing Partnerships to
Build Local Systems
Request for proposals:
Identify and describe the community
Describe composition of the community’s health care professionals,
health and related programs, community agencies, schools and
other formal and informal organizations and the current capacity of
these entities to partner with families and work in a collaborative
fashion to provide all CSHCN with a medical home.
Describe barriers that will be addressed.
Use Existing Partnerships to
Build Local Systems
Request for proposals:
Applicants are expected to have already established a formal or
informal community-based coalition (i.e. working group, committee,
task force) that includes a range of CSHCN stakeholders.
Initial activity can involve expanding membership of core group.
Describe membership of applicant and community’s a)
demonstrated potential to develop partnership and b) current and
potential capacity to serve CSHCN and their families.
IFCAP Communities
Jacksonville
IF C A P
Sarasota
Broward Co.
Circle
of
Home, Opportunities, Parents and Professionals,
Empowerment
Policy Team Members
●
●
●
●
●
●
●
Family Guidance Center
Northwest Health Services
Heartland Health
St. Joseph School District
Preferred Family Healthcare
Albany Regional Center
Buchanan County Juvenile
Court
●
●
●
●
●
●
●
●
The Center
St. Joseph Youth Alliance
Children’s Division
Key Family Contact
Youth Coordinator
State to Local Liaison
Parents
Youth
Goal
• The goal of Circle of HOPE, a
collaborative community effort, is to design
and create a system of care that will
enhance service delivery system for
children at risk or with severe emotional
disturbance and their families through
integration of physical, behavioral health
and educational services.
Population of Focus
• Diagnosis of SED or
• Displays inability to function in family,
school or community that is expected to
last more than one year.
• Age 8-13
Operating Principles
• Services will be coordinated across agencies and
service agencies.
• Information will be shared between agencies to eliminate
the need for parents, family members or youth to provide
the same information to multiple agencies.
• Data will drive decision-making
• No Wrong Door
• An infrastructure will be created to develop and sustain
the system of care over time.
• Family involved in all phases of planning,
implementation, evaluation of services.
Circle of Hope Service Model
Referral Sources
» Parents
» Schools
» Primary Healthcare
Screening
» Family Support Assistant- system navigator
Enrollment
» Functioning Evaluation
» Care Coordinator- facilitator/case management
Service Planning
» Wraparound Process
Transition Planning
Administrative
Staff
Service
Delivery Staff
Project Director
SOC Service
Providers
Andrea Aderton
Service Contracts
Evaluation
MIMH/LAN Resources
Psychiatrists
Clinical Coordinator
Elizabeth Keane
TA/Cultural
Competency
Clinical Supervisor
(Interim)
Vicky Myers
Nicki Samson
Marketing &
Communications
Key Family Contact
Resource Planning &
Development
Youth Coordinator
Joseph Turner
Jayme Neal
Administrative
Assistant
Wendy Tyrrell
Care
Coordinator
Mike Bolten
Care
Coordinator
Vacant
Family Support
Assistant
Donna Barber
FSA Vacant
Physicians
Behavioral
Health
Providers
Non Mental
Health Services
Children’s
Division
Juvenile
Justice
Schools
Key Initiative Partners
– MS Department of Mental Health
– Pine Belt Mental Healthcare Resources
– Mississippi Families as Allies
Other Initiative Participants
• Forrest, Lamar & Marion County and City
School Districts
• Juvenile Justice/ Youth Court
• Youth Development Initiative
• Health Department
• Department of Human Services
• Boys & Girls’ Club
Initiative Partners, continued…
•
•
•
•
•
•
•
•
Aldersgate
Big Brothers/Big Sisters
Forrest County Youth Court
Hattiesburg Police Department
Mississippi Children’s Home Services/CARES
MCHS (Adolescent Shelter)
Oseola McCarty Youth Center
Catholic Social & Community Services
…..and still more….
• School Attendance Officers – Forrest &
Lamar
• Center for the Prevention of Child Abuse
• Lamar County – Parents As Teachers
• Lamar County Sheriff’s Department
• University of Southern Mississippi –
Resource Center
And growing….
•
•
•
•
•
Junior Auxiliary
Harden House
Memorial Behavioral Health
Parham Group
Salvation Army Boys & Girls Club
Who is the target population?
When do services start?
• The target population for this grant is youth ages 10-18
with a serious emotional disorder AND substance
misuse in Forrest, Lamar, and Marion Counties.
• Service Implementation Schedule:
– Forrest County: October 1, 2007
• Currently accepting referrals
– Lamar County: October 2008
– Marion County: October 2009
Intensive Workshop Overview
•
•
•
•
•
•
Introduction to Concept Mapping.
Process – how it works.
Products.
Break (10 minutes).
How to use Concept Mapping.
Comparison to other planning/ evaluation
methods.
Introduction to
Concept Mapping
• Group discussion of planning and evaluation
experiences.
• Concept Mapping defined.
• Concept Mapping as a planning and evaluation
tool.
• Examples.
Planning and Evaluation
Experiences
• Strengths
• Challenges
• Evaluation, quality improvement, continuous
quality improvement?
Definition
• A structured conceptualization process which can be
used to organize the thoughts and ideas of a group of
participants or stakeholders who wish to develop
conceptual framework for planning and evaluation.
• The process is used to develop a concept map which is
a pictorial representation of all of the ideas
generated by the entire group and the relationships
of each of the ideas to each other.
• The participants express both their individual ideas
and interact with the entire group and develop
additional ideas that are expressions of the group.
Advantages
•
•
•
•
•
Time efficient
Easy to understand process
Facilitates broader community involvement
Effective engagement strategy
Moves away from the “jargon” of professionals and
evaluators
• Gives participants an equal voice
• Supports ownership
• Multiple uses e.g. planning, logic model development,
and evaluation
In Order to Avoid This
And Get Organized Input
from
Community Stakeholders
Examples from the Field
• National Traumatic Stress Initiative
• Comprehensive Community Mental Health
Services Program for Children and Their
Families
• State of Mississippi
• Maternal Child Health Bureau
Concept Systems, Inc.
Example of Clients
Process – how does it work?
•
•
•
•
•
Convening a group.
Brainstorming.
Sorting the responses.
Rating the responses.
Feedback session & products.
Convening a Group
The organizers determine who the
participants will be. They may include
administrators, staff, board members, family
members, youth and/or community leaders.
Develop the Focus Prompt
The Focus Prompt is the trigger for
Brainstorming. It sets the framework for
how the group thinks about the task.
Examples of a Focus Prompt
• “To develop a system of care in our
community, we must have the following
characteristics and functions …..”
Or
• “To have a system of care in our community,
we must…..”
Brainstorming
•
Generate ideas in response
to a prompt.
Sorting
•
•
Sort each idea into
groups you believe are
related to each other.
Four-cluster solution
(effectiveness)
Outcomes
Team process
Evaluation/assessment
Planning
Label each group.
Rating
•
Rate each idea as to importance using a 1-5
point scale.
•
Rate each idea as to feasibility using a 1-5 point
scale.
Brainstorming
• Participants are encouraged to generate
statements representing the entire
conceptual domain of the topic of interest.
• Theoretically no limit on the number of
statements, though excessive numbers may
become cumbersome.
The Brainstorming Process
•
As someone makes a statement, it is typed into the
computer and projected onto the screen.
•
Each successive statement is put on the screen.
•
Brainstorming is complete when the group cannot
generate any more statements.
•
We will do up to 100 statements.
•
The prompt statement gives you your task.
Four-cluster solution
(effectiveness)
Outcomes
Team process
Evaluation/assessment
Planning
Brainstorming Guidelines
•
•
•
•
•
•
•
Keep the focus on the task.
Any input addressing the task is OK.
No criticism of others’ input.
No editing, except for clarity.
Listen to others.
Watch for duplication.
Participate.
Four-cluster solution
(effectiveness)
Outcomes
Team process
Evaluation/assessment
Planning
Roles of the Facilitators
•
•
•
Make sure topic is thoroughly considered.
Get everyone involved.
Keep everyone on task.
Four-cluster solution
(effectiveness)
Outcomes
Team process
Evaluation/assessment
Planning
Focus Prompt
“To have a system of care in our
community, we must…..”
Sorting
After appropriate editing, each statement is
printed on an index card and a complete set
of cards is provided to each participant.
Sorting Statements
Your task is to organize the statements into
groups that make sense to you.
•
Each of you has two things:
1. Cards to sort into piles.
2. A label to go with each pile.
•
•
•
Sort the cards into piles and place one
label on each pile.
Don’t create piles with miscellaneous items
•
•
You must have more than 1 pile!
You cannot have a pile that has only 1 card in it.
Rating
• Utilizes a Likert scale
• Each respondent is asked to rate each item on
two separate dimensions
• Each item is rated on a 1 to 5 response scale
with 1 equaling strongly disagree and 5
equaling strongly agree
Ratings
Use the rating sheets to determine the
importance and feasibility of each
statement.
Remember, Importance then Feasibility.
Four-cluster solution
(effectiveness)
Outcomes
Team process
Evaluation/assessment
Planning
Feedback and Products
• Concept Map derived from sorting and
labeling.
• Ladder graph derived from ratings of
importance and feasibility.
• Go-zone map derived from ratings of
importance and feasibility.
How the data are analyzed
Multidimensional Scaling
• Concept Systems software program calculates
spatial relationships of paired data from the
total square similarity matrix much as one
would position cities on a blank map if the
distance between the cities was known.
• Further mathematical scaling is done using a
multivariate analysis to represent the location
of all of the statements on a two dimensional
point map.
Multidimensional Scaling, cont’d
• Points on the map represent individual
statements and their positions, relative to
each other, indicate are important.
• Points close to each other are conceptually
similar, while points that are far apart are
conceptually different.
• The point map forms the basis for the
construction of the cluster map, the point
rating map, and the cluster rating map.
Hierarchical Cluster Analysis
• Individual statements on the map are grouped
into clusters of statements that theoretically
reflect similar concepts.
• The resultant “cluster map, depicts how the
multidimensional scaling points were grouped.
• There can be as many clusters as statements,
and the number decided upon is determined
by the discretion of the analyst.
Hierarchical Cluster Analysis,
cont’d
• In the instance of approximately 100
statements, 20 down to 3 clusters are
examined.
• As the analysis moves from one cluster level
to the next lowest,
(e.g., from 10 to 9 clusters for instance) the
grouping is examined to determine if it makes
sense terms of the overall concept.
The Products
Point Map
50
37 796872
54
20 30
9
49
33
63 21
57
65
67
34 38
64
26
23
42
66
69
10
15
4
47 77
83
78
41
39
4862
5
35
76
82
11
86
43
29
16
28
81
58
8 17
59
73
4051
24
7
12
70
61
60
32
75
80
36
31
74
46 22
25
19
712
56
55
45
3
27
14
6 18
1 84
8553
44
13
52
Five Cluster Solution
Community
Development
Youth & Family
Focus
Quality
Services
Training/Evaluation
Communication/Collaboration
Six Cluster Solution
Community
Development
Youth &
Family Focus
Cultural
Issues
Quality Services
Training/Evaluation
Communication/Collaboration
Seven Cluster Solution
Community
Development
Youth Focus
Family
Focus
Cultural
Issues
Quality Services
Training/Evaluation
Communication/Collaboration
The Ladder Graph
• Shows the results of comparisons between
participant responses on importance and
feasibility, by cluster
• Depicts the correlation between these two
dimensions
• A correlation ranges from -1.00 to +1.00
• The correlation coefficient (r value) for the
level of agreement between the two
dimensions is significant at 0.93
Cluster Ratings
on Importance and Feasibility
Importance
Feasibility
4.1
4.1
Youth Focused Initiatives
Family Issues
Community Education
Family Issues
Evaluation
Advocacy
Service Delivery
Communication/Collaboration
Youth Focused Initiatives
Community Education
Funding
Communication/Collaboration
Evaluation
Advocacy
Service Delivery
Funding
2.9
r = 0.93
2.9
Go-Zone Plot
• Facilitates strategic planning. By focusing on
those statements that are perceived to be
the most important and most feasible (upper
right quadrant) action plans can be created.
• Facilitates development of a logic model for
systems change, by defining areas of
importance.
Go-Zones
4.71
30
20
53
22
Feasibility
6
25
45 7117
642711
70
6176
84
28
3.91
40
42
43
67
58
15
55 49
46 29 68
52
81
7882
35
3
3.38
10
26
36
348
14
72
38 51 7
44
24
798554
12
1356 5023573321
8386
16
63 6275
3
19 77
69 47
8037
74 66 59
32
31
5
1
2
4
18
6041
65
73
9
48
39
4.22
Importance
4.96
Twelve Highest Statements
•
•
•
•
•
Listen to youth.
Ask youth what they want
Deal with confidentiality.
Develop a plan for communications.
Train direct service staff to serve children
and families.
• Focus on strengths.
Twelve Highest Statements
cont’d
• Make sure that services are research/
evidenced based and staff are appropriately
trained.
• Get the right people hired to do the job.
• Get the right people to the table.
• Be family-centered.
• Define the goals of the system of care to the
agencies and to families being served.
• Conduct ongoing evaluation.
What has this exercise
accomplished?
• The concept mapping framework provides an
organized means of keeping participants on
task for a planning or an evaluation study.
• The methodology allows the participants to
describe their ideas and thoughts in a
language familiar to them rather than the
language of the evaluator or planner, and free
of the jargon often used in social science
literature.
• The results are presented in a graphic format
illustrating all of the ideas and their interrelationships at a glance to both the
participant group as well as to other
interested groups.
• Finally, and possibly most importantly, the
process increases group cohesiveness and
morale as it imparts a sense of ownership in
the decisions emanating from the results.
How Others Have Used
Concept Mapping
Purpose: Enhance, facilitate, and support
family and consumer involvement in the
development of responsive, family-centered,
and community based systems of care for
children, youth and young adults and their
families.
Why Concept Mapping ?
Circle
of
Home, Opportunities, Parents and Professionals,
Empowerment
1st Year Planning Process
• Policy Team Development
• Strategic Planning
» Vision, Mission, Goal Development
» Population of Focus
• Concept Mapping
» Broader Based Stakeholder Input
» Incorporation into the Logic Model
Clusters and Statements Sorted
• Principles
• Objectives
• Strategies
• Next Steps
Eight Cluster Solution
Community Education
Communication/
Collaboration
Youth Focused
Initiatives
Family Issues
Evaluation
Service Delivery
Advocacy
Funding for Services
From Concept Mapping to Logic
Model: Identifying Issues
• Project Staff and Executive Committee
began by examining Concept Mapping
(CM) Statements.
• Participants were asked to classify
statements into three issue categories:
Community, System or Practice Level
• Involved a bit of “backwards engineering.”
We had to take the “action” back to the
“issue for needing the action.”
From Concept Mapping to Logic
Model: Identifying Issues
Concept Mapping
Statements:
Framed as Actions
Issues:
Community,
System and
Practice Level
From Concept Mapping to Logic
Model: Identifying Values
• Some “action statements” actually
reflected a core value.
• We decided to extract these from the
“Issues” to make form the overall “values”
important to our system transformation
efforts.
• Added these to existing System of Care
values.
From Concept Mapping to Logic
Model: Identifying Values
Values
Concept Mapping
Statements:
Framed as Actions
Issues:
Community,
System and
Practice Level
From Concept Mapping to Logic
Model: Identifying Strategies
• Concept Mapping allowed the generation
of a list of “actions” needed to implement a
successful System of Care
• These “actions” were classified by
category; associated strategies were
easily obtained by using a worksheet
format and soliciting group participation
• We referred to these as “homework
assignments.”
From Concept Mapping to Logic
Model: Identifying Strategies
Values
Concept Mapping
Statements:
•Framed as Actions
•Categorized into
Groups
•Reframed into
Strategies
Issues:
Community,
System and
Practice Level
From Concept Mapping to Logic
Model: Outcomes
• Outcomes based on the “why” we wanted
to accomplish our tasks.
• Were extrapolated by examining our
issues, strategies, and associated action
steps.
• Brought us full-circle back to our “action
statements” from Concept Mapping.
From Concept Mapping to Logic
Model: Outcomes
Values
Concept Mapping
Statements:
•Framed as Actions
•Categorized into Groups
•Reframed into Strategies
Issues: Community,
System and Practice
Level
Outcomes:
Child and Family,
Community, System and
Practice Level
Example: Identifying Issues
• One of our Concept Mapping Statements
read:
– Develop a Mississippi Families as Allies
chapter in the Pine Belt area.
• Issue can be “reverse engineered” to read:
– There is a need for a Mississippi Families as
Allies chapter in the Pine Belt area.
• Statement and Issue were classified on
the Community Level.
Example: Making It a Value
• Our previous statement was:
– Develop a Mississippi Families as Allies
chapter in the Pine Belt area.
• Not only does it embody a needed action,
it also contributes to a System of Care
principle, which is:
– Family Driven
Example: Identifying Strategies
• The action statement indicates a need to do
something.
• That “need” to do something was embodied as a
Community Level Strategy entitled:
– Family Advocacy and Support
• That means to say, we want to improve the issue
by using the above mentioned strategy, which
takes us back to the “action” itself, which was:
– Develop a Mississippi Families as Allies chapter in the
Pine Belt area
Example: Outcomes
• The last step, the Outcome, was the easiest.
• We formulated a “why” for each issue and
strategy, then transformed it to an Outcome.
Why do we want a MS Families
as Allies chapter in the Pine
Belt Area?
So that families and youth have
a channel for advocacy and
support!