Title I School Improvement
1003(a) Grant Writing Workshop
Building a Logic Model With
SMART Objectives
Anduamlak Meharie
Office of Research
Logic Model
“A logic model is a systematic and visual way to present and
share your understanding of the relationships among the
resources you have to operate your program, the activities you
plan, and the changes or results you hope to achieve”
W.K. Kellogg Foundation Logic Model Development Guide.
“A logic model’s purpose is to communicate the underlying
‘theory’ or set of assumptions or hypotheses that program
proponents have about why the program will work, or about
why it is a good solution to an identified problem”
http://www.insites.org/documents/logmod.htm
Logic Model: Purpose
• Design and planning
– To clearly communicate the central goal of an initiative, why it
is important, and what will result from it.
– To select specific and measurable objectives expected to lead
to the desired result.
– To identify activities and resources needed to meet selected
objectives.
• Implementation
– To identify and collect data need to effectively monitor
initiative activities and improve programming.
– To create a common reference point for everyone involved in
the initiative.
• Evaluation
– To help determine if activities are being implemented as
planned.
– To help determine if the initiative is making progress.
– And, ultimately, to determine if the initiative has led to the
desired result.
Logic Models: Components
•
•
•
•
•
•
•
Goal(s)
Objective(s)
Activities
Input/Resources
Outputs
Outcome(s)/Impact(s)
Indicator(s)
Logic Model: Goals vs. Objectives
• Goal: A broad statement of intended outcomes for
a program, line of business or an organization.
– Goals are typically intended to guide the formation of (more
specific) objectives that can be linked to goals.
(McDavid and Howthrorn, 2006)
• Example
– To decrease the dropout rate of at-risk students in
grades 9-12 at Washington High School.
Logic Model: Goals vs. Objectives
• Objective: A statement of intended outcomes that is
focused and time-specific (that is, achievable in a
specified time frame). (McDavid and Howthrorn, 2006)
• Example
– By September 2013, at least 80% of all at-risk students in
grades 9-12 will be assigned an adult advocate who will be
responsible for meeting the student at least twice a week
to address academic and social needs, and communicate
with families.
– By May 2014, all core curriculum teachers at Washington
High School will receive evidence- based professional
development on (1) how to help struggling students, (2)
differentiated instructional methods, and (3) managing
student behavior.
S.M.A.R.T. Objective
• Specific
• Measurable
• Attainable
• Result-oriented and relevant
• Time-bound
Specific Objective
• Is your objective Specific?
• If you have a specific objective, you should be able
to answer the following six questions:
– What: What do you want to accomplish?
– Why: What are the reasons, purpose or benefits of
accomplishing the objective?
– Who: Who is involved? Who are the stakeholders?
– Where: Where is it going to happen?
– Which: Which attributes are important? (Requirements
and constraints; risk and protective factors).
Measurable Objective
• Is your objective measurable?
• If your objectives are specific and measurable, you
should be able to establish concrete criteria for
measuring progress toward the attainment of
each.
– How will you know when it is accomplished?
– What indicators will you look for to measure progress
and success?
– Are data to measure your progress and success readily
available? Or do you need to develop new measures,
e.g., surveys, focus groups, etc…?
Attainable Objective
• Is your objective attainable?
• Given resources available to you, the objective
must be realistic and attainable.
– Do you have sufficient financial capacity to accomplish
your objective?
– Do you have personnel with sufficient abilities and skills
to accomplish your objective (including evaluation)?
– Do you have sufficient time to accomplish your
objective?
– What other types of resources do you need to attain
your objective (i.e., technology, space, equipment,
etc…)?
Result-Oriented and Relevant
Objective
• Is your objective result-oriented and relevant?
• Ultimately, your objectives should help you
achieve your major project goal(s).
– Is it worthwhile?
– Is this the right time?
– Does this align with other efforts/needs?
Time-Bound Objective
• Is your objective time-bound?
• Objectives should have starting points, ending
points, and fixed durations.
– When will you achieve this objective?
– When will you undertake activities to achieve our
objective?
– When can you expect to see some short-term
outcomes?
S.M.A.R.T. Objective: Example
•
Goal 1: Within two years after the coaching program for at-risk youth at
Washington High School is established, the dropout rate of high school
students will fall below 1% at grades 10, 11, & 12.
•
Objective 1.1: As a result of the program attendance rates at Washington
High School will be at 100%.
Is the objective…
Objective
As a result of the
program attendance
rates at Benedictine
High School will be
at 100%.
Attendance rates at
Benedictine High
School will exceed
97% one year after
the apprenticeship/
coaching program is
established.
Attendance rates for
at-risk youth at
Benedictine High
School at grades 10,
11, & 12 will exceed
97% one year after
the apprenticeship/
coaching program is
established.
Specific?
Measurable?
Attainable?
Result-oriented/
Relevant?
Time-Bound?
Yes
Yes (X)
Yes
Yes (X)
Yes
No, revise (X)
No, revise
No, revise (X)
No, revise
No, revise (X)
Yes
Yes (X)
Yes (X)
Yes (X)
Yes (X)
No, revise (X)
No, revise
No, revise
No, revise
No, revise
Yes (X)
Yes (X)
Yes (X)
Yes (X)
Yes (X)
No, revise
No, revise
No, revise
No, revise
No, revise
S.M.A.R.T. Objective: Example
• Objective 1.1: As a result of the program
attendance rates at Benedictine High School will
be at 100%.
• Objective 1.1: Attendance rates for at-risk youth
at Benedictine High School at grades 10, 11, & 12
will exceed 97% one year after the
apprenticeship/ coaching program is established.
S.M.A.R.T. Objective: Exercise
1. Choose one of the objectives listed on your
grant application.
2. Using the worksheet provided, examine your
objective to ensure that it is S.M.A.R.T. with
respect to the major goal(s) of your project.
3. Revise as needed until the objective meets
all five (S.M.A.R.T.) criteria.
S.M.A.R.T. Objective
Worksheet
Goal:
Objective:
Is the objective…
Objective
Specific?
Measurable?
Attainable?
Yes
Yes
Yes
Result-oriented/
Relevant?
Yes
Time-Bound?
No, revise
No, revise
No, revise
No, revise
No, revise
Yes
Yes
Yes
Yes
Yes
No, revise
No, revise
No, revise
No, revise
No, revise
Yes
Yes
Yes
Yes
Yes
No, revise
No, revise
No, revise
No, revise
No, revise
Yes
Yes
Yes
Yes
Yes
No, revise
No, revise
No, revise
No, revise
No, revise
Yes
Logic Models: Components
•
•
•
•
•
•
•
Goal(s)
Objective(s)
Activities
Input/Resources
Outputs
Outcome(s)/Impact(s)
Indicator(s)
Logic Models: Components
• Activities: the actions that are needed to meet
your objectives.
• Resources: inputs needed design and implement
activities.
• Outputs: the tangible and direct results of
activities.
• Outcome(s): specific, attainable and measurable
changes that are likely to occur as a result of
activities.
• Indicator(s): data to monitor implementation, and
measure progress as well as success of initiative.
Logic Model
Goal 1:
Objectives
Activities
Inputs/Resources
Outputs
Outcomes
Indicator/Data Source
(Specific, Measurable,
Achievable, Relevant,
and Time-bound)
(what we do to achieve
objectives/goals)
(human, financial,
technology, etc…
resources needed for
activities)
(tangible and direct
products of activities that
lead to desired outcomes)
(short-, medium-, and
long- term desired results
of activities)
(indicators we look for to
measure implementation,
progress and success)
1.1
1.2
1.3
1.1.1
1.1.1
1.1.1
1.1.2
1.1.2
1.1.2
1.1.3
1.1.3
1.1.3
1.2.1
1.2.1
1.2.1
1.2.2
1.2.2
1.2.2
1.2.3
1.2.3
1.2.3
1.3.1
1.3.1
1.3.1
1.3.2
1.3.2
1.3.2
1.3.3
1.3.3
1.3.3
Logic Model: Example
Goal 1: To decrease the dropout rate of at-risk students in grades 9-12 at Washington High School.
Objectives
1.1 By September 2013,
at least 80% of all at-risk
students in grades 9-12
will be assigned an adult
advocate who will be
responsible for meeting
the student at least twice
a week to address
academic and social
needs, and communicate
with families.
Activities
Inputs/Resources
1.1.1 Identify at-risk
students
-School staff, community
members, and/or social
workers to work as
advocates
Outputs
Outcomes
1.1.1 Number of at-risk 1.1.1 By June 2013, at-risk
students identified by
students in grades 9-12 will
grade level
indicate a greater sense of
1.1.2 Identify, recruit,
school membership and
and train adult advocates
1.1.2 Number of
involvement.
-Monetary compensation advocates; list of
1.1.3 Clearly define the
for trainers and advocates. training dates, locations, 1.1.2 By June 2013, at-risk
roles and responsibilities
and participants;
students in grades 9-12 will have
of all stakeholders
-Cost of training facility,
reduced risky behaviors and
(guiding document)
printed materials, meals
1.1.3 A document that absentee rate.
for participants.
elucidates the roles and
1.1.4 Assign 10 or less atresponsibilities adult
1.1.3 By June 2014, at-risk
risk students per advocate -Staff to document,
advocates, students,
students in grades 9-12 will have
monitor and evaluate
school staff,
improved grades.
implementation and
parents/guardians, and
progress.
Ultimate outcome (Impact): The
other stakeholders
sum total of all objectives- will
1.1.4 List of advocates result in a decrease in dropout
and their assigned atrate of at-risk students in grades
risk students; number
9-12.
and average duration of
meetings with each
student; number and
average duration of
meetings with parents
of at-risk students
Indicator/Data Source
-Student survey to
measure effect on sense
of school membership and
involvement (maybe
pre/post).
-Student disciplinary and
attendance record
-Student academic record.
-List of all at-risk students
and those who
participated in the adult
advocate program to
ascertain the % of at-risk
students reached.
-Participant questionnaire
about the quality and
usefulness of training and
guiding document.
-Student and parent
survey to gather feedback
about the quality and
usefulness of the adult
advocate program.
Logic Model: Exercise
• Insert the objective you developed earlier in
the logic model template.
• Identify activities and inputs/resources
needed to achieve the objective.
• Explicitly state expected results of activities
in terms of outputs and outcomes.
• Identify indicators/data sources that will
allow you to document and monitor
implementation as well as measure progress
and success.
Logic Model
Goal 1:
Objectives
Activities
Inputs/Resources
Outputs
Outcomes
Indicator/Data Source
(Specific, Measurable,
Achievable, Relevant,
and Time-bound)
(what we do to achieve
objectives/goals)
(human, financial,
technology, etc…
resources needed for
activities)
(tangible and direct
products of activities that
lead to desired outcomes)
(short-, medium-, and
long- term desired results
of activities)
(indicators we look for to
measure implementation,
progress and success)
1.1
1.2
1.3
1.1.1
1.1.1
1.1.1
1.1.2
1.1.2
1.1.2
1.1.3
1.1.3
1.1.3
1.2.1
1.2.1
1.2.1
1.2.2
1.2.2
1.2.2
1.2.3
1.2.3
1.2.3
1.3.1
1.3.1
1.3.1
1.3.2
1.3.2
1.3.2
1.3.3
1.3.3
1.3.3
Evaluation
• It involves looking at your program and asking
–
–
–
–
Are we doing what we said we were going to do?
Is what we’re doing working?
How do we know it’s working?
Did we accomplish what we set out to accomplish?
• It is part of what you put in place beforehand to
help you run your programs.
– It should be about looking ahead not backwards.
• It is part of the continuum of planning (rolling
evaluation).
Evaluation
It is a systematic process of asking questions,
collecting information, and using the answers to
•
•
•
•
•
Set realistic goals and fine tune strategic decisions.
Measure progress and identify areas of improvement.
Identify training and technical assistance needs.
Be accountable and be credible to your constituents.
Motivate by providing documentation of your
achievements.
• Guide budget and resource allocation.
• Generate support for programs.
• Be an efficient organization that works toward its
mission.
Formative vs. Summative Evaluation
Formative evaluation (implementation and
progress)
• How well were activities implemented (fidelity)?
Was the target audience reached? What did they
think about the activity?
• How are resources allocated and used for
activities?
• What were other products of those activities.
• How are preliminary evaluation findings used to
improve implementation of activities?
• What obstacles were encountered and how were
they overcome?
Formative vs. Summative Evaluation
Summative evaluation (Outcome)
• Are the initiative goal(s) and objectives being
achieved? How fully are they being achieved?
• Do activities have beneficial effects on at-risk
students? If so, what are they?
• Do activities have any adverse side effects on the atrisk students? If so, what are they?
• Are some at-risk students affected more by the
services than others?
• Can the results be traced to the initiative itself? (Is a
control group possible?)What other factors may
have caused the results?
Evaluation: Planning
• What do you really want to know and for what
purpose?
• What evidence or indicators do you need to
answer your questions?
• Who’s in charge of evaluation? (roles and
responsibilities)
• What time frame are you evaluating? How much
time is allocated to conducting evaluation and
when?
• What is the budget for evaluation?
• Commit the plan to paper!!!!
Evaluation Plan Matrix
Evaluation Timeline
Objective
Source of Data
Responsible
Person (s)
Instrument
development
Data
collection
Data
analysis
Reporting
Next Steps
• Examine your remaining objectives and ensure
that they are all S.M.A.R.T.!
• Insert each objective in the logic model template
• Identify activities and inputs/resources needed to
meet objectives.
• Explicitly state expected results of activities in
terms of outputs and outcomes.
• Identify indicators/data sources that will allow you
to document and monitor implementation as well
as measure progress and success.
Next Steps
• Look at each of your completed SMART objectives and
identify data sources, time lines, and assign responsible
parties for collecting, analyzing, and reporting data about
your project.
– If necessary, assign responsibility for the design of new
measures (for example, surveys). Define a time line for
completion, administration, and analysis.
• As you assign responsibility, remember, evaluation is a team
effort, not just one person’s job.
• Put evaluation on your upcoming meeting agendas. Always
take some time to talk about where you are in your project,
what data you have collected, what it is telling you and what
you still need to know.
Thank You
Anduamlak Meharie
Coordinator – Research and Evaluation
WVDE Office of Research
http://wvde.state.wv.us/research
ph: 304.558.2546
fax: 304.558.1613
ameharie@access.k12.wv.us