How Logical Is Your
Logic Model?
Developing Useful and Robust
Project Logic Models
2/27/2014
1
The work of TEAMS is supported with funding provided by the National Science
Foundation, Award Number DRL 1238120. Any opinions, suggestions, and
conclusions or recommendations expressed in this presentation are those of the
presenter and do not necessarily reflect the views of the National Science
Foundation; NSF has not approved or endorsed its content.
2/27/2014
2
Strengthen the quality of the MSP
project evaluation and build the capacity
of the evaluators by strengthening their
skills related to evaluation design,
methodology, analysis, and reporting
2/27/2014
3
• Online Help-Desk for submitting requests
• Website at: teams.mspnet.org
• Webinar series targeted to specific
evaluation topics
• Tiered technical assistance for differentiated
services
• Instrument review and sharing
2/27/2014
4
How Logical Is Your Logic Model?
Developing Useful and Robust
Project Logic Models
Dave Weaver, Director
RMC Research Corporation
Portland, Oregon
February 27, 2014
2/27/2014
5
Webinar Objectives
• Provide a rationale for the need for a
well-defined logic model
• Summarize characteristics of a well-defined
theory of action and logic model
• Provide an opportunity to examine your
current theory of action and logic model
• Discuss strategies for developing useful and
robust logic models
2/27/2014
6
Why Are Logic Models Important?
• Common Guidelines for Education Research and
Development
– U.S. Department of Education, National Science
Foundation. (August, 2013).
– http://ies.ed.gov/pdf/CommonGuidelines.pdf
“The proposal should include a description of the initial
concept for the planned investigation, including a wellexplicated theory of action or logic model. The concept
and logic model should identify key components of the
intervention and should describe their relationships,
theoretically, and operationally.”
2/27/2014
7
What does it mean?
“well-explicated theory
of action or logic model”
2/27/2014
8
It starts with the
Theory of Action
A logic model is only as good as
the theory of action that it is
based on
2/27/2014
9
What Is a Theory of Action?
• Collective belief about causal relationships
between action and desired impacts
– Simple:
If . . .
Then . . .
– Complex:
If . . . and . . . and . . . and . . . Then . . .
• Collaborative interpretation of the literature
2/27/2014
10
Characteristics of a Robust
Theory of Action
• Describes project impact as close to the primary
target audience as possible
– Example: A description of what students do to learn
• Recognizable when it is going on
– Observable
• Defines fidelity of project implementation
• Can be a testable hypothesis
• Believable
– Interpretation of current literature
2/27/2014
11
Math Example (Common Core State Standards)
If teachers use developmentally appropriate yet challenging
tasks and activities that engage students in:
• Justifying—Explaining and justifying their reasoning
mathematically
• Generalizing—Identifying and verifying conjectures or
predictions about the general case
• Representing—Using representations (symbolic, notation,
graphs, charts, tables, and diagrams) to communicate and
explore mathematical ideas
• Applying—Applying mathematical skills and concepts to
real-world applications
Then student achievement and interest in mathematics will
increase.
2/27/2014
12
Does it describe project impact as close to
the primary target audience as possible?
If teachers use developmentally appropriate yet challenging
tasks and activities that engage students in:
• Justifying—Explaining and justifying their reasoning
mathematically
• Generalizing—Identifying and verifying conjectures or
predictions about the general case
• Representing—Using representations (symbolic, notation,
graphs, charts, tables, and diagrams) to communicate and
explore mathematical ideas
• Applying—Applying mathematical skills and concepts to
real-world applications
Then student achievement and interest in mathematics will
increase.
2/27/2014
13
Is it recognizable when it is going on?
If teachers use developmentally appropriate yet challenging
tasks and activities that engage students in:
• Justifying—Explaining and justifying their reasoning
mathematically
• Generalizing—Identifying and verifying conjectures or
predictions about the general case
• Representing—Using representations (symbolic, notation,
graphs, charts, tables, and diagrams) to communicate and
explore mathematical ideas
• Applying—Applying mathematical skills and concepts to
real-world applications
Then student achievement and interest in mathematics will
increase.
2/27/2014
14
Does it define fidelity of project
implementation?
If teachers use developmentally appropriate yet challenging
tasks and activities that engage students in:
• Justifying—Explaining and justifying their reasoning
mathematically
• Generalizing—Identifying and verifying conjectures or
predictions about the general case
• Representing—Using representations (symbolic, notation,
graphs, charts, tables, and diagrams) to communicate and
explore mathematical ideas
• Applying—Applying mathematical skills and concepts to
real-world applications
Then student achievement and interest in mathematics will
increase.
2/27/2014
15
Can it be a testable hypothesis?
If teachers use developmentally appropriate yet challenging
tasks and activities that engage students in:
• Justifying—Explaining and justifying their reasoning
mathematically
• Generalizing—Identifying and verifying conjectures or
predictions about the general case
• Representing—Using representations (symbolic, notation,
graphs, charts, tables, and diagrams) to communicate and
explore mathematical ideas
• Applying—Applying mathematical skills and concepts to
real-world applications
Then student achievement and interest in mathematics will
increase.
2/27/2014
16
Is it believable?
If teachers use developmentally appropriate yet challenging
tasks and activities that engage students in:
• Justifying—Explaining and justifying their reasoning
mathematically
• Generalizing—Identifying and verifying conjectures or
predictions about the general case
• Representing—Using representations (symbolic, notation,
graphs, charts, tables, and diagrams) to communicate and
explore mathematical ideas
• Applying—Applying mathematical skills and concepts to
real-world applications
Then student achievement and interest in mathematics will
increase.
2/27/2014
17
Science Example
• Students learn science when they:
– Articulate their initial ideas,
– Are intellectually engaged with important
science content,
– Confront their ideas with evidence,
– Formulate new ideas based on that
evidence, and
– Reflect upon how their ideas have evolved
2/27/2014
18
Online Survey 1
• Does your project have a theory of action
statement?
• If yes, to what extent does it meet the
criteria?
– Criteria for a robust theory of action:
•
•
•
•
•
Describes impact close to the primary audience
Is recognizable in practice
Defines fidelity of implementation
Can be a testable hypothesis
Is believable—Shared interpretation of literature
2/27/2014
19
What Is a Logic Model?
• A diagram that shows the logical connection
between project resources, activities,
outcomes, and expected impacts
• Incorporates a primary theory of action
• Can be viewed as a collection of theories of
actions
• Answers the question:
Why would the planned activities be
expected to have the desired impacts?
2/27/2014
20
Basic Logic Model
Black Box Issues
Inputs
Activities
Outputs
Outcomes
Impacts
Resources
Distal Impacts
Activities
2/27/2014
Proximal Impacts
21
Logic Model Columns
• Resources—What resources are or could reasonably be
available?
• Strategies and Activities—What will the activities,
events, etc. be?
• Outputs—What are the initial products of these
activities?
• Short-Term Outcomes—What changes are expected in
the short-term?
• Long-Term Outcomes—What changes are wanted after
initial outcomes?
• Impacts—What are hoped for changes over long haul?
2/27/2014
22
Characteristics of a Robust Logic Model
• Incorporates a primary theory of action
• Shows the logical connection between project
resources, activities, outcomes, and expected
impacts
– Is a collection of If … Then… statements
• Answers the question: Why would the planned
activities be expected to have the desired
impacts?
• Serves as a framework for project evaluation
• Guides fidelity of implementation
2/27/2014
23
Making Mathematical Reasoning Explicit (MMRE) Logic Model
Does it incorporate the primary theory of action for the project?
If teachers use rich mathematical tasks
combined with purposeful and probing
questions to engage students in discourse and
the use of the tools (notation, symbolization,
graphs, charts, etc.) of mathematics to:
• Explain and justify their mathematical
reasoning (justification)
• Develop and verify mathematical
generalizations
Then students’ reasoning skills and student
achievement in mathematics will increase.
Does it show the logical connections?
Does it show why the planned activities would be expected
to have the desired impacts?
Does it serve as a framework for project evaluation?
Does it guide fidelity of implementation?
Online Survey 2
• Does your project have a logic model?
• If yes, to what extent does it meet the
criteria?
– Criteria for a robust logic model:
• Incorporates the project theory of action
• Shows the logical connections between project
resources, activities, outcomes, and expected impacts
• Explains why the planned activities would be expected
to have the desired impacts
• Serve as a framework for the project evaluation
• Guides fidelity of project implementation
2/27/2014
30
Strategies for Logic Model
Development
• Engage project stakeholders
• Begin by drafting a primary theory of action
statement
– Go back to the characteristics of a robust theory
of action
• Ask probing questions to help stakeholders
think about why activities should work
• Take notes and work toward consensus
2/27/2014
31
Suggested Process
• First half-day session with stakeholders
– Learn about project activities, objectives, and goals
– Introduce logic model development process
– Gather input for project theory of action
• Between Sessions
– First draft of Theory of Action and Logic Model
• Second half-day session
– Gather input on draft
– Establish commitment to theory of action statement
• “Do you believe that this statement accurately states your
collective belief about why your project will achieve its
desired impacts?”
2/27/2014
32
Go Back to the Characteristics of a
Robust Theory of Action
• Describes project impact as close to the primary
target audience as possible
– Example: A description of what students do to learn
• Recognizable when it is going on
– Observable
• Defines fidelity of project implementation
• Can be a testable hypothesis
• Believable
– Interpretation of current literature
2/27/2014
33
Some of My Favorite Questions!
• What would it look like in the classroom if your
professional development was effective?
• What kind of cognitive engagement would
students be involved in?
• How would you know your project has been
successful?
• When this project is over what will you be able
to point to as a legacy of the project?
• What is it about your professional development
that you think is effective?
2/27/2014
34
Using the Technology
• Taking Notes
– Capture the language of the stakeholders
– Don’t be afraid to paraphrase
– Use outliner view
• Drafting the Logic Model
– Don’t use a table format—too linear and
constraining
– Use graphic software
• Visio
• MS Word
2/27/2014
35
Questions!
Please use your chat box to
submit questions for Dave
2/27/2014
36
John T. Sutton, PI
Dave Weaver, Co-PI
RMC Research Corporation
633 17th Street
Suite 2100
Denver, CO 80202-1620
RMC Research Corporation
111 SW Columbia Street
Suite 1030
Portland, OR 97201-5883
Phone: 303-825-3636
Toll Free: 800-922-3636
Fax: 303-825-1626
Email: sutton@rmcdenver.com
Phone: 503-223-8248
Toll Free: 800-788-1887
Fax: 503-223-8399
Email: dweaver@rmccorp.com
2/27/2014
37