In the Beginning There was Evaluation

advertisement

Creating and Using Logic

Models for Project Compass

Ginger Fitzhugh and Chris Kingsley

February 15 & 16, 2008

The Center for Youth and Communities

The Heller School for Social Policy and Management,

Brandeis University

“ Wouldn’t it be nice if just once they’d give us money, skip all the falderal, and say ‘go crazy.’”

Center for Youth and Communities, Brandeis University 2

What we’ll cover

• What is a logic model?

• Why construct one?

• How do you create one?

• A few hints

Center for Youth and Communities, Brandeis University 3

What is a logic model?

• A picture of how your initiative works – what you will do, with whom, why, and what will result (your theory of change )

• If we do X, then Y should happen, if the assumptions on which the initiative is based are true

 If we work together and change our policies and practices, our underserved students will remain enrolled and graduate.

Center for Youth and Communities, Brandeis University 4

The Project Compass logic model asks:

• What is the problem to be addressed?

• Who will be served through your initiative?

• What are your assumptions about how and why your initiative will work?

• What strategies/activities will you use?

• What outcomes do you expect?

• How will you measure success ?

• What are the long-term impacts you hope to achieve?

Center for Youth and Communities, Brandeis University 5

Parts of the Project Compass logic model

Name of Initiative

Problem Statement:

What is the problem to be addressed?

For

Whom

Target group(s)

Assumptions

What you know, think and believe about what’s needed and will work

Strategies/

Activities

Activities needed to achieve desired outcomes

Outcomes

Reasonably measurable year-by-year changes in policies, practices or target group(s)

Measures of

Success

Information needed to show whether initiative is successful

Long-Term

Impacts

Ultimate outcomes for initiative

Center for Youth and Communities, Brandeis University 6

Examples

“ Clamshell University” Project Compass Initiative

Problem Statement: Students of color at “Clamshell University” currently experience disproportionally lower rates of academic success, are more likely to drop out, and are less likely to graduate.

For

Whom

Assumptions

Students of color

Stakeholders from across campus

(administrators, faculty, students and community members) must collaborate for institutional change to be possible and sustained.

Faculty and staff in academic departments

Faculty lack information about how to advise underserved students who arrive academically underprepared.

Strategies/ Activities

Conduct cross-campus committee meetings to identify ways to streamline delivery of academic student support services

(President appoints, year 1 meet 3x each semester).

Develop and deliver professional development workshops to help faculty better advise underprepared students re: what courses to take and what support services are available on campus (2 opportunities each semester, starting with faculty who have academic advising responsibilities).

Outcomes

(A) Academic student support services are streamlined and, (B) underserved students are able to more easily access the services they need.

Measures of

Success

Long-

Term

Impacts

(A) Coordinators of services make accurate referrals. (B)

The number of underserved students accessing students support services increases.

Underserved students experience higher levels of persistence and success

Faculty advisors provide more accurate and complete information to underserved students re: coursework and support services.

The number of underserved students reporting that their faculty advisors informed them of appropriate coursework to take and support services increases. (Add questions to our endof-year survey.)

Center for Youth and Communities, Brandeis University 7

We hate logic models that…

• Are useless academic exercises

• Are created and never looked at again

• Are too generic

• Aren’t shared with stakeholders

• Are never used

Center for Youth and Communities, Brandeis University 8

Benefits of the process of creating a logic model

• Clarifies thinking

• Uncovers different understandings

• Makes it easier to identify gaps

• May lead to consideration of new ideas

• Helps focus complex work

• Helps decide what outcomes are important to track

• Helps create shared ownership

Center for Youth and Communities, Brandeis University 9

Benefits of having a logic model product

• Aids in planning

• Shows cause-and-effect relationships between strategies and outcomes

• Makes it easy to share a project description with others

• Supports accountability and evaluation

• Assists in replication

• Often is required!

Center for Youth and Communities, Brandeis University 10

How do you create a logic model?

Bring key stakeholders together. Then:

1.

Define the problem you will address

2.

Determine individuals or group(s) you will target

3.

State long-term impacts and work backwards

4.

Decide on outcomes

5.

Ask, “If we want to achieve these outcomes, what strategies should we use?”

6.

Ask, “If we’re successful, how will we know it?”

7.

Consider, “What assumptions underlay our approach?”

8.

Work backwards and forwards across categories

Center for Youth and Communities, Brandeis University 11

Steps for Creating a Project Compass logic model

Name of Initiative

Problem Statement:

What is the problem to be addressed?

For

Whom

Target group(s)

Assumptions

What you know, think and believe about what’s needed and will work

Strategies/

Activities

Activities needed to achieve desired outcomes

Outcomes

Reasonably measurable year-by-year changes in policies, practices or target group(s)

Measures of

Success

Information needed to show whether initiative is successful

Long-Term

Impacts

Ultimate outcomes for initiative

Center for Youth and Communities, Brandeis University 12

Quick Aside re: Assumptions

• Behind each project is a set of beliefs and/or knowledge held by those who plan a project about how change will come about

• The validity of the connection between strategies, outcomes and impacts depends on the existence of conditions necessary for success

• We call these assumptions

Center for Youth and Communities, Brandeis University 13

Core Assumptions for Project

Compass (abbreviated)

• Underserved students are an asset to the institution.

• Sustained, institution-level change supporting increased success and retention of underserved students requires ongoing collaboration from across the college.

• “Islands of excellence” which retain underserved students but exist at the margins of the institution will not result in broad-based cultural change unless they are scaled up.

Change in institutional culture supporting the success and retention of underserved students must be supported by research.

An ongoing culture of evidence and inquiry is essential to the formulation, implementation, and improvement of practices and policies supporting underserved students.

• Colleges and universities committed to institutional change to better retain underserved students will benefit from collaborative relationships with peer institutions with like commitments.

Center for Youth and Communities, Brandeis University 14

A Few Hints

Center for Youth and Communities, Brandeis University 15

“ I think you should be more explicit here in Step Two.”

Center for Youth and Communities, Brandeis University 16

Strategies vs. Outcomes vs. Measures of

Success Examples

STRATEGIES

What you do

Offer faculty development in culturally responsive pedagogies.

OUTCOMES

What changes

Faculty increase use of culturally responsive pedagogies.

Develop social opportunities and groups for underserved students.

Develop and use statistical methods to predict which students are at greatest risk for dropping out.

Student participants have a greater sense of belonging and engagement.

Advisors use better information to identify underserved students who might need additional support early in students’ academic careers.

Center for Youth and Communities, Brandeis University

MEASURES OF

SUCCESS

How you know

Faculty report implementing new teaching strategies

(survey/interviews) and students’ assessments of faculty teaching improve

(course evaluations).

Students who participate in new opportunities report groups helped them feel more engaged (survey).

The number of underserved students identified as needing additional support increases.

17

Questions to ask yourself as you work on your logic model

• Is it reasonable to expect the initiative’s planned strategies will lead to the expected outcomes?

• Are all the outcomes supported by strategies?

• Are all target groups, strategies, and outcomes included?

• Are there sufficient resources to undertake the activities?

• Do all the components lead to one or more outcomes?

• Are the outcomes really outcomes, not activities?

Are the outcomes reasonably measurable?

• Are all stakeholders in agreement about the logic model?

Adapted from Porteous et al., 2002

Center for Youth and Communities, Brandeis University 18

Final Hints

• Focus on being outcome-driven not activitydriven

• Check logical connections across categories

• Revisit and revise periodically

• Can use to develop more detailed action plans

• OK to develop a logic model that is multiple pages

• It will get easier!

Center for Youth and Communities, Brandeis University 19

Summing up

Logic models:

• Create shared understanding

• Help with planning, implementation, evaluation and reporting

• Link activities to results: fill gaps and avoid mismatches across categories

Center for Youth and Communities, Brandeis University 20

“My question is: Are we making an impact?”

Center for Youth and Communities, Brandeis University 21

Download