Measuring for Success Module Nine Reflecting on the Previous Session • What was most useful from the previous session? • What progress have you made since the previous session? • Any comments or questions about the previous session? Session Overview • • • • • Exploring the Value of Measuring Deciding What to Measure Using Measures to Promote Success Tracking Appropriate Measures Creating a Regional Plan for Measuring Success Group Discussion What things do you monitor or measure as part of your organization? What are you seeking to learn from those measures? Why Should We Measure? • • • • • • • Know if your plan is working Make course corrections along the way Ensure accountability Encourage continued participation of partners Generate enthusiasm and public interest Foster buy-in with potential new partners Generate additional support – internal and external Measuring for Success Means . . . Thinking about: • Who will make changes • How people will do things differently • What outcomes or impacts you expect • Which key indicators can help determine progress Deciding What to Measure Traditional Measures Measuring What We Do (Outputs) Versus Measuring the Difference We Make (Impacts – Outcomes) Outputs: What We Do Outputs My Organization Does Things # Workshops Conducted # Counseling Sessions # Tech Asst. Sessions # Materials Distributed To/for People # Clients Served # Stakeholder Groups # Partnerships Outcomes: Three Levels of Measuring toward Success Do You Know Your ABCs? Attitudes, Knowledge, and Skills Short Term Behavior Conditions Intermediate Long Term Outcomes Success Using ABCs to Promote Success Measuring ABCs: An Example Your team decides to focus on fostering small business growth by providing training on developing and managing web-based sales. Strategy: Web Training for Small Business Owners Thriving Small Business Sector Measuring ABCs: An Example Problem: You have been conducting the training for five years, but small business numbers are static or declining. Discussion: What are some possible points at which the strategy failed to produce the desired condition? Don’t Forget Outputs Counting what we did tells part of the story: • How many times did you offer the training? • How many small business owners attended the training? These traditional measures are still an important link to the overall success. Measuring ABCs: Attitudes/Knowledge/Skills Strategy: Offered Web Training for Small Business Owners Participants scored poorly on post assessment Measuring Attitudes, Knowledge, and Skills Course Correction? Measures: Behavior Strategy: Offered Web Training for Small Business Owners Participants did not start websites Measuring Behavior Course Correction? Measures: Condition Strategy: Offered Web Training for Small Business Owners Participants started websites but have seen no income or job growth. Measuring Conditions Course Correction? Points for Course Correction Outputs Web Training for Small Business Owners Number of Trainings & Participants Outcomes Attitudes, Knowledge, and Skills Scores on Post Assessment Behavior Number of Websites Started Conditions Thriving Small Business Sector Number of New Websites Generating Income Tracking Appropriate Measures Key Questions in Developing a Plan • What information do we need? Does it already exist? OR Will you need to gather it? • Who has access to the information? • When should we gather? How often? • How can we gather the information? Quick Guides for Measuring We can’t measure everything! Some quick guides: • Be reasonable. • Be thoughtful. • Be realistic. • Be smart. Measuring Short Term: Attitudes/Knowledge/Skills Common Methods Include: • Pre/post survey • Skill assessment/demonstration • Practical application exercise • Willingness to participate in follow-up activities • Others? Measuring Intermediate Term: Behavior Change Common Methods Include: • Logging behaviors • Monitoring of completion of follow-up activities • Follow-up surveys • Observation • Other ideas? Measuring Long Term: Condition Change Common Methods Include: • Secondary data • Surveys/ follow up with participants • Other ideas? Creating a Regional Plan to Measure FOR Success Measurement Plan Template What you want to measure Outputs: Your Strategy The Participants Short Term: Attitudes/Knowledg e/Skills you want to change Intermediate: Behavior you want to change Long Term: Condition you want to change What information you need Who has the information How you will get the information How often/when you will get the information Measurement Plan Example Your Plan Using Handout Two: Draft a Measurement Plan for the ABCs you identified in Module Eight. Measurement Plan Feedback • Did you identify any new strategies or steps that you would need to incorporate in your plan of work (from Module Eight)? • What opportunities do you have in place for course corrections? • How will this plan help your team be successful? • What additional information might you need to guide the strategy to success? Last But Not Least: Inputs Inputs are the people, organizations, and resources that will be contributing to the success of your initiative. Without inputs, your regional plans cannot move forward. Measuring Inputs How are we doing as a regional team? It’s not just about numbers! How We Do Our Work • Meeting evaluations • Reflection questions • Quarterly evaluation meetings that focus on process and impact • Focus groups • Interviews • Other methods? Your Team: Partnership Evaluation • What has been a peak experience in this project? • Describe the best meeting we have had so far. • What would help future meetings be even more successful? • What has been a high point in our collaboration? • How might we make it even better? Partnership Planning • What plans can we put in place to ensure the viability of this partnership? • What strategies will work best with this regional collaboration? Your Plan: Wrapping It Up • How many points do you have for checking and correcting your course along the way? • Have you identified someone to take charge of each check-point? • Is the total measurement package manageable? • Are there points at which the plan could “fall through the cracks?” • Do you have the right mix of resources? Keeping Your Eyes on the Goal Did the strategies we implemented: • Move us closer to achieving our regional goals? • Make effective use of our regional assets, strengths, and opportunities? • Generate additional problems? • Other thoughts? Module Debriefing What are the takeaways from this session? • What topics did you find most helpful? • What did you find confusing? • What do you hope to implement as part of your regional team’s activities? • Other items you want to mention? Post Assessment Please take a few minutes to help us . . . Measure FOR Success . . . by completing the postassessment.