Program & Proposal Enhancement Tools Welcome These tools were initially developed and assembled as Proposal Enhancement Tools by a team at Purdue University under a continuous quality improvement grant from the Provost’s Office. Their goal was to provide a self-help process that would make projects and grant proposals more competitive. The tools have been adapted by Lena Etuk and Sam Angima at Oregon State University Extension Service to make them useful for the needs assessment process. You can use these tools within the context of needs assessment in two ways: 1) before you conduct a needs assessment to remind yourself of the information you need to gather, or 2) after you conduct a needs assessment to organize your findings to strengthen your program or apply for funding. If you choose to use the tools after you conduct a needs assessment, it is important to remember that needs assessment has three phases: Phase 1. Pre-assessment: Get organized and find out what is already known about the issue Phase 2. Assessment: Gather new data about the issue, if necessary Phase 3. Post-assessment: Design and implement solutions And that these are the key questions guiding all phases of a needs assessment: 1. What is the current situation for clientele, employees, and administrators? 2. What should the situation be? 3. If there is a gap between what is and what should be, then: a. What are the consequences for not changing the current situation? b. What is causing the gap? c. What are some possible solution strategies to reduce the gap? Make sure that the needs assessment you conducted or are about to conduct answers all of these questions. 1 Program & Proposal Enhancement Tools Knowing your target audiences, anticipating impact, and identifying evaluation methods are critical elements that distinguish successful programs. Yet these elements are often given only cursory attention when we are planning. The exercises and questions in the eight enclosed worksheets are designed to help you effectively identify and describe 1) the target audiences for your program, project, or new course, 2) the anticipated impact of your efforts, and 3) the measures you propose to evaluate your success. These worksheets will help you verify the next best steps in your program or proposal writing process and clearly delineate what is needed to support your planning. They’ll move you from the general to the specific, prompting you to organize and analyze information you have gathered from a variety of sources and experiences. Contents 1. Needs Assessment Checklist 2. Initial Issue Description & Anticipated Outcome Worksheet 3. Audience Definition Worksheet 4. Aggregation Worksheet 5. Projecting Anticipated Impact & Costs/Resources Worksheet 6. Inputs to Outcomes Worksheet 7. Evaluation: Measurement Levels & Methods Worksheet 8. Where Do You Go from Here? Extension and Experiment Station Communication Oregon State University 541-737-3311 2 Needs Assessment Checklist □ Ask these key needs assessment questions throughout the process 1. What is the current situation for clientele, employees, and administrators? 2. What should the situation be? 3. If there is a gap between what is and what should be, then: a. What are the consequences for not changing the current situation? b. What is causing the gap? c. What are some possible solution strategies to reduce the gap? Phase 1 – Pre-Assessment □ □ □ Get organized, by gathering background information about the issue of concern and the focus for the needs assessment Form a Needs Assessment Committee (a small group of people who include clientele and other stakeholders to help focus the needs assessment, gather and analyze reliable data, brainstorm ideas, and prioritize decision making for actions identified in the needs assessment) Work with the Needs Assessment Committee (NAC) to: □ □ □ □ Learn as much as possible about preliminary “what is” and “what should be” conditions from available (secondary) data sources Learn as much as possible about consequences for not bringing the current situation in line with the desired situation from available (secondary) data sources Learn as much as possible about causes of gap between “what is” and what should be” from available (secondary) data sources Synthesize the data, make it meaningful, and decide to move to Phase 2 or Phase 3 of the needs assessment Phase 2 – Assessment 3 □ □ □ If Phase 1 information did not provide all the answers to make needs based decisions, determine the best method for collecting new information about “what is” and “what should be” conditions from primary data sources, as well as consequences for not changing the current situation and causes (e.g., interviews, surveys, focus groups, etc.) Collect new information about “what is” and “what should be” conditions, consequences, and causes for not changing the current situation Analyze new information in conjunction with information gathered in Phase 1 Phase 3 – Post-Assessment □ Work with Needs Assessment Committee (NAC) to prioritize needs □ Deliberate and articulate causes of high-priority needs □ Convert causes into solution strategies □ Prioritize solution strategies based on feasibility and other criteria □ Complete Program & Proposal Enhancement Tools □ Use completed Program & Proposal Enhancement Tools report to seek funding or support for a program □ Develop action plan □ Implement the plan □ Monitor the implementation of the plan □ Evaluate the effectiveness of the program in reducing the gap □ Report outcomes to NAC, stakeholders, and policy makers □ Evaluate the needs assessment process 4 Initial Issue Description & Anticipated Outcome Worksheet Directions: After Phase 1 and Phase 2 of your needs assessment you should have worked with your Needs Assessment Committee (NAC) in Phase 3 to identify a high priority solution strategy for the need you assessed. Precisely describe the current situation and the desired situation you want this strategy or your program to address: Describe the current situation. Describe the desired situation. For example: Currently, 15% of children (under age 18) in Oregon are obese For example: In 2030, only 5% of children (under age 18) in Oregon will be obese 5 Initial Issue Description & Anticipated Outcomes (continued) Briefly identify any “gaps” that exist between the current and the desired situation. List the cause(s) of the “gaps” and the sources of information you used to identify the cause(s). Cause: Source: Cause: Source: Cause: Source: Cause: Source: Cause: Source: Cause: Source: List, in order of priority, who will benefit by addressing this issue and how they will benefit. Who: How: Who: How: Who: How: Who: How: List, in order of priority, the specific actions (developed in Phase 3 of the needs assessments process) you and your NAC identified will be required to address this issue. Describe how addressing the issue will help to fulfill Oregon State University’s Land Grant mission in Research, Extension, and/or Teaching. 6 Audience Definition Worksheet Directions: Based on the findings from your needs assessment, define some of the characteristics of your target audience—the people listed as top priority in the previous worksheet. What is their age range? What is their educational background? □ □ □ □ □ Where do they live, primarily?: □ □ □ □ □ □ less than high school high school graduate/GED some college undergraduate degree advanced degree Income □ □ □ □ □ □ □ urban suburban rural mix, mostly urban mix, mostly suburban mix, mostly rural Gender □ □ □ □ □ Less than $20K $20K-$34K $35K-$44K $45K-$59K $60K-$74K overwhelmingly male overwhelmingly female equal mix unequal mix, some more males unequal mix, some more females $75K-$100K more than $100K Ethnic origin __________________________________ (if appropriate) Why do members of this audience “need” the information, service, or program you listed as the top priority action in the previous worksheet? What unique knowledge/skills/abilities do these audience members possess now? What knowledge/skills/abilities will audience members possess after you have addressed this issue with them? 7 Audience Definition (continued) Directions: Using your answers to the previous questions as a guide, take a few minutes to describe briefly the characteristics of your secondary audience and other potential audiences. Secondary Audience Who are they? Why are they important? Potential Other Audiences (People who are interested but not directly involved in the issue) Who are they? Why are they important? Who are they? Why are they important? Who are they? Why are they important? 8 Aggregation Worksheet Directions: Using the information you compiled in the previous worksheets as a starting point, follow the instructions below. Note: This is where you can start aggregating your information and using it to answer new questions. This is where you can get a big enough picture to make some important decisions. Describe how you will address this issue based on the decision process in Phase 3 of the needs assessment process. (The program you propose implementing) Describe what your primary audience will be better able to do when you address this issue. Describe the knowledge/skills/abilities your primary audience will be able to demonstrate after you address this issue. List or describe others who are addressing this issue currently or are likely to address it within the next year. Describe how you/we are better positioned to address this issue. Describe how you/we will know this issue has been addressed successfully (i.e. how will you/we know success?). 9 Describe the probabilities of success. 10 Aggregation (continued) Describe any barriers to success. List or describe the audience(s) that have verified the demand to address this issue. What is the size of this audience? How long will the audience maintain interest in this issue? (i.e. other than your target audience, who is supporting this program/project?) List all potential partners who may effectively collaborate with you on this effort. Who might be negatively affected by this program and what effect will this have on the overall success of the program? Review all the information you compiled in the previous worksheets. Have you identified an audience with a need that you/we are best suited to fulfill? This is your go/no go point. 11 Projecting Anticipated Impact & Costs/Resources Worksheet Issue & Audience: _____________________________________________________________________________ _____________________________________________________________________________ Directions: Remember, a strong anticipated impact statement is a SMART Anticipated Impact Statement: it’s Specific, Measurable, Attainable, Relevant, and Trackable. Use the formula below to help you develop your SMART Anticipated Impact Statement(s). Don’t forget your secondary audience. Audience + Action Verb + Specific Area + Measurement + Completion = Anticipated Impact Statement Note: The Anticipated Impact Statement may not be expressed in the exact sequence described above. However, each element of the formula must be represented in each Anticipated Impact Statement you create. Completion + Audience + Measurement + Action Verb + Specific Area = Anticipated Impact Statement By the end of the program, producers who participate in the Marvelous Marketers program will realize an extra $10,000 to $20,000 in annual returns through improving their marketing skills. Action Verb + Measurement + Audience + Specific Area + Completion = Anticipated Impact Statement The Brighter Futures program will reduce the repeat pregnancy rate in Lucky County from 50 percent to 10 percent by targeting young, at-risk mothers with a two-year education campaign. Audience + Action Verb + Specific Area + Measurement + Completion = Anticipated Impact Statement The number of older women participants in the Financial Empowerment program who feel they have the ability/power to affect their own financial positions positively will increase from under 50 percent to 75 – 90 percent over the course of the eight-week program. Completion + Audience + Action Verb + Measurement + Specific Area = Anticipated Impact Statement Through successfully completing our Food Safety Plus educational program, food service workers will demonstrate at least a 50-percent increase in their knowledge of food safety. Your Anticipated Impact Statement (version 1): Your Anticipated Impact Statement (version 2): 12 Projecting Anticipated Impact & Costs/Resources (continued) Issue & Audience: _____________________________________________________________________________ _____________________________________________________________________________ Directions: Some of the basic, initial costs to consider are listed below. Place a check mark next to all potential costs that apply, record an approximate amount, and provide a brief rationale for the cost in the space provided. Note: An expanded version of this form is available through your business office. Potential costs: □ □ □ □ □ □ □ □ □ Development costs Amount Provide rationale for these costs below. $ Instructional design (if appropriate) Technology access/acquisition Program materials Instructor/Facilitator costs Facilities costs Travel/lodging/meals Administrative/overhead costs FTE(s)* Number of FTE(s) x time on task x salary = project cost □ □ □ Other Other Other *Total $ 13 Inputs to Outcomes Worksheet Issue & Audience: _____________________________________________________________________________ _____________________________________________________________________________ Directions: Describe the benefits of addressing the issue in the space provided below. Remember, the benefits may accrue over time. Inputs: Provide a rationale for the resources you have determined need to be devoted to addressing the issue. Outcomes: Describe the value of addressing this issue, quantifying whenever possible. (Estimate total cost/resources, including human resources, as you described them on the “Projecting Anticipated Impact & Potential Costs/Resources” worksheet.) (Consider economic value [input/output], environmental value, and human value as well as the potential short- and long-term results.) 14 Evaluation: Measurement Levels & Methods Worksheet Issue & Audience: _____________________________________________________________________________ _____________________________________________________________________________ Directions: Below are four Measurement Levels through which you can evaluate the effectiveness of your project. By the time you complete this worksheet, you will be able to select (and check) those Measurement Levels that apply. □ □ □ □ Level 1: Reaction to the program - All programs. Level 2: Learning - Programs that require participants to retain information or apply a skill. Level 3: Application - Programs that require participants to change their behavior when they return to work, school, home, etc. Level 4: Return on Investment (ROI) - Programs that are a top priority to the college/department/unit and can be linked to financial numbers. (ROI may be calculated over time, and social/environmental/fiscal outcomes may be considered.) Directions: Check the best answer to each question, and follow the appropriate instructions. 1. □ □ Do “stakeholders” require evidence of results? Yes. Levels 3 and 4 of evaluation must be completed. Use the “Measurement Methods” table (see reverse side) to help you select the best measurement methods. Check Levels 3 and 4 above. No. Complete # 2 on the next page. 15 2. Perform the exercise below, and follow the appropriate instructions. Rate the scope of this initiative (the average of a. and b.): a. The number of people who might be influenced by this initiative 1 = very limited; 10 = very extensive and comprehensive. 1 b. 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 8 9 10 8 9 10 Rate the profile of this initiative (the average of a. and b.): a. The type of attention this initiative will generate 1 = very low; 10 = very high. 1 b. 2 3 4 5 6 7 The status of the people who will openly support this initiative. 1 = very low; 10 = very high. 1 3 The range of backgrounds these people represent. 1 = very limited; 10 = very extensive and comprehensive. 1 2 2 3 4 5 6 7 Total the two ratings. Scope rating (the average of scope a. and b.) + Profile rating (the average of profile a. and b.) Total Rating Is the Total Rating equal to or greater than 13? □ Yes. □ No. _____ _____ _____ Consider applying Levels 3 and 4 of evaluation. Use the “Measurement Methods” table to help you select the best measurement methods. Check Levels 3 and/or 4 if you determine they are appropriate. Select Level 1 evaluation, and determine if Levels 2, 3, and 4 would be appropriate. Use the “Measurement Methods” table to help you select the best measurement methods. Check the levels you determine are appropriate. 16 Measurement Levels & Methods (continued) Issue & Audience: _____________________________________________________________________________ _____________________________________________________________________________ Directions: Select the appropriate Measurement Levels, and check the best methods for performing them. Note: You may want to use two or more measurement methods in order to validate the outcomes of the evaluation. Measurement Methods Level 1 The evaluation form you use could solicit reaction to: Level 2 Learning could be demonstrated by: Level 3 Application could be determined by: Level 4 ROI could be determined by: (ROI may be calculated over time) □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ Content Duration Handout materials Videos Exercises Simulations Games Learning environment Instructor/facilitator Self-assessments Team assessments Facilitator assessments Performance testing Simulations Case studies Skill practices Role plays Exercises Follow-up surveys Follow-up questionnaires Observation Interviews with participants Follow-up focus groups Program assignments Action planning □ □ Performance contracting □ □ □ □ 17 Control groups Trend line analysis Forecasting methods Participants’ estimate of impact (percent) Supervisor’s estimate of impact (percent) Management’s estimate of impact (percent) Use of previous studies Subordinate’s report of other factors Calculating/estimating the impact of other factors Use of customer input Where Do You Go from Here? Congratulations! By completing the Program & Proposal Enhancement Tools, you’ve organized, analyzed, and compiled a valuable information file. Incorporate the information you’ve developed at the appropriate points in any type of proposal you submit for this program to give you an edge in getting it approved by your supervisor, accepted by a granting agency, and funded by a grant or contract. To really stand out against the competition, put your Anticipated Impact Statement(s) right up front in your abstract, and repeat throughout. The Program & Proposal Enhancement Tools will be posted on the Oregon State University Extension website, in the “Employees” section here <http://extension.oregonstate.edu/employees/employee-resources/evaluation/needs> as soon as the worksheets have been converted to a PDF fillable form file. Good luck! 18