Metrics Summary, December 9, 2010 Kelly Humphry This is a (DRAFT) summary of the metrics/reporting requirements for the following: PRIA 2008 – required by grant PRIA 2010 – required by grant PESP Gold Level participation – 300 schools = PRIA 2010 deliverable (specific metrics are negotiable) School IPM 2015 – potential metrics for measuring school IPM implementation nationally IPM STAR – voluntary certification for schools achieving high level IPM implementation 2008 Report Card Survey – annual state survey given to IPM leaders to assess the status of IPM Tim Stock’s School Survey – successful online survey designed for school districts in the Western Region The PRIA 2010 grant requires us to establish a comprehensive set of online metrics and an online reporting method that school districts will use to report their coalition and demonstration performance measures. PRIA 2010 also requires PESP Gold Level participation metrics from 300 schools, comparative analysis for each state through the use of report card surveys, asthma incidence and pesticide use metrics in schools, and quarterly tracking for our outreach efforts (webinars, presentations, trainings, publications, conferences, newsletters, website hits, expanded membership, “Ask the Expert” usage, and listserv postings). This additional reporting will be the responsibility of state IPM professional leads, working group members, the national coordinator, and coalition leaders. The national coordinator will collect these metrics and compile them for grant reporting and to assess how our School IPM 2015 strategy is progressing towards IPM implementation in all schools by 2015. It is important that we do not replicate reporting burdens for the same people/entities. It is also important that we encourage regular reporting from all parties by designing easy to use online forms and surveys. The metrics listed above can be generally organized into the following four categories. There is some overlap in the metrics, but there should be no overlap in reporting by any one entity. *Metrics for demonstrations and coalitions are similar and could potentially be collected using one document (online or PDF form). The user would choose which type of project they are reporting on and only fill out that part of the form. Annual Report Card: This survey was developed in 2008 with input from PMSP meeting participants and others. Purpose: To document annual performance measures by state for progress towards the 2015 goal. A PRIA 2010 objective states that the report card should also collect information for comparative analysis for each state addressing laws, regulations, support and activities. Who reports: Completed by key IPM professionals in each state (Extension, state lead agency, NGO’s). A joint report after dialogue with knowledgeable colleagues is ideal. Submitted to: IPM Institute - national coordinator. National coordinator will then summarize report card results on an annual basis. Due: October 1. Results summarized by December 31. Action Item: This survey will be revised prior to recirculation in 2011. National steering committee - please review during first quarter of year 2011 and return to national coordinator by March 31, 2011 with comments. See current version attached. *Metrics/Performance Measures for school demonstrations: These metrics are required for our PRIA 2008 and 2010 grants as well as for the initial and final assessments in the schools to prove IPM implementation success. This information overlaps with the School IPM 2015 strategy’s “potential metrics” and can be used for satisfying (some) criteria for IPM STAR and for PESP Gold Level participation. Purpose: Must first establish a baseline assessment of pest problems, asthma incidence, and pesticide use/current treatment and training; then follow up with regular documentation of IPM program performance in these areas. Who reports: School Districts. More specifically – IPM coordinator with assistance from facility manager or the point person for pest management decisions in the school. Submitted to: National coordinator for quarterly or yearly grant reports. Due: Quarterly or yearly? Action Item: Finalize list of metrics needed, and finalize best method for reporting: survey, online form, PDF form solicited and sent via email or mail. See attached draft for list of potential performance measures. *Metrics/Performance Measures for coalitions: These metrics are required by our PRIA 2010 grant and build upon successful completed demonstrations in the region. An MOU is required at the beginning of coalition formation and baseline metrics are required when the coalition starts a new school demonstration. Metrics overlap with demonstration reporting requirements above. Purpose: Must first establish a baseline assessment of pest problems, asthma incidence, and pesticide use/current treatment and training; then follow up with regular documentation of IPM program performance in these areas. Who Reports: Coalition leaders Submitted to: National coordinator for quarterly or yearly grant reports. Due: Quarterly or yearly? Action Item: Finalize list of metrics needed, and finalize best method for reporting: survey, online form, PDF form solicited and sent via email or mail. See attached draft for list of potential performance measures. Metrics for grant reports and for benchmarking PMSP strategy objectives: These metrics are usually reported via updates during working group conference calls, listservs, on websites, and via formal and informal communications between key regional and national IPM professionals working on the School IPM 2015 strategy and grant management. I would like to create a simple PDF document for quarterly tracking of our outreach efforts (webinars, presentations, trainings, publications, conferences, newsletters, website hits, expanded memberships, “Ask the Expert” usage, and listserv postings). Purpose: To document outreach efforts for PRIA 2010 grant objectives and reports and to measure our progress towards our School 2015 goal. Who reports: State IPM leaders, coalition leaders, working group members, national steering committee members. Submitted to: National coordinator Due: Quarterly Action Item: National coordinator will create PDF form that can be filled out online or sent via email and returned. School IPM Demonstration & Coalition Performance Measures Regional Working Groups metrics needed for the PRIA 2008 and 2010 grants (DRAFT) For each demonstration site, please report the following: - Numbers of participating school systems and students, staff impacted. - Before and after overall pesticide use (quantity and number of times used) and pest complaints in participating school systems. - Estimated number of students with asthma - Asthma attacks in schools resulting in asthma related absences in staff and students - Allergens present in school - Acreage of landscaped area, square footage of building - Before and after pest-management-related costs. - Number of pest problems solved without a pesticide application. - Before and after pesticide inventory and product-use lists (this includes biopesticides, reducedrisk and conventional pesticides as well as non-chemical products and practices such as monitoring traps, door sweeps, rodent proofing, etc.). - Number of workshops and meetings, and attendees. - Participant evaluations of workshops and meetings. - Numbers of school systems with: o Written IPM policies and plans o Trained IPM coordinators o IPM or other safety related committees formally charged with IPM. - Qualifications required for individuals applying pesticides in schools. - Numbers of staff trained in IPM. - Numbers of schools including IPM education in students’ curricula. - Numbers of schools using pest management professional service providers with advanced optional qualifications, e.g., Quality Pro Schools, Green Shield Certified. - Number of PESP and/or IPM STAR certified school systems. - For Gold Level PESP participation: Has participated in PESP events (number of events; e.g. meetings, workshops). - Internal and external promotion of PESP in school district. - Is your demonstration externally funded? For each coalition site, please report the following: - Numbers of participating school systems and students, staff impacted. - (Optional) Before and after overall pesticide use and pest complaints in participating school systems. - Estimated number of students with asthma - Asthma attacks in schools resulting in asthma related absences in staff and students - Allergens present in school - Acreage of landscaped area, square footage of building - (Optional) Before and after pest-management-related costs. - (Optional) Number of pest problems solved without a pesticide application. - Before and after pesticide inventory and product-use lists (this includes biopesticides, reducedrisk and conventional pesticides as well as non-chemical products and practices such as monitoring traps, door sweeps, rodent proofing, etc.). Include quantity and number of times used. - - Number of workshops and meetings, and attendees. Participant evaluations of workshops and meetings. Numbers of school systems with: o Written IPM policies and plans o Trained IPM coordinators o IPM or other safety related committees formally charged with IPM. Qualifications required for individuals applying pesticides in schools. Numbers of staff trained in IPM. Numbers of schools including IPM education in students’ curricula. Numbers of schools using pest management professional service providers with advanced optional qualifications, e.g., Quality Pro Schools, Green Shield Certified. Number of PESP and/or IPM STAR certified school systems. For Gold Level PESP participation: Has participated in PESP events (number of events; e.g. meetings, workshops). Internal and external promotion of PESP in school district. Is your coalition externally funded?