United States Department of Agriculture Citizen Science Project Plan for [Name of your project] on the [Name of your unit] Forest Service Unit Name Month Year Project Leads: Name Address Address Address Phone No. Name Address Address Address Phone No. In accordance with Federal civil rights law and U.S. Department of Agriculture (USDA) civil rights regulations and policies, the USDA, its Agencies, offices, and employees, and institutions participating in or administering USDA programs are prohibited from discriminating based on race, color, national origin, religion, sex, gender identity (including gender expression), sexual orientation, disability, age, marital status, family/parental status, income derived from a public assistance program, political beliefs, or reprisal or retaliation for prior civil rights activity, in any program or activity conducted or funded by USDA (not all bases apply to all programs). Remedies and complaint filing deadlines vary by program or incident. Persons with disabilities who require alternative means of communication for program information (e.g., Braille, large print, audiotape, American Sign Language, etc.) should contact the responsible Agency or USDA’s TARGET Center at (202) 720-2600 (voice and TTY) or contact USDA through the Federal Relay Service at (800) 877-8339. Additionally, program information may be made available in languages other than English. To file a program discrimination complaint, complete the USDA Program Discrimination Complaint Form, AD3027, found online at http://www.ascr.usda.gov/complaint_filing_cust.html and at any USDA office or write a letter addressed to USDA and provide in the letter all of the information requested in the form. To request a copy of the complaint form, call (866) 632-9992. Submit your completed form or letter to USDA by: (1) mail: U.S. Department of Agriculture, Office of the Assistant Secretary for Civil Rights, 1400 Independence Avenue, SW, Washington, D.C. 20250-9410; (2) fax: (202) 690-7442; or (3) email: program.intake@usda.gov. USDA is an equal opportunity provider, employer and lender. NOTES: Blue text in this template consists of instructions, examples, and guidance (references to relevant sections in the Citizen Science Toolkit). It is intended to be deleted and replaced with your own content. Be sure your project plan is written in plain language so that everyone can easily understand it. Refrain from using acronyms and jargon, and define terms. Ensure it complies with accessibility requirements (section 508 of the Rehabilitation Act) before sharing it with the public. To update the Contents, put your cursor in the list and press F9. This template borrows heavily from: https://www.epa.gov/sites/production/files/201606/documents/quality_assurance_template_for_citizen_science.pdf Contents Introduction ..................................................................................................................................... 1 Project Overview .......................................................................................................................... 1 Community Input ........................................................................................................................ 1 Project Plan ...................................................................................................................................... 1 Problem Definition ...................................................................................................................... 1 Project Objectives ........................................................................................................................ 1 Team Members ............................................................................................................................ 1 Project Timeline .......................................................................................................................... 2 Budget Break-Out ........................................................................................................................... 2 Project Design ................................................................................................................................. 4 Project Location .......................................................................................................................... 4 Existing Data............................................................................................................................... 4 Data Quality Objectives .............................................................................................................. 5 Sampling Design ..........................................................................................................................7 Data Users ....................................................................................................................................7 Volunteers and Training ................................................................................................................10 Specialized Experience ................................................................................................................7 Assessments and Oversight .........................................................................................................7 Data Management........................................................................................................................7 Project Evaluation.......................................................................................................................... 11 Reporting and Sharing Results ...................................................................................................... 11 How this Plan will be Updated ...................................................................................................... 11 Appendix A: Key Messages ............................................................................................................ 13 Appendix B: Communication Tools ............................................................................................... 14 Appendix C: Forms and Agreements ............................................................................................. 15 Appendix D: Project Organization Chart ....................................................................................... 16 Appendix E: References Cited ....................................................................................................... 17 Tables Table 1. Primary points of contact ................................................................................................... 1 i Citizen Science Project Plan Introduction Project Overview Provide the context and briefly describe who, what, where, when, why and how for your project. Describe why citizen science is the best approach and the specifics of the project location. Reference Chapter 1 and Chapter 2. Community Input Describe how the community (public and partners) was involved in the development of the project. Community engagement is iterative and it is important to recognize the need for feedback loops and periodic adjustments to the plan over time. Project Plan Problem Definition Clearly state the problem and environmental questions being addressed by the project. State why this work needs to be done, identifying the reasons for conducting the work and/or the lack of information relating to the project. Discuss any previous work or data that has been collected as they relate to this project. Project Objectives Describe how the project objectives will answer the problem presented in the problem definition above. Include information about meeting science or land management objectives, potential benefits of the project for the Agency and partner organizations, how the project will meet Forest Service and partner strategies and/or decisions, community engagement and outreach goals, etc. Include the tasks that will be completed to provide or collect information to address the problem. Reference Chapter 2. Team Members Fill in the name, title, organization affiliation for key Forest Service, partner, and community contacts in the table below. State what work/task each individual will be doing throughout the project. Reference Chapter 3. Table 1. Primary points of contact 1 Project Name Name Title Organization Responsibilities (specific to this project) Email Address Phone Number Project Timeline In the table below, list all major project activities that will be performed during the course of the project including major milestones. Provide estimates of the timeframe expected for the activities to be conducted and/or completed. Table 2. Project Activities and Timeline Activities Individual/organization responsible for activity completion Timeframe work will be done Budget Break-Out Break down the estimated costs for your project. Below is a table with some typical items to be considered in a project budget. Listing the details of who paid for what, will also help with your project evaluation and reporting requirements. Include in the Notes section, the specifics of the estimated hours and cost/unit where applicable. Table 3. Budget Break-Out Expense Category Expense Description CitSci Fund Cost Personnel Salaries and Wages Fringe Benefits 2 Forest Service Unit Cost Partner Cost Citizen Science Project Plan Stipend Contracted Services Total Personnel Travel Mileage Per Diem Total Travel Equipment and Supplies Communications and Outreach Total Equipment and Supplies Technical Costs Database Development App Development Data Analysis Software Training Development Total Technical Costs Other costs Total Other Costs Totals 3 Project Name Total Project Budget Project Design Describe the information collection design including protocols used, volunteer training, data collection instruments, data management, quality assurance/quality control measures, data analysis, and sharing of results. Describe available data and those being acquired that will benefit the project. Reference Chapter 4. Project Location Provide a description of the site and sampling locations and how they were chosen. Provide the rationale for selecting sample locations and what is going to be sampled. Provide a map showing the location and any other relevant information for the project. Tie this information back to the goals and objectives of the project. Existing Data Identify all existing data that will be used for the project, and their originating sources. Specify how the existing data will be used, and the limitations on their use. • In the Existing Data section state what existing data you will use. • In the Data Source section state where that data will come from. • In the How Data Will Be Used section state the need for this data and/or what purpose it will be used for. • In the Acceptance Criteria section state what the requirements are for the data in order for them to be used in the project. For example, if you are looking for temperature data for a water body collected in July, then temperature data collected in June would not be acceptable for the project. Data collected with a certain instrument or by a certain method are also instances where the collected data may not be acceptable for the project. Table 4. Existing Data Sources Existing Data Data Source How Data will be Used 4 Acceptance Criteria Citizen Science Project Plan Data Quality Objectives Use this template to develop the data quality objectives that define the type, quantity and quality of data needed to answer specific questions, and support proper decisions. Fill in all information appropriate for the project. Complete this template for field, existing data and laboratory activities, if your project includes these components. Precision Precision is defined as the ability of a measurement to consistently be reproduced. Repeated measurements are usually used to determine precision. In the case of repeated measurements, one would see how close those measurements agree. If repeat measurements will be taken state how close those measurements need to agree by. Examples: Field - Duplicate temperature profiles will be taken at all three sampling stations during each sampling event. The temperature readings must agree within ±0.2°C. Laboratory - Results for duplicate algae (chlorophyll a) samples must agree within 25%. Existing data - Check to see that precision was checked during the original project that produced the temperature data we are using. Bias Bias is defined as any influence in the project that might sway or skew the data in a particular direction. Taking samples from one location where a problem is known to exist, instead of taking samples evenly distributed over a wide area, is one example of how data can be biased. State any biases that could potentially exist and how they will be addressed in the project. Examples: Field - All of our sampling locations are in the middle of the Twisting River. We may miss temperature or algal dynamics near the banks of the river. Given the limits of the budget and the screening level nature of the project, mid-point stream samples are sufficient. Laboratory – Blank filters and calibration standards will be used. Calibration standards with concentrations from 0.05μg/L- 200μg/L will be used. Resulting fluorometer readings must be within 10% of the known concentration. Existing data- The buoy is located upstream of the discharge point. Baseline temperature data will only be used from that one location in the Twisting River. There is not any baseline temperature data for areas downstream in our study area. Representativeness Representativeness is how well the collected data depicts the true system. Describe how the collected data will accurately represent the population, place, time and/or situation of interest. Examples: Field- We are sampling within a 1 km section of the river, upstream, at the discharge and downstream to capture temperature and algae (chlorophyll a) in the Twisting River near 5 Project Name the Hot Pink Purple Spotted Hippo power generation plant. We are only assessing during April – October. Therefore, data is not meant to be representative of conditions during winter months. Existing data-The temperature data from the buoy records temperatures upstream of the discharge point from the power generation station and should provide representative baseline temperatures of the Twisting River. Comparability Comparability is defined as the extent to which data from one data set can be compared directly to another data set. The data sets should have enough common ground, equivalence or similarity to permit a meaningful analysis. State if the data is intended to be compared to other data sets and how this will be achieved. Examples: Field- We are using a YSI sonde and standard methods so our data will be comparable with data from other studies. Laboratory – We are using standard EPA Method 445.0 so our data will be comparable with data from other studies. Existing data- Our YSI sonde reads temperature to the same sensitivity as the XYZ University Buoy. So we will have comparable baseline temperature data. Completeness Completeness is the amount of data that must be collected in order to achieve the goals and objectives stated for the project. State how much data will need to be collected in order for the project to be considered successful. This can be stated as a total number of samples or a percentage of data collected. Examples: Field- We will collect and analyze 100% of algae (chlorophyll a) samples, 42 samples and 14 duplicates throughout the project. If weather or other issues impede a sampling event, the event will be rescheduled. Existing data-Temperature data from the buoy from April-October for the past 4 years will be used in this study. If the buoy was not functioning properly during any period of that time, that data will not be used. Sensitivity Sensitivity is essentially the lowest detection limit of a method, instrument or process for each of the measurement parameters of interest. State the sensitivity needed for the instruments, methods or processes used for the project in order to obtain meaningful data. Examples: Field- The YSI sonde has a sensitivity of 0.1°C for a temperature range of -5°C to 95°C. Depth measurements will be made in cm. Existing data- The temperature sonde on the buoy has a sensitivity of 0.1°C. 6 Citizen Science Project Plan Laboratory- Method 445.0 has a method detection limit of 0.05μg chl a/L Sampling Design For this section, describe and justify the data collection activities. Include location specific information, such as GPS coordinates or landmarks, for the data collection locations. Provide information about the frequency of sampling and the collection of quality control samples. Include information about your plans for sample identification and transportation. Specialized Experience If any individuals have specialized experience that will be utilized by the project complete the specialized experience table. State who the individual is, what specialized experience they have related to the project and their years of experience. Table 6. Specialized Experience Person Specialized Experience Number of Years of Experience Assessments and Oversight Assessments and project oversight include various reviews to identify shortcomings or deviations from the project. For each type of assessment, describe procedures for handling quality assurance and project deviations encountered during the planned project assessments. Fill in all necessary information. Table 7. Data Quality Assessments and Oversight Assessment Type Frequency of Assessment What is Being Assessed Who will Conduct the Assessment How Issues or Deviations will be Addressed On-Site Field Inspection 2 weeks into sampling season and mid-season Undergraduate and high school volunteers XYZ University against QAPP/SOP Ms. Brooke Trout Re-train if necessary Data Management Describe the data management processes used throughout the life of the project. Data management includes: recording and transcribing field notes, logging and retrieval of 7 Project Name instrument data, transmittal of automated field and laboratory results, data transformation and reduction procedures, compilation of survey results, and data storage, retrieval and security uses throughout the project. Describe the way data handling errors will be controlled (i.e. spot checks for transcription and calculation errors). Create a table, or provide a narrative statement in response to these questions. Data Management Process How will you manage data so that it may be traced from planning through sampling and analysis to assessment to final use and storage? Data Management Procedures Do you have existing standard operating procedures (SOPs), such as for record keeping, document control, storage and retrieval of data, that you will reference? If not, how will you describe the procedure(s)? Make sure through either SOPs, tables, text, etc., you include data management of all phases of your project, including, as appropriate, planning, field, lab, assessment, storage and retrieval, project planning, data from other sources, etc. Field & Laboratory Data Management Table How will all data be recorded? Will data be transcribed from datasheets to an online database? What percent of data will be checked for accuracy and transcription errors? Who will check for discrepancies in data entries, How? How will lab results be delivered and by whom? How will data that did not meet the QC requirements of the lab be qualified? Will data once received be entered into the electronic database? By whom? If applicable will electronic files be backed up daily? How will original data be stored and for how long? How will you correct raw data for QA and QC? How will you ensure access to data by appropriate parties in various stages of processing (e.g., raw, under QA review, final)? Data Handling Will data be generated by hand (such as in the field), collected from literature or other sources (existing data), from computerized equipment or instruments and/or computer generated (such as in the lab or during review of the data)? Management Requirements Do you plan for your data to be used by or incorporated into Forest Service or other Agency databases? List those here and how and when you will add your data to these systems. 8 Citizen Science Project Plan Data Review and Usability Include in this section the types of checks that will be performed during and at the end of the project to determine if the data collected is usable for achieving the objectives of the project. Although data verification, validation, and usability are typically conducted sequentially, it may be beneficial (and more cost effective) for many projects to combine steps. For example, the entity conducting the verification could also conduct the first step of the validation process concurrently. Create a table, or provide a narrative statement in response to these questions. Data Review Procedure Describe the overall methods or procedures you will use to review the data, including verification and validation. If issues are found, who will resolve them and how? How will you communicate the results and any qualifications to the user(s) so they understand them? Data Review List the kinds of data checks you will perform, such as: deviations from the QAPP or SOPs; reviews of calibrations; transcription and calculation errors; and missing data. Data Review Criteria Describe the criteria you will used to determine whether to accept, reject or qualify data. (For example, some data may be outside acceptable limits, but may be used and will be considered “qualified”.) Data verification Describe your verification process. You should address completeness goals, correctness and conformance of collected data against the planned methods and/or procedures. Examples of records commonly included in the verification process include: field logs; chain of custody forms (field and laboratory); laboratory receipt records; refrigerator and freezer logs; certificates for standards or solutions; instrument logs or bench notes; instrument readouts (raw data); calculation worksheets; quality control (QC) results; and internal laboratory checklists. Data validation The validation process should address the analytical quality of collected data. Data validation is usually performed by people independent of the activity being validated. Validation typically occurs after verification and ensures that the data and information used in decision making are of appropriate quality for their intended use. It is, unfortunately, possible for the plans to have been followed meticulously, but the end results not suitable for meeting the project objectives. The most usual cause for failure in validation is that the circumstances that led to the collection of data may have changed during the project’s life-time. Validation should address: how the sample collection, handling, field analysis, and analytical laboratory project data were performed; the specific method used to produce data for each analytical group, matrix, and concentration level; 9 Project Name the procedures and criteria used for data information operations, which may include: the electronic or manual transfer, entry, use, and reporting of data for computer models, algorithms, and databases; correlation studies; and data plotting; how field sampling, handling, and analysis activities are documented (such as QC signatures in field logs and QC checklists); and the name of the person, identified by title, responsible for data validation. Data Usability Data usability is determined following the verification and validation processes. How will you reconcile your project’s final results with your original objectives? How will you examine the data to see if there are differences between the results and the original objectives or expectations? How will you document and communicate with the data user any differences or limitations on the use of the data? Data usability is the final inspection of the data and discusses previously identified analytical methods that will be used to analyze the data to identify possible anomalies or departures from assumptions established in the planning phase of data collection. These departures should describe how reconciliation with user requirements will be documented, issues resolved, and how limitations on the use of the data will be reported. Any potential limitations of data interpretation should be documented in the metadata records associated with these data and included in the final project report. Data Analysis Identify who will analyze the data and describe what data will be analyzed and under what timeframe. Describe how the data will be summarized or analyzed (e.g., qualitative analysis, and descriptive statistics, or inferential statistics) to meet project objectives. If descriptive statistics are proposed, list how the data will be summarized (e.g., mean, median, standard error, or minimum and maximum values). If an inferential method (which allows you to apply sampling results to a general population) is proposed, an indication of what is specifically proposed (hypothesis test, confidence interval, or confidence limit) should be indicated. Data Users State who will use the data and what decisions or conclusions will be made based on the data. Include any action levels or standards to which the data will be compared. Volunteers and Training Reference Chapter 5. Describe your communication and outreach efforts, how volunteers will be on-boarded and involved throughout, volunteer retention, recognition, mentorship, etc. In this section, state any required training that an individual involved with the project would need. Also include any refresher trainings that may be conducted. 10 Citizen Science Project Plan • In the Personnel/Group to Be Trained section, state who will need the specific training and how many people will be trained. • In the Description of Training section, state who will perform the training and what kind of information the trainee will learn. • In the Frequency of Training section, state how many times the training will be conducted during the project. Table 5. Volunteers and Planned Training Personnel/Group to be Trained Description of Training Frequency of Training Project Evaluation Define qualitative and quantitative methods for measuring success of your project, based on the objectives described earlier. Include metrics to evaluate success related to the science and information gathered, the volunteer involvement and experience, partnership objectives. Describe how the information collected was used for land management or other application, and how much your actual costs deferred from your estimated project costs. Reference Chapter 6. Reporting and Sharing Results Describe how you will share your results with the volunteers, community, and anyone locally that will benefit from learning the outcomes of your effort. Consider if you will publish the results in a scientific paper, technical guide, on data.gov, in a Forest Plan or in some other form. Describe your science delivery plan and how you will make the science available to a broad audience. Include how you will report your accomplishments to the Forest Service. Reference Chapter 7. How this Plan will be Updated Describe how and when you will evaluate your strategy based on your objectives and timeline, and how it may need to be adapted over time so identified processes are improved upon when needed. Share how input from partners, stakeholders and at-large public or external factors will be considered. Consider describing what the milestones of public participation are. 11 Project Name Appendix H provides an example of how you might consider tracking the results and outcomes of public participation at various key steps. Documenting how public involvement contributed to the development of a plan is required (FSH 1909.12, sec.21.43). Table 2. Project Plan Updates Version Date Changes Made v.1 v.2 v.3 12 Citizen Science Project Plan Appendix A: Key Messages Use this appendix to describe the key messages that will be part of your communication and outreach to the public during the planning process. 13 Project Name Appendix B: Communication Tools Use this appendix to describe the tools for reaching internal Forest Service stakeholders (staff at the local unit, Regional Office, and Washington DC Office) as well as external individuals and entities (groups, agencies, others). Consider communication tools that are required and tools that will be used when adequate resources are available. Ensure any communication tools are approved for agency use. Some examples: Mailing Lists and Listservs; Meetings / Workshops – Forest Service hosted; Meetings/ Workshops – Group hosted; Press Releases; Public Notices Stakeholder / Cooperating Agency Letters; Webpages; Interested Parties Briefings / NGOs updates; Outreach briefings to Community Contacts; Working Groups; Workshops; Roundtables; Open Houses; VTC / Conference Calls; Webinars; Podcasts; Talking Points; PowerPoint Presentations; Briefing Papers; E-Bulletins; Calendars; Newsletters, People, Places and Things; Blogs and Organizational Newsletters; Brochures; Fact Sheets; approved Social Media tools such as Twitter, Facebook, or YouTube; Speeches; Videos; Brown Bags; Advertisements or Public Service Announcements; Newspaper Editorials; Radio Spots in English and Other Languages; Ranger in the Classroom Programs; Online open houses; listening sessions; Fairs and Festivals; Annual Days (e.g. Get Outdoors Day). Consider developing a list of social media accounts, internal and external blogs and newsletters, popular forums or meetings, and local newspapers or magazines. Describe any resources you will use to reach audiences where English is their second language or to make your documents and media 508 compliant. Include a description of which communities will most likely need these resources translated and any other information about how they will be developed and distributed. Include the contact information for the businesses or organizations you are using as resources. 14 Citizen Science Project Plan Appendix C: Forms and Agreements Include volunteer forms, photo release forms, job hazard analysis, partnership agreements, and any other relevant form for the project. Reference Chapters 3 and 4. 15 Project Name Appendix D: Project Organization Chart An organization chart shows the lines of communication and reporting for the project. Fill in the names of the individuals and their titles. 16 Citizen Science Project Plan Appendix E: References Cited 17