Monitoring Case Study Case Study: Water for All The following case study is intended to serve as a guide to help you think through the monitoring plan and the methods you might wish to use in your project. Instructions: Read the following case study in the two parts indicated and discuss some of the questions listed at the end of each section. Note: All names and places are fictitious. Part 1 Water for All is a not-for-profit humanitarian organization committed to household water treatment projects in several areas of Honduras.. The organization is responding to the needs of the poor by improving their water quality through the use of the biosand filter. Water for All designed their project to provide biosand filters to the surrounding communities along with an education program focused on operation and maintenance of the filters along with basic knowledge of water use, safe water storage, hygiene and sanitation. Water for All has 2 filter technicians, 1 Community Health Coordinator, and 3 Community Health Promoters and receives ongoing technical support from CAWST. To date, Water for All has installed 500 filters and is looking to expand their services to more rural communities. Filters for the project are constructed at a central location and transported to users in surrounding communities after they had attended a workshop and requested a filter filters. No production records of the filter construction or delivery records were kept and no visits to households after installation were carried out. Many months later Water for All began receiving complaints from a number of users whose filters developed problems. Some of these filter problems were due to a manufacturing error, some due to installation errors and some due to users’ incorrect operation and maintenance. When Water for All investigated further they discovered that many communities had similar problems and decided to visit all the filters they had installed to check filters were correctly functioning and to replace those with the manufacturing fault. Unfortunately this exercise proved very difficult since they neither had records of exactly how many filters they had constructed nor where they had delivered them. Discussion What are the main problems faced by Water for All? What you think that Water for All could have done to improve their project? NOTE TO FACILITATOR –A brainstorming session might bring out many of the items which are discussed in the following paragraphs. Part 2 April 2009 1 Monitoring Case Study As a result of this experience, Water for All carried out a thorough review of their project. The reviews involved researching baseline health conditions and water quality, the security situation and talking to communities and their leaders to assess how the project met their needs. After the analysis of this review, and with the support of CAWST, Water for All made major changes to their project which included the addition of a comprehensive monitoring system. One of the main strengths of the programme and its monitoring system is that it engages the communities and allows them to play a significant role in both the implementation and monitoring. Two local citizens are selected in each community to train and act as community health promoters, encouraging their neighbours to improve their hygiene behaviours, correctly maintain their filters and provide support in case of problems. Using prepared checklists, the community health promoters routinely follow-up with the end users and complete monitoring forms to record their findings. The major elements of the monitoring system are: Production record keeping and review Water for All use a numbering system, painted on each individual filter, to allow them to follow it from production to installation and beyond. Each filter is constructed and inspected by the manufacturing technician using a prepared checklist. The checklist includes checks on material quantities and methods and well as checks on the finished filter for cracks and adequate flow of water through the tube. Completion of these checks is recorded in a monitoring form alongside the date of manufacture and the technician who produced the filter. These records are reviewed by the project manager every 3 month to ensure they are complete and accurate. Records are also kept for material stocks. These record deliveries of materials to the workshop and the quantities used to produce each batch of filters and their associated components. These records are used by technicians to order new materials and are reviewed by the project manager every 3 month to ensure they are complete and accurate. A record of all the costs for the materials and labour that was used to build the filters is also kept. The project manager can then calculated the total project costs and the unit cost for each filter. This is used to determine the price that will be charged to the end users as well as when preparing proposal to various donors. Filter Delivery and installation record keeping and review Records are kept of all filters installed in users’ houses. These records include the name of the user, date of installation, the person installing, receipt of financial contribution and a checklist for the correct installation of the filter. These records are reviewed by Water for All Management every month to ensure they are complete and accurate. Production and Installation Spot checks In addition to the regular record keeping the project manager carries out spot checks a minimum of 4 times a year twice a year of both filter production and filter installation. In reality he carries out these checks every few weeks. The spot checks for the filter production include checking the quality and storage of materials, the availability and condition of tools, the filters are being produced using the April 2009 2 Monitoring Case Study correct methodology, the quality of the filters and that record keeping is being done adequately. The spot check for filter installation follows the same format as the checks carried out by the installing technician. Regular follow up visits to users by community health promoters The key element of the Water for All monitoring program is the visits by community health promoters to filter users. After installation, community health promoters visit users’ houses at 1 week, 1 month, 3 months, 6 months, 1 year and 3 years. Using a checklist the community health promoters check that the filters are functioning properly and give the opportunity to users to discuss hygiene issues and any problems they have experienced. The agents record their findings on a prepared form which is collected by project staff after completion. Regular follow up visits to community health promoters by project staff Project staff regularly visit community health promoters to ensure they are active, collect monitoring forms and address any problems/ training needs. These visits normally coincide with the monitoring schedule of the community health promoters but are more frequent where community health promoters need additional support. These visits also generally involve spot checks of 2-3 of the filters in that community to ensure that the agent has actually made the visits and to double check that they have correctly understood how to complete the monitoring forms. The community health promoters are also encouraged to visit the Water for All office whenever they can. Problem Log Every time a problem is found it is entered into a problem log stored at the main office. Once the problem is solved it is also reported in the log. This allows all staff to know the problems that are occurring in the field as well as how and when the problem was solved. It also allows the project manager to ensure problems are being solved and easily identify the scale of problems that are occurring in the field. The project manager reviews the log every month and all staff are encouraged to regularly review the document. The problems and solutions are discussed at the regular meetings of the staff and with the community health promoters. The monitoring plan and timeline for the Water for All project is given overleaf. Discussion What monitoring methods were used by Water for All? How effective do you think these methods are? What other methods could they have used? Which of these methods you would like to adopt for your own project? April 2009 3 Monitoring Case Study Monitoring and Evaluation Plan Performance questions & objectives How many filters have we produced Are the filters of acceptable quality Information needs and indicators Number of filters/ diffusers and lids made 95% conform to quality control checklist Are we using the correct quantities of materials Conformity to expected material usage How many filters have we installed? Number of filters installed Are the filters correctly installed? 100% conform to installation guidelines Are users paying their contribution? Are people correctly using the filters Are there any problems with the filters? 80% able to pay financial contribution 85% conformity to user guidelines Number of reported problems Are the community health promoters carrying out their role? Community health promoters actively carrying out their role April 2009 Data collection methods Technician’s production records Technicians production records, production spot checks & problem log Technicians production records and stock records Technicians installation records Technicians installation records and spot checks Technician installation records Sampling and frequency Production records, every time an item is completed – Reviewed quarterly Technicians – Review by Project Manager Production Spot checks 4 times a year Project Manager Stock records every week – Reviewed Quarterly Technicians – Review by Project Manager Problems logged when found – log reviewed monthly Technicians – Review by Project Manager Installation records, every installation – Reviewed monthly Installation spot checks 4 times a year User monitoring forms User monitoring 1 week, 1 month, 3 months, 6 months, 1 year and 3 years Community health promoter spot checks Spot checks at community health promoter follow up at similar intervals Problem log Responsibility / Action Problems logged when found – log reviewed monthly Technicians – Review by Project Manager Project Manager Community health promoters – Review by community health promoters / project manager Community health promoters Community Health Promoters – Review by Project Manager information Resources required Identify problems, reference in case of problems and material ordering Checklists, monitoring forms, , filing space, data input time, manager’s time Identify problems, check user contribution and reference in case of problems Checklists, monitoring forms, , filing space, data input time, manager’s time Technician training for completion of records Identify problems, reference in case of problems, general project statistics Checklists, monitoring forms, community health promoter per diems, filing space, data input time, manager’s time Community health promoter training to carry out visits and complete forms Use of 4 Training needs Technician training for completion of records Data management Transfer to computer and file hard copy. Summary given in monthly report Transfer to computer and file hard copy. Summary given in monthly report Problems transferred to problem log. File hard copy. Summary given in monthly report Monitoring Case Study Monitoring and Evaluation Timelines Month Installation Checks Jan Review Records Feb Review Records Review Records and Spot Checks Mar Review Records and Spot Checks Apr Review Records May Visits to user Continuous Installation Checks Continuous production Checks Production and Stock Checks Jun Visit 1 Month 1 Visit 2 Visit 2 Month 2 Month 3 Visit 3 Visit 3 Month 4 Month 5 Review Records and Spot Checks Month 6 Visit 4 Jul Visits to community health promoter Installation Visit 1 Review Records Review Records and Spot Checks Month Visit 4 Review Records Month 7 Aug Review Records Review Records and Spot Checks Sept Month 8 Review Records and Spot Checks Month 9 Oct Review Records Month 10 Nov Review Records Review Records and Spot Checks Dec Month 11 Review Records and Spot Checks Month 12 Visit 5 April 2009 Visit 5 5