EVALUATION MANAGER CERTIFICATION PROGRAMME WORKSHOP EVALUATION REPORT Turin, Italy July, 2013 Index I‐INTRODUCTION....................................................................................................................... 3 II‐WORKSHOP OBJECTIVES AND CONTENT ............................................................................... 4 Certification Programme Overall Objective ...................................................................... 4 Workshop Main Objective................................................................................................. 4 III‐PARTICIPANTS’ EVALUATION ................................................................................................ 5 3.1 Multiple Choice Questions .......................................................................................... 5 3.2 Open questions results................................................................................................ 8 3.3. Daily comments on the course content: like, dislike and proposals of change ....... 11 IV‐ CONCLUSION FROM PARTICIPANTS’ EVALUATION ........................................................... 16 V‐ RECOMMENDATIONS FOR A FUTURE EDITION .................................................................. 17 ANNEXES ................................................................................................................................. 18 Annex I – Evaluation main results ................................................................................... 19 Annex II ‐ Summary of participants’ evaluation results ................................................. 20 Annex III ‐ Comparison with ITC‐ILO’s 2012 benchmark ................................................ 22 Annex IV – List of Participants ......................................................................................... 23 Annex IV Timetable ......................................................................................................... 30 Annex V – Evaluation questionnaire ............................................................................... 31 2|End‐of‐training report‐ EMCP WORKSHOP EVALUATION REPORT Evaluation Manager Certification Programme I‐INTRODUCTION In the following report a summary of the comments made by the participants after each day is presented with a summary of the evaluation questionnaires filled‐in by the participants. Every day, participants were asked to identify what they did like, dislike and to propose constructive changes for a future edition of the workshop. In addition to that, a questionnaire was distributed at the end of the training to the participants who attended the “Evaluation Manager Certification Programme (EMCP)” workshop, which was jointly organised by EVAL and the ITC‐ ILO from 1 to 3 July 2013 in Turin. Sixteen (16) officers1 from various ILO field offices and technical departments, who are involved in managing project evaluations, have benefited from the training sessions and have completed the first part of the certification programme. This first part consisted in the transfer of theoretical and practical knowledge in line with the required core competencies to become an ILO evaluation manager. A second phase is foreseen, in which the trained participants will have to manage a real project evaluation under EVAL’s guidance and supervision. After the successful completion of this second phase, participants will be awarded the certification. At the end of the workshop, all participants had the opportunity to express their level of satisfaction in writing. 16 out of the 16 participants have completed the ITC‐ILO standard evaluation questionnaire built around the various aspects forming a training activity: 1 18 participants were initially enrolled, 2 could not make the journey to Turin for personal reasons (Lost of a family member and Medical forced rest) 3|End‐of‐training report‐ EMCP program organization training objectives, contents and clarity trainer’s performance interaction between participants related services and facilities usefulness This complete questionnaire can be found in the annex section. II‐WORKSHOP OBJECTIVES AND CONTENT The workshop was designed by EVAL with the collaboration of the Sustainable Development and Governance Programme of the ITC‐ILO in order to certify a maximum of 25 individuals who have the responsibility to manage evaluations of ILO technical cooperation projects. The workshop objectives and content were the following: Certification Programme Overall Objective To upgrade the quality of evaluation management in the ILO and to expand the pool of qualified candidates that have volunteered in the past or that will volunteer in the future to become Evaluation Managers. Workshop Main Objective To acquire the technical and practical knowledge, tools and techniques for the core competencies of an evaluation manager in order to effectively manage evaluation of ILO technical cooperation projects. 4|End‐of‐training report‐ EMCP III‐PARTICIPANTS’ EVALUATION 3.1 Multiple Choice Questions In the following section the most significant comments made by participants and the results extracted from the 21 multiple choice questions are presented. The evaluation criteria are built on a 1 to 5 scale, 1 standing for the minimum satisfaction and 5 for the maximum. The complete and detailed results are presented in a table in the annex section. The overall score given to the training, based on the question 21 “are you satisfied with the overall quality of the training activity?” is 98% (4.88). 100% of the participants responded with the rank of 4 or 5 indicating that participants are highly‐satisfied with the workshop. In the field of andragogy (adult‐education), satisfaction is usually correlated to usefulness, in that sense through questions 19 and 20, it is deemed that participants found the workshop useful and that they are planning to apply the leanings for the benefits of their employer, the ILO. (see table below) Usefulness of the training and global satisfaction Questions and answers Are you satisfied with the overall quality of the activity? How likely is it that you will apply some of what you have learned? How like is it that your institutions/employer will benefit from your participation in this activity? Average % 4.88 98% % of answers between 4 ‐ 5 100% 5.00 100% 100% 4.94 99% 100% Training objectives and content: questions 2 to 3. The programme objectives, content and clarity have been evaluated positively; the overall average for the two questions covering these aspects is 93 % (4.66). The course objectives and content were presented through the preparatory readings sent prior to the workshop and repeated on the first day through a visual of the core‐competencies for an evaluation manager in line with the presentation of the workshop timetable. During the workshop before each session, the facilitators have constantly referred to that visual in order to link content with objectives. Information received before the activity Question 1 asked if before participating in the workshop, participants were given clear information to understand if the course could meet their learning needs. The score given to this question is 84% (4.19). The whole EMCP has been firstly advertised through a flyer communicated via email in two different waves within the ILO and secondly the course content as well as the logistic arrangements have been communicated through emails to the enrolled participants prior to the workshop in Turin. 5|End‐of‐training report‐ EMCP Information received before the activity Questions and answers Average % 4.19 84% Before participating in the activity, were you clear about its objectives, contents and methods? % of answers between 4 ‐ 5 81% Learning methods The participants are satisfied with the learning methods used to conduct the training. The feedback given in the written component of the feedback as well the score of 90 % (4.50) illustrate well the participants’ satisfaction on the approach and learning methods used to deliver the workshop content. The mix of presentations, discussions and interactive group works based on case studies reproducing ILO realities were the main learnings methods used. The EVAL’s facilitator had the leadership with the presentations and Turin’s facilitator's effort was focused on supervising the interactive group works. The way the activity was delivered Questions and answers Average % 4.69 94% % of answers between 4 ‐ 5 100% 4.63 93% 100% 4.50 90% 94% To what extent were the activity's objectives achieved? Given the activity's objectives, how appropriate were the activity's contents? Were the learning methods used generally appropriate? Gender Issues The participants were asked to assess if the course adequately integrated gender issues into the training with the question “Have gender issues been adequately integrated in the training?” the median score given is 82% (4.13). Gender issues were addressed in the training as a cross‐ cutting theme however the training was not designed to address gender issues as a primary component. A deeper analysis of results shows that only 75% gave a 4‐5 to that question and that men have been generally more severe on the gender question. Compared to ITC‐ILO’s benchmark (3.93) the score is however good. The fact that the group of participants was equally gender‐balanced may have contributed to that good result. Number of participants Number of women Ratio of Women 16 8 50% Facilitators’ performance The facilitators’ performance score are excellent, all score given are above 90% for both, Craig Russon and Guillaume Mercier. This indicates that participants were highly satisfied with 6|End‐of‐training report‐ EMCP their performance. The comments made through the open questions also demonstrate that high level of satisfaction (see comments under the section 3.2 on this report) Craig Russon (EVAL) Questions and answers Average % % of answers between 4 ‐ 5 4.75 4.88 4.88 95% 98% 98% 100% 100% 100% Average % % of answers between 4 ‐ 5 4.81 4.88 4.63 96% 98% 93% 100% 100% 94% Overall Contribution Technical Skills Teaching Skills Guillaume Mercier (ITC‐ILO) Questions and answers Overall Contribution Technical Skills Teaching Skills Workshop environment: the group of participants, the media/material and logistical support. The participants were asked to evaluate the contribution of the other participants and the interaction level among the group; in response to the question “Did the group of participants with whom you attended the activity contribute to your learning?” the average response is 98% (4.88). This shows that group's dynamic has played a major role in contributing to the success of this workshop. Regarding the materials used, the participants have judged them as excellent, with response to the question with a 90 % (4.50). Hard copies of the ILO evaluation manager's handbook, the ILO policy guidelines of RBM evaluation and a manual with all slides and cases studies were distributed to participants. All those above listed references as well as many others were put together on a USB key and given to the participants in their soft version. Participants rated the activity at 93 % (4.63) for the logistical organisation. Many participants have however expressed dissatisfaction about the tea breaks and the food offered at the cafeteria. Participants gave as well a high score of 96% (4.81) for the secretariat and the work done by Carmelisa Magli from the ITC‐ILO who has dealt with most of the administrative issues in collaboration with Estève Dal Gobbo from HRD. 7|End‐of‐training report‐ EMCP 3.2 Open questions results In the next section, the main comments and feedbacks given in writing by participants through the evaluation questionnaire and the daily feedbacks are presented and regrouped into five clusters of comments: 1. The aspects of the workshop that were the least useful or that will need to be improved 2. The aspects of the workshop that were the most useful 3. Suggestions for the resource persons to improve the overall quality of their contributions 4. Other participants’ particular comments and suggestions 5. Daily feedbacks on the workshop content 1. The aspects of the workshop that were the least useful or that will need to be improved Examples used in the exercises, most were negative or examples to be criticised. I think positive examples would also have a value to add General presentation a bit long A few small adjustments on exercises and case studies could be improved Session on ethics Reduce number of words presentations The combinations of presentations and exercises Case studies selected Fish bowl can be re‐designed to involve other participants too The module of soft skills: touched on too many subjects and necessarily superficially Many papers. It’s better to prioritize electronic options Short time for exercises Some examples of the exercises were not good Time to read the exercise was not enough The opening speeches Some of case studies 2. The aspects of the workshop that were the most useful • All except first day presentations that could be shortened to give space to new stuff to be learnt • All aspects especially the checklist and exercises • Resource persons were excellent • Training material well designed and appropriate • The participants were also engaged and provided useful insights • Concise and objective • Practical • Dynamic • Diversity of the group • Plenary exercises • Balance between lectures and exercises • Good basis of documentation (handbook, policy, etc) • Content/training manual/handbook • Facilitation • Practice • Group work • Duration (3 days) • Good selection of participants • Presentations 8|End‐of‐training report‐ EMCP • • • • • • • • • • • • • • • • • Handbook and other documents The practical examples Content of the workshop Dynamic presentations by the facilitators Networking Only ILO participants with knowledge of the house (no other organisations) Gender balance of the group The combination of theory with exercises The training manual was very useful The practical exercises The enthusiasm of the facilitators Group works Most of the examples Structure of the course Content Material Facilitation 3. Suggestions for the resource persons to improve the overall quality of their contributions • Only the exercises could be improved by choosing better examples/text/documents • Change some exercise (exercises to include good practices) • Adjust time needed for certain sessions, i.e. more time for roles, influential evaluations, briefing, inception report, reporting (less time for dissemination) • Honestly both were excellent and most beyond the call of duty (most appreciated) • Craig is an amazing trainer • Both were very good! And complement each other • More time to exercises • Using some techniques to form the groups • Would be interesting to have a five‐day workshop (or a phase 2) and have a comprehensive simulation of an evaluation/maybe visiting a project site • Allow the ITCILO’s facilitator to contribute more • Don’t change...excellent work from both! • Choose better case studies • Include in other training courses recommendations and suggestions from participants 4. The participants have also made some other particular comments and suggestions, listed here below: • More time for exercises – it was a bit rushed • Engage resource persons from the field • All good! • I would have expected more contents on methodological aspects of evaluations: Types of qualitative and quantitative research techniques that are out there. Perhaps something on which types of techniques can apply to specific types of projects. Indeed to judge on the value of an evaluation manager, some more basis on research techniques would be useful • The course content is very good but should be delivered in 5 days considering it as a certification course • Better choice of case study (update them) • Deliver full documents of an evaluation exercise prior the training for reading • Four days would be better to cover the material 9|End‐of‐training report‐ EMCP • • • • Some of the exercises should be changed Extend the duration of the course to provide more time to learning Better define target groups (programme officers and technical officials) It would be great to include recently case studies and good examples ! 10 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P 3.3. Daily comments on the course content: like, dislike and proposals of change DAY 1 LIKE Slides and handouts Time for questions and discussion Well organized Detailed information provided on several issues Craig! Open and communicative Overview Interactive, Informative and Timely DISLIKE CHANGE Many papers First two exercises: define them better 3 days may be not enough to introduce such a large theme Too many frameworks that confuses us Morning session a little easy… General concepts are known, need to get into specifics practical challenges More time to roles!! Less time to general stuff Modules 1‐3 can be done in a morning session Devote more time to roles and responsibilities Revise exercise 3 as it is confusing and unclear Nothing to change!! More practical examples No change needed Not enough time for exercises All ok Improve explanations for exercises Give more time for exercises Keep up with timetable and finish ppt Put material in thumb drive Openness and availability to answer our questions Participatory approach and engagement of all participants It is providing a good introduction to the work and responsibilities of an evaluation manager The exercises at the end of each module are very useful Dynamic presentation and good resources Great moderator / trainer Open discussions Small group size Last exercise Last exercise was good Plenty of information, Dynamic straight‐forward information and Usefulness I would like more examples in explanation of complex topics The facilitator was great!! Book of material I really like the enthusiasm of the facilitator. Great job! Clear presentation materials, Clear presentation, Quality Discussion Useful activities Exercise of module 4, Atmosphere, Presentations (structure, timing, pace) The process of EVAL: RBM, HRBA, Gender: Practical knowledge through group work Facilitation, Flexibility Practical exercise, Group work I really liked the simple approach to a complex topic Presence of EVAL’s Director 12 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P Not enough time spent on roles Too much time spent on RBM principles that should be common knowledge Too many words presentations Nothing! More exercises Nothing to change Less printing The instructor team has no woman and/or field evaluator Include more information on evaluation policies and documents such as Evaluation policy Evaluation strategy, etc. Include a warm‐up exercise / introduction Put all documents in thumb drive I won’t change anything Nothing! DAY 2 LIKE DISLIKE Exercises The exercises are great The written material is very useful Would be important to have more time for the groups exercises Exercises going beyond the time Wordy presentations CHANGE Provide a sample of inception report Bring in an external collaborator (evaluator) to provide insights Exercises need to leave more time devoted …shorter presentations? Or rather longer session! The practical exercise on TORS can be improved. Not sufficient time to fully take all in The discussion on the time frame (module 7) too long. It is evident that it will depend on specific situations (case by case) Service at coffee breaks Give more time to exercises and de‐briefing More time for exercises Fishbowl: making it more participative so that all participants can ask questions and join Excellent facilitation Useful information provided Great exercise the fish bowl .. really enjoy it Role play Good mix of lectures and exercising The content of today’s subject and exercises should be delivered in 2 days Again, thanks for the enthusiasm! Great job! Resource management working on plans, budgets Review the time used for the ppt and the exercises more realistically. You may need more than 3 days. It is important that participants understand the instructions and the purposes of the exercises Better selection of case studies Presentations Exercises : different methodologies Comprehensive material The workshop has been dynamic and the time is passing fast I like the exercises I really liked the stakeholder engagement exercise Time for discussions, questions, experience sharing 13 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P Coffee breaks: bad service. Time management We went late for the coffee breaks and there was no food anymore Give more time for exercises Provide examples of good practices (TORs, etc..) I don’t like some answers because they don’t consider field reality Concert better the examples during the exercises DAY 3 LIKE DISLIKE CHANGE The workshop as a whole You could had used better examples Plenary/collective exercise to reach Conclusions : examples on the ethics guidelines Little time for the exercises Nothing to say but a very successful programme Liked the three days Facilitation : TOR Improved time management Birthday celebration of Guillaume Friendly facilitators Good environment and group work Good “match” between resource persons Good content Excellent facilitation I really liked the whole course! It was very well constructed and fantastically delivered. Congratulations and thank you for your dedication and professionalism (Andrés) Nothing Wordy slides Would be great to have a one‐stop‐shop at EVAL’s website with all guidance documents and templates That it is over Session on ethics: not clear to me the purpose of this module Sometimes the discussions went into unrelated topics Some of the examples were not so good but group works were excellent Content very good but need two days for today’s content and more exercises Time distribution into 5 days (realistic assignment of time for presentations, participation, exercises Improve exercises tools with comments from participants, provide info in pendrive at the end to add relevant additional info Maybe make the programme a bit longer in order to better assimilate the content Dynamic presentations, documentations and exercises Time management Number of participants It was a very useful training course! Great work Excellent group! Excellent teachers! Guillaume’s Birthday Time Management 14 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P Exercises should include positive examples Real report should/could b used if text was “frozen up” by groups Maybe consider an extra day to give time for more in depth exercises. I felt that particularly we did not have sufficient time for the TORs exercise More collective/Plenary exercises Send case stud before course starts or the day before as “homework” Distribute good examples Keep discussion more focused Change examples for exercise (skip the Tajikistan example) Ethics module Explain purpose of each module 15 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P The examples – TOR, Report , Inception reports should not be very good examples. An example that has both good/good aspects. Use examples of evaluations managed by some colleagues who are taking part in the workshop Better ways of forming groups that will facilitate one person to work in groups with different people Time: perhaps 4 days would have been better Case study example IV‐ CONCLUSION FROM PARTICIPANTS’ EVALUATION In general the workshop has been highly evaluated. As presented in the annex section, the results of the evaluation show that this workshop received higher score on all assessed criteria compared to the ITC‐ILO’s benchmark levels. It appears that one of the critical success factors of the workshop has been the group of participants itself. The selection process and the establishment of eligibility criteria by EVAL for participants to qualify for the certification programme have resulted in the selection of a group composed of 16 motivated, interested and high‐level ILO’s officials. In addition to that, as many mentioned, the fact that the workshop was delivered entirely by ILO staff and for ILO staff that it was an “in‐house” workshop has contributed to create an open and positive learning environment. This has resulted in a workshop evaluated as useful for participants individually as well as for their employers, the ILO. The use of mixed learning approaches has been also mentioned as an important success factor. Both facilitators have experience in the two main technical fields of the workshop; the field of evaluation and the field of teaching. The combination and complementarity have been underlined as a favourable condition in the delivery of the workshop. However, it was noted that the facilitators should try to manage the time better. Some group works could not have been realised fully because of the time constraint. Time management has been somehow, a major issue in the delivery of the workshop. The structure and content of the workshop organized around the core‐competencies for an evaluation manager seems in general an excellent approach. The need for small improvements has already been identified by both the facilitators and participants which are going to be addressed in the next edition of the course in particular at the level of some case studies. It was also suggested to adopt a carry‐through case study of an evaluation to cover the modules for the area of core‐competencies on technical and professional skills. The pre‐training phase could also be improved. Information sent prior to the commencement of the face‐to‐face training course could be enhanced and shared with participants at an earlier stage from the workshop, enabling participants to better prepare themselves. V‐ RECOMMENDATIONS FOR A FUTURE EDITION Based on this first edition and with regards with the workshop, it is recommended: To improve the group work by preparing a carry‐though case study (based on the WEDGE Project) that can be used to cover the areas of core‐competencies on technical and professional skills. To modify the fish bowl exercise, the one used around the UNEG ethical code and to provide a ‘younger” and better evaluation report as a sample for analysis. To incorporate success stories about ILO Evaluations management into the workshop as examples of good practices and as concrete illustrations to support the presentation of the theoretical aspects To add a pre‐training survey and to send the training material at least one week before the trainings for participants to arrive better prepared To continue using a strict selection process of participants, this is critical in keeping high, the certification programme credibility. In that sense the group size should be maintained at 20 maximum and in order to enrol a minimum position’s grade and experience should be officially established. The workshop has been also an opportunity to look at EVAL’s policies and procedures from an “outsider’s” perspective. In that sense, few proposals of change or improvements were made during the workshop. It is therefore recommended: To adopt a responsibility matrix (or a workflow diagram) in order to clarify who does what in the evaluation process. From the appointment of the evaluation manager to the dissemination of the evaluation. If possible, to clarify how evaluation managers are appointed. This process appears highly random and political, in other words it seems to be a grey area. For EVAL, to have a reflection about extending the responsibility of the evaluation manager to the dissemination phase of the process For EVAL, to reflect about recruiting consultant (evaluator) not only based on their qualifications (CVs) but also incorporating other selection criteria that could be presented in a complete technical proposal (equivalent to the inception report) for important evaluation assignments contracted to firm. (Qualification of the firm, understanding of the context and TORs, quality and relevance of the proposed methodology, qualification and experience of the key staff, work plan, etc.) To coordinate with PARDEV in order to evaluate the possibility to merge the annual project progress report with the annual evaluation report. 17 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P ANNEXES 18 | E n d ‐ o f ‐ t r a i n i n g r e p o r t ‐ E M C P Annex I – Evaluation main results Annex II ‐ Summary of participants’ evaluation results A906415 ‐ Evaluation manager certification programme Language: EN Date: 01/07/2013 to 03/07/2013 Venue: Turin Centre Number of Participants: 16 Returned Questionnaires (number and percentage): 17 ; 106.2% Questions Before participating in this activity, did you have enough information to understand whether it could meet your learning needs? To what extent were the activity's objectives achieved? Given the activity's objectives, how appropriate were the activity's contents? Was the balance between theory and practice appropriate? Have gender issues been adequately integrated in the training? Were the learning methods used generally appropriate? How would you judge the resource persons' overall contribution? Craig Russon ‐ RP ‐ Overall quality RP ‐ Technical skills RP ‐ Teaching skills Guillaume Mercier ‐ RP ‐ Overall quality RP ‐ Technical skills RP ‐ Teaching skills Did the group of participants with whom you attended the activity contribute to your learning? Was the atmosphere within the group pleasant? Were the materials used during the activity appropriate? Would you say that the logistics of the activity were well organized? Would you say that the administrative support/secretariat was efficient? How likely is it that you will apply some of what you have learned? How likely is it that your institution/employer will benefit from your participation in the activity? Are you satisfied with the overall quality of the activity? Average aver. %4&5 4.19 4.69 4.63 4.19 4.13 4.50 4.81 4.75 4.88 4.88 4.81 4.88 4.63 4.88 4.69 4.50 4.63 4.81 5.00 4.94 4.88 4.63 81% 100% 100% 88% 75% 94% 100% 100% 100% 100% 100% 100% 94% 100% 94% 94% 94% 100% 100% 100% 100% 95% 2012 ref 3.68 4.18 4.27 3.93 4.29 4.43 4.32 4.31 4.38 4.47 4.46 4.46 4.43 4.28 Demographic information Sex Female Male freq 8 8 percent. 50% 50% Annex III ‐ Comparison with ITC‐ILO’s 2012 benchmark Evaluation manager certification programme A906415 01/07/2013 – 03/07/2013 Annex IV – List of Participants SWITZERLAND** Full Name Mr. Sergio Andrés IRIARTE QUEZADA 1 : Institution: Position: Address: Tel. / Mobile: ILO HQ Geneva Knowledge Management Officer Geneva ; +4122 799 7849 Fax: Web: E-mail: iriarte@ilo.org Full Name Mr. Natanael PEREIRA LOPES NETO BRAZIL : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: ILO CO-Brasilia Senior Programme Assistant Brasilia ; BRAZIL + 5561 2106 4611 + 5561 3322 4352 lopesn@oitbrasil.org.br 2 EGYPT Full Name Ms. Nashwa BELAL 3 : Institution: Position: Address: Tel. / Mobile: ILO DWT/CO-Cairo Senior Programme Officer Cairo; EGYPT +202 2 7350123 ext 030 Fax: Web: E-mail: +202 2 7360889 Full Name Mr. Luca FEDI belal@ilo.org, belal.nashwa@gmail.com **duty station EGYPT 4 : Institution: Position: Address: Tel. / Mobile: ILO DWT/CO- Cairo Senior Programme and Operations Officer Cairo ; EGYPT +20 227 350 123 ext 000 Fax: Web: E-mail: fedi@ilo.org Full Name Ms. Makda Getachew ABEBE ETHIOPIA : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: ILO RO Addis Ababa Programme Officer, Regional Programming Unit Addis Ababa ; ETHIOPIA +251 11 544 4063 +251 11 544 5573 getachew@ilo.org 5 HUNGARY Full Name Ms. Maria BORSOS 6 : Institution: Position: Address: Tel. / Mobile: ILO DWT/CO-Budapest Programme Officer Budapest; HUNGARY +36 1 30149 07 Fax: Web: E-mail: +36 1 353 3683 www.ilo.org/budapest borsos@ilo.org Full Name Mr. Thomas KRING INDIA 7 : Institution: Position: Address: Tel. / Mobile: ILO DWT/CO-New Delhi CTA, Manager of the subregional project ”Way out of Informality\” New Delhi ; INDIA +91 8527896394 Fax: Web: E-mail: +91 1124602111 kring@ilo.org Full Name Ms. Eszter SZABO HUNGARY : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: ILO DWT/CO-Budapest Senior Programme Assistant Budapest ; HUNGARY +36 1 3014915 +36 1 353 3683 www.ilo.org/budapest szabo@ilo.org 8 CHILE Full Name Mr. Andrés YUREN 9 : Institution: Position: Address: Tel. / Mobile: ILO DWT/CO-Santiago Senior Specialist on Employers Activities Santiago ; CHILE +562 2580 5500 Fax: Web: E-mail: +562 2580 5580 yuren@oitchile.cl Full Name Ms. Cybele BURGA SOLIS PERU 10 : Institution: Position: Address: Tel. / Mobile: ILO RO Lima Regional Monitoring and Evaluation Officer PO Box: 14-124; Lima; Peru - PERU +511 6150373 Fax: Web: E-mail: +511 6150400 burga@oit.org.pe Full Name Ms. Rose KARIKARI ANANG SOUTH AFRICA : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: ILO DWT/CO-Pretoria Senior Specialist, Employers Activities Pretoria ; SOUTH AFRICA +271 28188045 +271 28188087 anang@ilo.org 11 SWITZERLAND Full Name Ms. Valentina FORASTIERI 12 : Institution: Position: Address: Tel. / Mobile: ILO HQ Geneva Senior Specialist on Health Promotion and Training/Coordinator Health Promotion and Well-Being Training Cluster Geneva ; SWITZERLAND +4122 7997167 Fax: Web: E-mail: www.ilo.org/safework forastieri@ilo.org Full Name Ms. Maria Beatriz MELLO DA CUNHA SWITZERLAND 13 : Institution: Position: Address: Tel. / Mobile: ILO HQ Geneva Sectoral and Programme Issues Specialist Geneva ; SWITZERLAND +4122 799 6562 Fax: Web: E-mail: cunha@ilo.org Full Name Mr. Oktavianto Akhirsyah PASARIBU SWITZERLAND : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: ILO HQ Geneva Programme Analyst Geneva ; SWITZERLAND +4122 799 7989 +4122 799 8515 pasaribu@ilo.org 14 THAILAND Full Name Mr. Matthieu COGNAC 15 : Institution: Position: Address: Tel. / Mobile: ILO RO-Bangkok Youth Employment Specialist Bangkok ; THAILAND +66 22881797 Fax: Web: E-mail: +66 cognac@ilo.org Full Name Mr. Richard HOWARD THAILAND : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: ILO DWT/CO-Bangkok HIV and AIDS Specialist, Asia and the Pacific Bangkok ; THAILAND +662 288 1765 +662 288 3043 howardr@ilo.org 16 SWITZERLAND ** Full Name Mr. Craig RUSSON Resource Persons : Institution: Position: Address: Tel. / Mobile: ILO HQ Geneva Senior Evaluation Officer, EVAL Geneva, Switzerland +41 22 799 7310 Fax: Web: E-mail: www.ilo.org Full Name Mr. Guillaume MERCIER russon@ilo.org ITALY : Institution: Position: Address: Tel. / Mobile: International Training Centre of the ILO Senior Programme Officer Viale Maestri del Lavoro, 10, 10127 Turin, Italy +39 011 693 6405 Fax: Web: E-mail: +39011 693 6351 www.itcilo.org/sdg G.Mercier@itcilo.org Full Name Ms. Carmelisa MAGLI ITALY : Institution: Position: Address: Tel. / Mobile: Fax: Web: E-mail: International Training Centre of the ILO Programme Secretary Viale Maestri del Lavoro, 10, 10127, Turin, Italy +39011 693 6532 +39011 693 6351 www.itcilo.org/sdg C.Magli@itcilo.org ** duty station Annex IV Timetable Annex V – Evaluation questionnaire Evaluation manager certification programme A906415 Jul 01, 2013 Jul 03, 2013 Turin Centre EVALUATION FORM Please complete the questionnaire below. This will help us to improve our activities. Please be totally frank, for we are interested in your opinion, whether it is positive or negative, and we shall take it into account in planning future activities. This questionnaire is anonymous. Please give only one answer to each question. Sex Male Female Type of organization Please indicate only one response. If you are involved in more than one type of organization, please select the one you are representing during this activity. Trade Union organization Private enterprise Ministries of Labour Training/academic institution Employer organization International Labour Organization Government/public institution UN organization Intergovernmental organization Currently Unemployed Non governmental organization Other 1 Questions on the training activity Please give each aspect set out below a mark from 1 to 5, with 1 being the minimum and 5 the maximum. On this scale, the average is 3. If you think that a question does not apply to you, or that you do not have the information needed to answer it, please leave it blank. 1 Before participating in this activity, did you have enough information to understand whether it could meet your learning needs? 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 you say that the administrative support/secretariat was efficient? 10 Would 1=not at all; 2=not really; 3=to some extent; 4=yes; 5=absolutely 1 2 3 4 5 likely is it that you will apply some of what you have learned? 11 How 1=very unlikely; 2=unlikely; 3=not sure; 4=likely; 5=certain 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1=not at all; 2=not much; 3=to some extent; 4=yes; 5=yes, in detail 2 To what extent were the activity's objectives achieved? 3 Given the activity's objectives, how appropriate were the activity's contents? 4 Have gender issues been adequately integrated in the training? 5 Were the learning methods used generally appropriate? 6 How would you judge the resource persons' overall contribution? 7 Did the group of participants with whom you attended the activity contribute to your learning? 1=not at all; 2=not really; 3=partly; 4=mostly; 5=fully 1=totally inappropriate; 2=not appropriate; 3=partially appropriate; 4=appropriate; 5=very appropriate 1=not at all; 2=not really; 3=partly; 4=mostly; 5=fully 1=totally inappropriate; 2=not appropriate; 3=partially appropriate; 4=appropriate; 5=very appropriate 1=truly disappointing; 2=rather unsatisfactory; 3=satisfactory; 4=good; 5=excellent 1=not at all; 2=not really; 3=to some extent; 4=yes; 5=yes, very much so 8 Were the materials used during the activity appropriate? 9 Would you say that the logistics of the activity were well organized? 1=not at all; 2=not really; 3=to some extent; 4=yes; 5=yes, very much so 1=not at all; 2=not really; 3=to some extent; 4=yes; 5=absolutely How likely is it that your institution/employer will benefit from your participation in 12 the activity? 1=very unlikely; 2=unlikely; 3=not sure; 4=likely; 5=certain you satisfied with the overall quality of the activity? 13 Are 1=not at all; 2=not really; 3=to some extent; 4=yes; 5=absolutely 14 How would you suggest the resource persons improve the overall quality of their contribution? (You may name someone in particular if you wish.) 15 Do you have any observations or suggestions? 2 Questions on the services provided by the Centre Please indicate your level of satisfaction with the services provided by the Centre by giving each aspect a score from 1 (very poor) to 5 (excellent). For services you did not use, please leave it blank. 16 Reception - Courtesy and friendliness 1 2 3 4 5 17 Reception - Problem solving attitude 1 2 3 4 5 18 Reception - Assistance during check-in and check-out 1 2 3 4 5 20 Accommodation and Room - The comfort in your room 1 2 3 4 5 21 Accommodation and Room - The cleaning of your room 1 2 3 4 5 22 Accommodation and Room - The standard equipment in room and bathroom 1 2 3 4 5 23 Accommodation and Room - Laundry/ironing service 1 2 3 4 5 25 Catering - Coffee lounge/canteen staff 1 2 3 4 5 26 Catering - The food for breakfast 1 2 3 4 5 27 Catering - The food for lunch 1 2 3 4 5 28 Catering - The food for supper 1 2 3 4 5 29 Catering - The food at weekends 1 2 3 4 5 30 Catering - The variety of food 1 2 3 4 5 31 Catering - Food and drinks provided for class coffee breaks 1 2 3 4 5 32 Catering - The coffee lounge 1 2 3 4 5 1 2 3 4 5 19 Reception - Comments on this service: 24 Accommodation and Room - Comments on this service: 33 Catering - Comments on this service: 34 Training Room - The comfort int he training room 3 35 Training Room - The standard equipment in training and conference rooms Training Room - The cleanliness and appearance of training and conference rooms Other residential services - Organized leisure-time social activities (variety 37 and number) 36 38 Other residential services - The medical service Other residential services - The Centre's travel unit (official travel arrangements) Other residential services - The "Jet Viaggi 3000" travel agency (private 40 travel arrangements) 41 Other residential services - Comments on these services: 39 4 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5