Human Resources Division: TOOLKIT FOR RU LINE MANAGERS/SUPERVISORS HOW TO DEVELOP A WORK-SAMPLE ASSESSMENT AS PART OF THE RU RECRUITMENT & SELECTION PROCESS FOR SUPPORT STAFF POSTS Copyright: HR Division, October 2012 1 What is a work sample assessment? A work-sample assessment is an internationally recognised assessment tool used in the employee selection process. It measures the competencies of a specific job. Competencies = knowledge, skills and personal attributes needed for the job. These are listed in the section on job requirements in the job profile. The assessment is based on typical tasks of the job. The job applicant does the assessment to demonstrate whether or not s/he has the relevant competencies for the job e.g. a secretary who is required to do typing in the job does a typing test (work sample assessment) to assess his/her typing skills (competency). Why use a work sample assessment? This method has very high levels of validity i.e. it is very good at measuring the competency for a job accurately and is a good way to predict if someone can do (or not) that element of the job e.g. the cooking test for a cook is a good way of assessing his/her ability to follow a recipe, measure our ingredients accurately, use basic conversions, apply cooking techniques etc. This method is also very reliable i.e. it measures the competency consistency e.g. the painting assessment measures painting skills time after time. In contrast, interviews typically have low validity and reliability. When does one use a work sample assessment? One should always use the best selection tool (e.g. interview, work sample, presentation, and referee reports) to measure the required competencies in a valid and a reliable way. Some competencies just cannot be assessed through an interview e.g. written communication skills, typing skills, attention to detail. A work sample is a more accurate way of testing these competencies. Work samples are an excellent way of testing the technical skills in a job e.g. cooking skills for a cook, painting skills for a painter, design skills for the web designer, organisational skills for a manager. The work sample can be used together with other methods of assessment i.e. the results of the work sample are combined with the results from the interview to achieve an overall picture of the competencies of the applicant; OR; The work sample can also be used as a “hurdle” in a selection process i.e. only those applicants who meet a particular level of competency in the work sample, proceed onto the next stage of the selection process. 2 OK, so I want to use a work sample assessment, now what? For each recruitment and selection process for a support staff post: A job profile with job requirements exists. These job requirements include the education and experience requirements as well as the competencies for the post. Before a strategy meeting of the Selection Committee, your HR Generalist will meet with you and explore your views on the recruitment and selection process. If the use of a work sample is a good idea, your HR generalist should be suggesting this to you. The proposal of the recruitment and selection strategy is then tabled with the Selection Committee at its first meeting and agreed on. Where the Committee agrees that a work sample assessment will be used, it will fall to the manager/supervisor, together with HR, to develop this tool and to conduct the assessment of applicants. Specifically, it is the manager’s responsibility to design the work sample assessment and then to assess candidates and to feed back the results to the Selection Committee. It is the HR generalist’s responsibility to ensure that the assessment meets the necessary quality requirements and is a fair and effective assessment and to where necessary, assist with the administration of the assessment. Are there work sample assessments that I can use? For jobs that are common across the University e.g. administrator roles, the HR Division will have work sample assessments available. These are developed with industrial psychologists. There may also be some work sample tests for particular aspects of a job e.g. for the IT components of a job or the finance components. These work samples are being or have been developed with the IT and Finance Division. The HR Division will also keep copies of assessment used to date to develop a library of possible assessments. 3 How do I make sure the work sample assessment is fair? Make sure that the exercises in the assessment are typical of the job and they are substantive elements of the job. Typical tasks of the job can be used e.g. if the manager needs to be able to discipline a staff member, you would write this exercise using a typical scenario and ask the applicant to tell you how they would go about dealing with this situation. If you want to test the cleaner’s knowledge of hygiene requirements, ask them to show you how they would go about cleaning a room. Be clear on what competencies are being measured and that these are competencies required for the job e.g. if you want to assess someone’s numerical ability, know what kind of numeracy is required for the job and make sure the assessment tests these particular numerical skills. Make sure that the exercise is pitched at the right level e.g. don’t require someone applying for a supervisor job to solve problems that managers typically resolve. If you have external candidates, make sure that the exercise does not require knowledge of Rhodes procedures and policies. This creates an unfair advantage for internal candidates. Also be careful that your assessment does not benefit those staff already working this area. You need to test the generic knowledge and skills for the job, not the knowledge of the processes and procedures within a particular department. Make sure your instructions are clear. If some exercises are more important than others and you want candidates to finish the important exercises, you must tell the candidates to do those exercises first. In the absence of a clear instruction of what to do first, second etc., you can’t penalise an applicant who does not do the exercises in the order required. Make sure that the exercises (questions, scenarios) are clearly and concisely worded. Where possible, get someone to do the actual assessment so that you can test that your questions or instructions are not ambiguous. Watch out for “double jeopardy” in your assessments e.g. you ask an applicant to do an Excel spreadsheet to test their numeracy skills and the applicant does badly. Is this because of their Excel spreadsheet skills or their numeracy skills? You will need to have another exercise that is not Excel dependant to test their numeracy skills. Make sure that the average person can finish the assessment or finish a reasonable number of items in the test. In some assessments, too much work is deliberately given so that the person identifies the priority areas and works under pressure. But if you are not measuring prioritisation skills or time management skills, then rather make sure that the assessment can be completed by the average person. Remember that candidates will be nervous and allow for this in the time given. Remember that no-one is perfect. You are looking for the candidate that best meets the competencies for the job and therefore the needs of your department. 4 How do I make sure the work sample assessment is effective? Decide on whether to use the work sample assessment as a hurdle or together with other assessment tools. The way to decide this is to ask yourself: would I hire the person if they can’t do x? (x being the skills you are measuring) e.g. would I hire a secretary who is not computer literate? If the answer is “no”, then why bother to interview them? You are wasting the candidate’s time and the Selection Committee’s time. In this case, the work sample assessment is used as a “hurdle”. Make sure that you are testing the key competencies of the job particularly if you use a “hurdle” approach. e.g. don’t test someone’s excel skills if they are not going to use spread-sheets on the job as they may be eliminated from the selection process on the basis of this. Make sure that the assessment helps to separate out those who cannot do the job from those that can. Test the critical competencies in multiple ways e.g. if the job requires a person to be numerate, give more than one exercise to test numeracy. The reason for this is that sometimes candidates struggle with a particular question but manage to demonstrate their competency in another question. Be clear on what you want to achieve in the measurement of the competency. o Do you want to identify the top candidates? o Do you want to identify if applicants can do the job? o Do you need to know who is the best candidate, the second best etc.? If you want to be able to differentiate between candidates rather than just know that candidates are competent, your assessment needs to be designed in a way that gives you this information. 5 Development: What exactly do I do? a) Identify the competencies that are best measured by the work sample assessment. Remember that you don’t need to test everything with the work sample as there will be other methods of selection. A good rule of thumb is to focus on the technical competencies of the job, the competencies that are difficult to assess in an interview e.g. computer skills, written communication skills, design skills, financial skills, numeracy, accuracy etc. Collect source materials/scenarios to be used as job related case studies that will help you measure the competencies identified. Gather information through communication with departmental staff (position aligned co-worker, line manager, HoD). Gather dummy letters, emails, memos, standard operating procedures (SoPs) for departmental processes, etc. Develop your questions for the assessment. Remember that some questions may be asked verbally of candidates (e.g. I want you to paint that wall making sure that you follow all the right methods of doing so) while other assessments will be paper-based (i.e. the person writes their responses). Develop the model solutions. Remember to be flexible to different ways of doing tasks. Different methods may still demonstrate the necessary competence. Also be careful of requiring staff to do it the “Rhodes way”. Develop the assessment grid. Below is an example. a.Competencies b) b. Evidence of competency c) c. Questions/ d. Rating e. Weighting f. Comments exercises used (Knowledge, Skills and Attributes) Computer literacy: able to work with a word processor Excellent Able to draft correspondence Able to create tables Able to open other files and use these and save these appropriately Able to create own folder Questions 1, 3 and 5 of assessment a) These are the competencies that you are assessing. These are the competencies in the job profile. b) This is the evidence that you are looking for in the exercises. c) This shows which questions are measuring what competencies. Remember that multiple questions can test the same competencies. d) Decide on a rating scale. See below. e) Decide on the weighting of competencies e.g. H, M, L or a percentage of the job f) Write any comments that may give insight into the assessment of the candidate. 6 A pro-forma of this is available from your HR Generalist. Decide on rating scale. The two main ways of rating are either (a) competency model: competent or not yet competent, or (b) a differentiating model: rating candidates on a likert scale (5, 4, 3, 2 or 1 where 5 represents “excellent”). If you wish to use a likert scale, you must be clear on what evidence means a 5 versus a 4 versus a 3 etc. If you wish to use the differentiating model, you are advised to use a 3 point scale as follows: 3: superior demonstration of competency 2: adequate demonstration of competency 1: inadequate demonstration of competency Send this to your HR Generalist for feedback and the quality check. Once you have received feedback, update and revised accordingly. The checklist that the generalist will use is available in Appendix 1. Administration: What exactly do I do? Make sure that you have provided the HR Generalist with the following electronically: o A copy of the informed consent form (see Appendix 2) o A copy of the instructions for applicants o A copy of all material for applicants e.g. any spread-sheets or documents that they will work with o Clear instructions of what the HR Generalist needs to do e.g. if loading of documents onto computers needs to take place. Once loaded, it is advised that you check that this is accurate. Liaise with the HR Generalist as regards whether you need to be available during the assessment process. If the assessment is a practical one e.g. cooking, painting, you will need to be present. If the assessment is computer based or paper based, you don’t need to be present. The HR Generalist will do the actual administration of the test if it is computer or paper-based. If you do the administration, you are required to get the candidate to complete the informed consent documentation. See Appendix 2. 7 Assessment: What exactly do I do? Once the applicants have completed the assessment and where relevant, the HR Generalist will get the completed work to you. This will either be in the printed format or electronically, as requested by you. You then need to mark the assessments using the assessment grid. If the assessment is a practical assessment, you will mark the applicants after the completion of the various task/s. You are encouraged to have at least 2 assessors as this provides a more objective opinion. Produce one assessment grid per applicant. See Appendix 3. Where the assessments are a hurdle, advise the HR Generalist of which applicants are to proceed to the next phase of the assessment. The HR Generalist will contact the successful and unsuccessful applicants and advise them accordingly. Follow-up with the applicant in the interview: What exactly do I do? It is often the case, that one wants to ask the applicants how they experienced the assessment and to probe how they believed they fared. This is often a good reflection of the candidates’ ability to reflect on their own level of competence. In professional jobs, this is important. At other times, one may want to establish from an applicant what the thinking was behind approaching a question in particular way. This gives one insight into why the applicant followed a particular course of action. At other times, one may want to find out why a candidate did not fare well on a particular assessment. These explorations are done by the assessors of the work sample assessment, in the interview. 8 Feedback to the Committee: What exactly do I do? Typically the assessment data is presented to the committee after the interview phase but before the discussion about candidates commences. it is not advisable to present the findings before the interview because the interview is then NOT an independent set of data that can be used to validate or refute the work sample assessment results. The quality of your selection process is improved through triangulation (using independent sources of information). You can provide a copy of each assessment grid for each candidate to the Committee. You can highlight the strengths and weaknesses and raise any concerns. Sometimes the members become “seduced” by the candidate in a selection interview and like to ignore the assessment results. It is important that the Committee is reminded to focus on ALL selection data. Remember that the discussion of candidates must focus on their merit relative to the competencies of the job. Common selection errors include the “halo” or “horns” effects. This is when some data is allowed to create the overall impression of the applicant, either good (halo effect) or bad (horns effect). Now what? You are encouraged to reflect on the effectiveness of the work sample assessment and update it where necessary. Do this soon after the selection process, while the development and assessment is still fresh in your mind. With the successful applicant, once they start, share with them their assessment results and use this as a basis to identify development needs from the start of their employment. 9 HR Checklist for Work Sample Assessments COMPETENCIES 1 The competencies being measured: Are these in the job profile for the post? 2 Can the work sample validly and reliability measure these competencies? Iis there a better assessment tool? 3 Do the exercises designed measure the required competencies? Is there a clear link between these? 4 Is the competency operationalized in terms of the evidence being looked for in the assessment exercises? 5 Does the work sample focus on the most critical competencies? EXERCISES 6 Are the exercises used typical of the kind of work done in this job? 7 Does any exercise require Rhodes knowledge? If yes, is this fair to all applicants? 8 Does any exercise require departmental knowledge? If yes, is this fair to all applicants? 9 Are the instructions clear and unambiguous? 10 Are the exercise/s clearly worded? If relevant is the English appropriate to the level of job? 11 Is the time needed for the entire assessment clear to the candidate? Is this time allocation reasonable? 12 Is there any “double jeopardy”? 13 Are critical competencies measured more than once? ASSESSMENT 14 Does the assessment grid can the relevant competencies from the job profile? 15 Does the assessment grid specify the evidence needed from the exercises? 16 Does the assessment grid specify the rating to be used? Competent/Not yet competent or a likert scale? 17 Are the weightings for the competencies specified? 18 Does the assessment grid have an overall assessment and identify strengths and areas of development? 19 Is there a model solution? STRATEGY 20 Hurdle approach: are the right competencies focused on? 21 Compensation approach: is this the right approach? 22 Competence or differentiation assessment: is the assessment appropriate to achieve these aims? PROFESSIONALISM 23 Is the material presented professionally? 24 Is the informed consent form included and updated? 10 INFORMED CONSENT FORM Work Sample Assessment a. At Rhodes University the assessment of occupational competence is a requirement for the selection process for a number of posts. b. The purpose of the assessment is to offer candidates and employees the opportunity to demonstrate the knowledge, skills, attributes and values required by the relevant job profile. Assessment tools measure whether candidates meet the requirements of a specific position. This particular assessment will focus on the technical skills required for the post for which you have applied. c. This outcome of this assessment may determine whether or not you go through to the next stage of the selection process, the interview. d. Rhodes University requires your consent before proceeding with the assessment process. e. You may choose to not be assessed or withdraw your participation without explanation at any point during the assessment. Should you however choose not to complete the assessment, you forfeit the opportunity to demonstrate your competence. This means that you cannot proceed to the next stage of the selection process. f. The Human Resources at Rhodes University ensures that: Assessors are suitably trained; Assessment instruments are culture and gender fair; Assessments are valid in that they measure competencies required on the job; Guidelines for responsible assessment are followed. g. The responsibilities of the Test Administrator include: Establishing rapport with candidates and putting them at ease; Providing clear instructions; Administering instruments properly. h. Responsibilities of Assessors include: Accurately scoring and interpreting results; Accurately reporting the findings to members of the selection committee; Storing the assessment data; i. Your responsibilities include: Reading instructions carefully; Requesting clarification prior to the start of the assessment process; Working as efficiently as possible; Demonstrating your competence. i. The responsibilities of the Rhodes University’s HR Division include: Managing complaints, queries, disputes or appeals relating to the outcomes of assessment and selection process. 11 I, (enter name and surname below) Surname Name Identity or passport number a. agree to participate in an assessment conducted under the auspices of the Rhodes University HR Division; b. understand that my participation is voluntary; c. have received a clear and comprehensive explanation of the: reasons why I am being assessed; the nature of the assessment methods and procedures; how the results will be used; d. give permission that my results may be made available to: the Assessors so that they may assess the results of my assessment; the Rhodes University Selection Committee; the person in the HR Division who administered this assessment tool; e. give permission, should I be appointed in the position applied for, that my results may be available to my Manager to be used to assess my development needs. f. accept that the Assessors may not be able to explain all aspects of the assessment to me until the assessment has been completed. Signature of candidate : Date: ………../…………./201 : ………………………………………………………… 12 Rhodes University Assessment Assistant to Director/Deputy Director: HR: Competency Assessment Grid for Work Sample Test Name: __________________________________ Date of assessment: _________________________ Rating scale for individual competencies: 4 = above average; 3 = average and suitable for appointment, 2 = below average competency and could be developed, 1 = not competent Competencies (Knowledge, Tested by: Rating: Comments Skills and Attributes) Excellent Computer literacy: able to work with a word processor Advanced ability to communicate in English and reporting writing skills Excellent administrative (good org skills and planning) and shows initiative manage multiple demands and Able to work under pressure Able to draft correspondence Able to create tables Able to open other files and use these and save these appropriately Able to create own folder Words use in correct way Correct grammar, spelling, punctuation Appropriate sentence structure Meaning is clear Correct layout of letters Appropriate diary management Completed all important tasks in in-basket Was able to priorities Followed through on very urgent items to make sure these did not fall through cracks Identified follow through and made plans for own absence Informed manager of what had been done/not done Put documents in order so that someone coming to her desk would be able to locate documents etc. Set up own diary system in relation to manager’s diary in order to have reminders and follow-ups 13 Ability to network and access required information, ability to evaluate quality of Was efficient in methodologies used to get through the work. Identifies not only immediate work but other related work to be done Project plan methodology was sound (alignment with dates of committees, included relevant processes and relevant people, identified gaps to follow up on with manager), considered implementation issues (e.g. communication, training) Able to access website and get necessary information Able to discern which information is appropriate – only relevant information included information with an attention for detail Conceptual ability to High level of accuracy learn quickly and grasp what is needed in terms of information. Service commitment ethic, to continuous improvement Errors minimal Documents presented professionally Follow through on tasks identified Instructions provided where followed Able to grasp key issues in tasks Able to think laterally with regards to tasks i.e. did not only follow instruction but could think of related and other issues Project plan was conceptually sound and identified relevant issues Response to complaint was appropriate Able to differentiate between own issues and that of others Referral of problem to the correct source was done Preparedness to explore if there was an error Concern with service was demonstrated in tasks in general, evidence of service to the manager in making his/her life easier 14 Overall rating for in basket and numerical assessment: 4= above average on competencies. To go through to interview stage. 3 = average. To go through to interview stage. 2 = below average in multiple competencies. Not suitable for the post at this stage. Regret. 1 = below average and unsuitable for post at this point in time. Regret Strengths of the candidate include: Areas of development of the candidate include: Name of assessor: ______________________________ Signature: __________________________ Date: __________________________ Name of assessor: ______________________________ Signature: __________________________ Date: __________________________ Name of HR Generalist: ___________________________ Signature: __________________________ Date: __________________________ 15