How to use Trait Activation Theory in Assessment Centre Practice
advertisement
Workshop Trait Activation How to use Trait Activation Theory in Assessment Centre Practice Prof. Dr. Filip Lievens Ghent University, Belgium [filip.lievens@ugent.be] http://users.ugent.be/~flievens/ Objectives Presenting the knots & bolts of using trait activation theory in assessment centres Developing assessment tools & rating aids grounded in trait activation theory Prof. Dr. Filip Lievens Workshop Trait Activation Structure Relevance Theory Practice Evidence Epilogue Relevance Prof. Dr. Filip Lievens Workshop Trait Activation AC Guidelines AC designers should attempt to design exercises that evoke a large number of job-related behaviors because this should give assessors enough opportunities to observe job-related behavior. International Task Force on Assessment Center Guidelines (2009) “Behaviors, not exercises, are the currency of assessment centers. Exercises are merely the stimuli to elicit behaviors” Howard (2008, p. 101) Yet … Fast stream of ongoing behavior Few behaviors per dimension Assessors rely on red “hot” behaviors to rate multiple dimensions Reliability/validity of within-exercise dimension ratings ‣ Prof. Dr. Filip Lievens Exception: Communication as a dimension Workshop Trait Activation Traditional AC Matrix Dimensions Exercises Sensitivity Communication Case analysis Discussion Role-play × × (×) × × × Relevance for HRM Lack of theories in staffing Lack of theories in assessment Lack of theory-based design Prof. Dr. Filip Lievens Stress Resilience × In-basket Presentation Planning × (×) × × Workshop Trait Activation Trait Activation: The Theory History Situation – Trait debate ‣ Situationists Personalists => Interactionists Renaissances ‣ ‣ ‣ Big Five / Five-factor Model Situation Interactionism ‧ ‧ Prof. Dr. Filip Lievens Cognitive-Affective Personality System (Mischel & Shoda, 1995) Trait Activation Theory (Tett & Burnett, 2003) Workshop Trait Activation Trait Activation Theory: Axiom 1 Situational trait relevance ‣ ‣ ‣ ‣ A situation is relevant to a trait if it provides cues for the expression of trait relevant behavior. A qualitative feature of situations that is trait specific; it is informative with regard to which cues are present to elicit behavior for a given latent trait. Cues can be at the task, social or organizational level. Cf. The channel you are listening to on the radio Trait Activation Theory (TAT) Underlying trait Activation Exercise situation Task Social Organizational Behavior in AC exercise Prof. Dr. Filip Lievens Workshop Trait Activation Which traits are activated in exercises? Note. Table adapted from Lievens et al. (2006). E = extraversion. A = agreeableness. C = conscientiousness. ES = emotional stability. O = openness to experience. LGD = leaderless group discussion. Which dimensions are linked to traits? Note. Table adapted from Lievens et al. (2006). E = extraversion. A = agreeableness. C = conscientiousness. ES = emotional stability. O = openness to experience. Prof. Dr. Filip Lievens Workshop Trait Activation Trait Activation Theory: Axiom 2 Situational strength ‣ A continuum that refers to how much ambiguity there is with regard to how the situation is perceived. ‣ Strong situations = unambiguous demands, likely negating almost all individual differences in behavior. ‣ Weak situations = ambiguous demands, enabling variability in behavior to be observed ‣ Cf. The volume of the radio Trait Activation Theory (TAT) Underlying trait Activation Exercise situation Task Social Organizational Behavior in AC exercise Prof. Dr. Filip Lievens Strength of situation (4 C’s) - Clarity - Consistency - Consequences - Constraints Workshop Trait Activation Trait Activation Theory: Axiom 3 Aggregation principle ‣ The sum of a set of measurements is more stable than any single measurement from the set. ‣ Cf. tests, scales or questionnaires Trait activation: The Practice Prof. Dr. Filip Lievens Workshop Trait Activation Assignment 1: Applying TAT Convene in small groups. How can we apply trait activation theory in assessment centre exercises? Discuss possible approaches/tools to evoke behavior in assessment centre exercises? Approaches “Deliberately introduce multiple dimension-relevant items or problems within the exercise and to score such items” (Brannick, 2008, p. 132). “Designers should construct specific stimuli to elicit the kinds of behaviors to be measured and guide assessors to their placement and relative importance… designers must do more than create work samples; they must develop simulations that will best elicit the desired behaviors. We need to develop a much better understanding of the kinds of AC challenges that will bring out the behaviors associated with current and evolving positions and diverse business challenges.” (Howard, 2008, p. 101) Prof. Dr. Filip Lievens Workshop Trait Activation Alternative matrix Sensitivity × × × × × × Planning Stress Resilience × × × × × × × × × × × × × × × × × × × × × Assignment 2: Exercise scenarios Convene in small groups. Generate short situations (task & group) that activate: ‣ Openness to experience ‣ Emotional Stability ‣ Conscientiousness ‣ Extraversion ‣ Agreeableness Prof. Dr. Filip Lievens Workshop Trait Activation Side benefit: Alternate form construction Item generation theory Situational templates as radicals ‣ Determine difficulty ‣ Can be used across many exercises Content as incidentals ‣ Do not determine difficulty ‣ Cosmetic (contextual) layers Cost & time efficient (more basic templates)? Side benefit: SJT / SI items You are talking to an important customer on the phone when another customer interrupts, saying s/he is unhappy with your store and wants attention right now. S/he is speaking in a loud voice that other customers can hear. What do you do? ‣ ‣ ‣ ‣ Help this customer immediately and hang up the phone Ask this individual if s/he can wait until you are done on the phone Ask the customer on the phone if you can call him/her back Wait to help this customer until you finished with the individual on the phone A customer is telling you a personal story that is completely unbelievable and that you think is factually incorrect. What do you do? ‣ ‣ ‣ ‣ Prof. Dr. Filip Lievens Smile and nod in agreement with the customer, and say you agree with the story Smile and nod in agreement with the customer, but say nothing Nod in agreement with the customer, but say nothing Nod in agreement with the customer, then correct the customer’s story Workshop Trait Activation Role-player prompts One purpose: Realism ‣ “You are an experienced IT engineer, etc.” Another purpose: Elicit behavior via prompts ‣ Predetermined verbal & nonverbal cues that a role-player consistently provides in an AC exercise across candidates to elicit behaviors related to specific job-related dimensions ‣ Multiple prompts per dimension Exercise instructions One purpose: Realism ‣ Provide background information about the fictitious organization, key persons, & problems ‣ Provide general expectations about what ought to be done Another purpose: Evoke behavior ‣ Predetermined specific instructions given to candidates at the beginning of AC exercises to elicit job-related behavior Prof. Dr. Filip Lievens Workshop Trait Activation Assignment 3a/3b: Prompts & instructions Convene in small groups & study the exercise of A&DC. Develop verbatim role-player prompts for evoking behavior. Develop verbatim exercise instructions for evoking behavior. Focus on behavior related to the dimensions of ‣ ‣ ‣ ‣ Problem analysis & solving Planning & organization Interpersonal sensitivity Tolerance for stress Vary the situational strength of one of the prompts/instructions. Technological stimuli Same logic as prompts Examples Prof. Dr. Filip Lievens Workshop Trait Activation Overview TAT Trait activation Trait evaluation Demonstrating candidate behavior Observing & rating candidate behavior Reporting candidate behavior Exercise Scenarios Exercise Instructions Role-player Prompts Technological Stimuli Assessor selection & training Assessor aids (checklists) Feedback reports Trait Activation Theory (TAT) Underlying trait Activation Exercise situation Task Social Organizational Behavior in AC exercise Evaluation Prof. Dr. Filip Lievens Job performance (+/0/-) Strength of situation - Clarity - Consistency - Consequences - Constraints Workshop Trait Activation Assessor screening/selection Assessors can be screened on the basis of their • Trait knowledge • Trait-situation knowledge (TAT) ⇒ Christiansen et al. (2005) developed such an inventory ⇒ Do you want to test yourself? Example items of inventory You have been working under the supervision of a senior co-worker for six months. You know your job as well as you will ever know it. The senior co-worker has found another job and the department head has said she is thinking of finding someone else to supervise you. ‣ ‣ ‣ ‣ Sociability Empathy Complexity Need for Autonomy Your current job requires that you work alone at a computer station. You have just heard of an opening in customer service. No additional training is needed and the salary would be the same as in your current position. ‣ ‣ ‣ ‣ ‣ Prof. Dr. Filip Lievens Risk Taking Complexity Empathy Sociability Organization Workshop Trait Activation Assessor training Adjust assessor training? • Let’s familiarize assessors with exercise stimuli. Assessor rating aids Adjust behavioral checklists? • Let’s incorporate exercise stimuli that activate behavior. Feedback reports Dimension-based ‣ “You score weak on resilience.” Exercise-based ‣ “You score weak in the oral presentation.” TAT-based ‣ “You score weak in situations that put you under pressure.” Prof. Dr. Filip Lievens Workshop Trait Activation Trait activation: The Evidence Lievens, F., & Schollaert, E. (2011). Reshaping exercise design in assessment centers: Theory, practice, and research (pp. 47-60). In N. Povah & G. C. Thornton III (Eds.) Assessment centres and global talent management. Gower: Surrey, UK. Assignment 4: Quiz Indicate whether the following statements are “True” or “False”. 1. When role-players use prompts to evoke behavior, they note down 20% more behaviors. 2. For all dimensions, the use of prompts leads to assessors observing more behavior. 3. The use of exercise stimuli (e.g., role-player prompts) planted into exercises lowers candidates’ perceptions of the opportunity to perform in the exercises. 4. When stimuli are built into assessment center exercises, candidates feel that the exercises become interpersonally “colder”. 5. So far, it has not been possible to evoke more candidate behavior via the use of specific exercise instructions. 6. The inter-rater reliability among assessors is higher when exercise stimuli are used to activate behavior. 7. It makes a difference whether assessors are familiarized with the exercise stimuli that are used to evoke behavior. 8. When assessors know which exercise stimuli evoke which behavior, they make more distinct evaluations on the dimensions. Prof. Dr. Filip Lievens Workshop Trait Activation Research issues ‣ ‣ ‣ ‣ ‣ ‣ ‣ What is the impact on the observation process? What is the impact on the rating process? How does it affect reliability? How does it affect validity? What are the effects on what is being measured? How do candidates experience this type of exercises? … Research (1) Exercise stimuli ‣ Use of prompts by role-players quadrupled. Assessees ‣ Favorable perceptions of job-relatedness & opportunity to perform were not affected. ‣ Favorable perceptions of interpersonal warmth were not affected. Prof. Dr. Filip Lievens Workshop Trait Activation Research (2) Assessors ‣ No effects of instructions ‣ When assessors were not familiarized with prompts ‧ Only effects on observation (20% more behaviors) ‧ More classification errors ‧ No effects on other psychometric properties ‣ When assessors were familiarized with prompts ‧ Positive effects on observation, classification, inter-rater reliability, convergent validity, and discriminant validity Research (3) What are determinants of performance? ‣ Spontaneous vs. Elicited ‣ Motivation vs. Ability ‣ Typical vs. Maximal Prof. Dr. Filip Lievens Workshop Trait Activation Epilogue Assignment 5: Reflection & action Alternate exercises What were your key learning points? What are the biggest advantages of using trait activation theory in assessment practice? What are potential disadvantages How will you implement trait activation theory in your assessment practice? ‣ What are the opportunities/ hindrances? How will you persuade others to use trait activation theory in their assessment practice? Prof. Dr. Filip Lievens Workshop Trait Activation Thank you More info on http://users.ugent.be/~flievens Reading list Christiansen, N. D., Wolcott-Burnam, S., Janovics, J. E., Burns, G. N., & Quirk, S. W. (2005). The good judge revisited: Individual differences in the accuracy of personality judgments. Human Performance, 18, 123–149. Haaland, S., & Christiansen, N. D. (2002). Implications of trait-activation theory for evaluating the construct validity of assessment center ratings. Personnel Psychology, 55, 137–163. Lievens, F., Tett, R.P., & Schleicher, D.J. (2009). Assessment centers at the crossroads: Toward a reconceptualization of assessment center exercises. In J.J. Martocchio & H. Liao (Eds.), Research in Personnel and Human Resources Management (pp. 99-152). Bingley: JAI Press.Thornton & Muller Hanson Lievens, F., & Schollaert, E. (2011). Reshaping exercise design in assessment centers: Theory, practice, and research (pp. 47-60). In N. Povah & G. C. Thornton III (Eds.) Assessment centres and global talent management. Gower: Surrey, UK. Meyer, R. D., Dalal, R. S., & Hermida, R. (2010). A review and synthesis of situational strength in the organizational sciences. Journal of Management, 36, 121-140. Schollaert, E., & Lievens, F. (in press). Building situational stimuli in assessment center exercises: Do specific exercise instructions and role-player prompts increase the observability of behavior. Human Performance. Schollaert, E., & Lievens, F. (2011). The use of role-player prompts in assessment center exercises. International Journal of Selection and Assessment, 19, 190-197. Tett, R. P. & Burnett, D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88, 500-517. Prof. Dr. Filip Lievens
