Spotting diamonds in the rough:
Mining for top performers with psychological
assessments
Ilona Jerabek, PhD
PsychTests AIM Inc.
Table of contents
•
•
•
•
•
•
•
•
Introduction
Hiring pitfalls
Types of assessments and tests
When to use tests
Selecting the right assessments
Evaluating results/reports and analyzing gaps
Summary and take-home message
Quick overview of PsychTests’ services
How to recognize talent
Selection process
• Screening process – review
of résumés
• Interview
• References and background
checks
• Assessment
• Work sampling
So what if they all look like Superstars?
Screening pitfalls
Résumé
References
• Candidate’s side
• Candidate’s side
–
–
–
–
Cookie cutter
Omissions
Spinning
“Embellishment”
• Recruiter’s side
– Funky résumés or those with things
in common with recruiter get to the
top of the pile
– Screening out people who don’t
meet requirements exactly but who
are close (especially when done by
software)
– Choosing only people who will give
good reference
– Can cheat … fake references
– May not have contact info (turnover)
• Reference
– Many provide only confirmation of
employment
– Fear of being sued if they share
something negative
• Recruiter
– Not asking the right questions
– Not reading between the lines
Interview pitfalls – The candidate
• Candidates well-trained in
interviewing skills
– Rehearsed responses
– Spinning
– Outright cheating
• Good candidates
inexperienced in
interviewing process
– Disadvantaged – have to think
about answers on the spot
– Can miss diamonds in the rough
So you have to
 Read between the lines
 Probe deeper
 Ask unusual questions
 Ask situational questions
• Cultural differences
–
–
–
–
Humbleness
Language issues
Acceptability of spinning
Appropriateness of sharing and
talkativeness in this situation
– Respect for hierarchy/authority
Interview pitfalls – Interviewer bias
• First impression
– It takes seven seconds to make a
first impression
– 50% of it is based on the person’s
appearance
– First opinions are formed in the
first 12 minutes of an interview
• Halo effect
– strength (or weakness) in one
area is generalized to other areas
• Similarity bias
– Looking for someone similar
to self
• Primacy and recency effects:
– The interviews we remember
most are the first and the last of
the group
– We tend to remember the
beginning and the end of
individual interviews
• Rationalization
– Especially of negative aspects
if we like the candidate
• Self-fulfilling prophecy
– Based on first impression,
interviewer subconsciously asks
leading questions, resulting in
confirmation of said first
impression
Consequences of hiring errors
On company level
On individual level
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
High turnover
Low engagement
Poor morale
Lower productivity
Lower quality of work
Lower overall efficiency
Poor customer service
Opportunity cost
“Us vs. Them” attitude
Low job satisfaction
Low motivation
Presenteeism
Absenteeism
Social loafing
Low initiative
Negativity
Poor work ethics
High likelihood of leaving
But if you get it right …
People who perform well have:
• Increased self-awareness
• Improved morale
• Increased retention
• Increased engagement
• Enhanced productivity
• Better job satisfaction
. . . profits surge!
Solution?
• Hire the right person the first time around
– Hire those who match the job, team and company
– Manage the gap between personality and “job persona”
• Facilitate onboarding
–
–
–
–
Facilitate integration into new team and work environment
Train to improve areas that need to be developed
Tailor job functions/tasks to new hire’s strengths and preferences
Prevent future performance issues – address potential problems before they start
• Increase retention of top talent
–
–
–
–
Career development
Personal development
Keep them challenged & interested
Identify leadership potential
If you can’t measure it, you can’t
control it
“Assess and conquer”
• Define the job
• Assess the incumbent
• Assess the candidate
• Perform gap analysis
• Assess and manage
performance
• Assess and manage
promotions
• Retain your best people
Assess the job:
Define the framework
• Task analysis
– Task description
– Shadowing
• Job analysis
• Job description
• Personality profile of the
ideal candidate
• Key characteristics and
attitudes
• Behavioral benchmarks for
the position
• Reach consensus among the
stakeholders
Assess the incumbent:
Create a blueprint
• Strengths
– What traits and attitudes
contribute to good
performance?
– What are the downsides?
• Challenges
– What are the traits and
attitudes that hinder
performance?
– What do you wish you
could change in the
incumbent?
Assess the candidate:
Fit the pattern
• Screen out candidates with the
wrong attitudes or poor work
ethics
• Find people with the right
profile for the position
–
–
–
–
Skills
Personality
Attitudes and values
Work environment preferences
• Accurately predict successful
performance
• Recognize true team players
Many pieces in a puzzle
• ‘Hire for attitudes, train for
skills’ vs. ‘Hit the road
running’
• Cannot possibly assess
everything => identify the
information that is essential
to success on the job
Types of tests
Skills and knowledge
Psychological
• Technical competency
• Personality assessments
–
–
–
–
Theoretical knowledge
Practical skills
Experience
Level of expertise
• Psychological competencies
–
–
–
–
–
–
–
Leadership skills
Management skills
Emotional intelligence
Coping skills
Assertiveness
Communication ability
Soft skills
– Objective
– Projective
• Aptitude, IQ and achievement
tests
– Mechanical reasoning
– Analytical reasoning
• Attitudes assessments
• Rarely used in HR setting
– Neuropsychological tests
– Direct observation tests
Personality tests
• Personality traits
– Is this person a person a good fit into a company’s atmosphere and image (e.g. extrovert
vs. introvert, straight-laced vs. laidback, independent-minded vs. compliant, etc.)?
– Which specific traits are a must for a position (e.g. integrity, self-control, empathy)?
– What makes this person unique? Some quirks are adorable, some may cause problems.
• Values – What is important to this person?
• Beliefs – What views of the world, others, and self does this person hold?
• Attitudes – What kind of a work ethic can you expect? Is s/he assertive,
proactive, risk-taker, procrastinator?
• Motivators – What drives this person?
• Emotions – What makes this person tick? What stresses this person out?
• Behavior – If put in a specific situation, how will this person respond?
What makes this person behave this way?
When to use assessments:
Pre-hire
• Prescreen before interview to identify candidates who
are clearly a bad fit
– narrow down candidate pool to those who meet minimum
requirements (lenient cut-off points)
• Assess candidates who make the first cut for job fit (skills,
personality, attitudes)
– narrow down candidate pool to those with best potential (use
benchmarks to assess match)
• Interview top candidates with test results in mind
– Probe deeper to validate test results
– Evaluate trade-offs
– Evaluate culture fit
When to use assessments:
Onboarding
• Focus on-the-job training and coaching on areas that
need development
• Use insight from tests to manage employees effectively
• Tailor job functions to the new hire's strengths and
preferences
• Facilitate integration into existing teams
• Prevent future performance issues by addressing
potential problems before they start
When to use assessments:
Career development
• Use assessments as a learning and personal development
tool
• Optimize training process
–
–
–
–
Identify training needs (individual and groups)
Offer custom-tailored learning opportunities
Determine the effectiveness of your training process
Monitor individual progress
• Assess who is ready for a promotion, more responsibility
or learning opportunities
• Groom young employees for future leadership positions
When to use assessments:
Engagement & Talent retention
• Optimize motivation
– Understand what motivates individual employees and manage
accordingly
– Ensure right job fit
– Help them grow
– Keep them challenged and interested
• Problem prevention
– Develop leadership and management skills of those in charge
– Monitor trends company-wide
– Identify budding problems before they escalate (e.g. stress
levels)
Tests, tests, tests …
What assessments CAN do
• Cut costs and save time by
– improving the size and quality of
candidate pool by assessing large
number of candidates during prescreening process
– narrowing down the candidate pool
and enabling HR staff to focus on
good fit candidates
• Assess traits and skills that are
relevant to a job (job-specific or
custom-designed competency
and personality tests)
• Identify a person’s strengths and
areas that will need
development/training
• Improve objectivity of selection
process
– Paint a more complete picture of a
person’s skills and personality than
what is offered in a résumé or
references
– Provide objective, standardized
assessment, free of individual
interviewer biases
– Give a chance to real diamonds in the
rough … great candidates who are
humble or untrained in interviewing
skills
– Use existing top-performing
employees as benchmarks to whom
you compare potential candidates
What assessments CANNOT do
• Replace human judgment
• Make the hiring decisions instead
of you
• Predict performance with 100%
accuracy
• Guarantee candidate’s future
success
• Assess all factors that might
possibly play a role in an
individual’s performance
• Guarantee accuracy for everybody
(some particular circumstances in
one’s life might skew results)
• Prevent test-taker from “gaming”
the test
BUT there are ways to minimize
cheating and social desirability
bias
– Validity scales (impression
management, acquiescence, social
desirability index)
– Situational/behavioral questions
rather than adjectives or short
phrases
– Well-designed questionnaire
(avoiding leading questions,
ambiguity, double-barrelled
questions)
How do I judge the quality of a test?
• Statistics – Reliability and Validity
– Test must be created based on APA (American Psychological Association) standards
– Overall “Cronbach’s Alpha” (statistical test of reliability) should be 0.70 or higher
– Determine if any studies have been done on the test
• What population has this test been used on? (Age, Gender , Ethnicity, Education,
Job field)
• Sample must be at least 500 or more people
• The larger the test (i.e. the more scales) the larger the sample should be
– What have the studies conducted on the test revealed?
• If the test is job specific - e.g. sales - determine how people with sales experience
compare to the rest of the population – they should score better
• Determine if test is biased toward a certain population i.e. if one population is
scoring significantly better than the rest – this should not be the case
• Evaluate adverse impact on protected groups – score differences of 10% or more
are unacceptable and the use of the test can be legally challenged
Other important questions to ask a
test creator or distributor
• Are the test items and test results job-relevant?
– If not, the use of test can be legally challenged
• What research and which theories were used to create this test?
• Has the test been revised? What were the revisions, and when was the
last revision?
– Revisions are necessary – new scientific research may reveal new information, social and
economical changes affect norms or context of test items, language changes
• How often are statistical analyses run on this test?
– If significant revisions have been made to the test (new questions, new scales), new data
must be collected and analyzed
• Is this test suitable for my test population?
– Lower level vs. higher-level managers
– Cold sales vs. warm
– Customer service reps in retail vs. call centers
Test Structure – Questions & Reports
Overall tips
• Ask for a sample of the questions or get a demo
• Ask for a sample report
• Assess the length of the test
– Realistically, the entire personality cannot be assessed in 10 minutes
– On the other hand, a 3-hr test would be impractical
• Determine whether the test is assessing all the traits and skills that you
need
• Determine what benchmarks are available, and whether you can create
custom benchmarks
• Verify whether suggested interview questions based on test results are
available
Test items
• Should be relevant to what the test is supposed to assess (face validity)
• Should not include questions about sex life, romantic life – personal life
questions can be used, but cannot be too personal and need to be
justified (e.g. emotional intelligence)
• Must be clear and easy to read, and appropriate in terms of reading level
required
• NOTE: For sake of clarity, some questions need to be direct (e.g. I am
punctual), but may point to the right answer
– “Faking good” on a test can be prevented to a large degree BUT
– Tests in which a lot is at stake (e.g. pre-employment test) should include a scale that
determines whether a person answered in a socially-desirable way – a Faking or
Impression Management scale
Test report
• Verify whether the report offers sufficient information for you to make a
clear decision about a person’s potential
• Ideally, test report should include
– Clear, easy-to-understand graphs with scores
– Definitions of each trait or skill assessed
– Interpretations of a person’s score
• What does a high score indicate?
• What does a low score indicate?
• How does this impact performance?
– Highlights areas that need improvement
– Tips on how to improve on weaknesses
• Unless you have access to a psychologist or a trained HR professional,
make sure the report is written in layperson terms
Mind the gap
GAP
Perform gap analysis
• Measure the difference
between the candidate and
the requirements of the
position
• Determine the actions that
support these required
changes/behaviors
• Identify significant gaps –
changes in behavior and
playing a role of the “job
persona” are required to
close the gap
• Predict the energy
requirements to make these
changes and the resulting
frustrations
Manage the gap
Organize
Adapt
• Assign tasks according to
personal preferences whenever
possible or let employees
volunteer for assignments
• Team up people with
complementary skills, personality
and preferred team roles
• Ensure support and availability of
tools/materials that can help to
bridge the gap
• Adjust management style to what
works for individual employees
• Adapt communication and
information transfer to suit an
individual’s learning style or
intelligence type
• Customize training process to give
employees the skills and tools
they need to succeed
Find your diamond in the rough
• Know what you want
• Assess personality, attitudes
and skills of candidates using
psychological tests
• Probe deeper using interview
questions based on test results
• Keep your eyes open and
avoid personal biases
• Evaluate tradeoffs
• Keep polishing your diamond
And most importantly …
Keep polishing your diamond to really make it sparkle
PsychTests’ solutions
ARCH Profile
Most popular tests
• Online delivery system with more
than 80 assessments available
• Features
•
– Import users or set up manually
– Assign assessments to users
– Invite users via email
• Decision support tools
–
–
–
–
–
–
Instant reports (web or PDF)
Benchmarks (industry, custom)
Interview questions
Comparison tools
Job analysis
360 & multi-rater assessments
•
•
•
•
•
•
•
•
•
•
•
•
•
Multi-dimensional Emotional Intelligence Quotient
(MEIQ)
Multi-dimensional Intelligence Test (MIT)
Work Accident Likelihood Assessment (WALA)
Entrepreneurial Aptitude Profile (EntAP)
Customer Service Profile (CSP)
Leadership Potential Assessment (LEAP)
Logic IQ Test (LIQT)
Management Skills and Styles Assessment (MANSSA)
Salesperson Personality Profile (SPPP)
Coping Skills Assessment (COSA)
Employee Attitude and Personality Test (EAPT)
IT Aptitude Personality & Attitude Profile (ITAPAP)
Work Integrity Test (WINT)
Big 5 Personality Test (AMPM)
PsychTests AIM Inc.
• Website
– http://www.archprofile.com/corporate/g/v/i/PP1
• Test catalogue
– http://www.archprofile.com/corporate/g/v/i/PP1_cat
• White papers
– http://corporate.psychtests.com/resources