Where Leadership Assessment
Centers Are Going and
How They Will Get There
Improvements in Selection/Promotion
Accuracy and Development Success
Presented by:
William C. Byham, Ph.D.
Chairman & CEO
Development Dimensions International, Inc.
March 2013
1
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
2
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Organizations
want:
3
1. Accurate and understandable
information on leadership and
management strengths and
weaknesses of candidates for key
positions.
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Of 12,000 senior
leaders, only 32% rate
their bench as strong
12,000 leaders participating in DDI’s Global Leadership Forecast 2011
1 in 3 Employees Feel Their Frontline
Supervisor is Ineffective as a Leader
5
Source: DDI Global
Leadership Forecast, 2011
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Organizations
want:
6
2. Help to speed learning and
application of new skills—change
behavior.
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
63% of Firstand Mid-Level
Managers Feel
Their
Leadership
Development is
INEFFECTIVE
Source: DDI Global
Leadership Forecast, 2011
7
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Organizations
want:
8
3. Better, more obvious links with
training and development solutions.
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Organizations
want:
9
4. Reliable assessment judgments
when people are assessed by
different assessors—often in different
locations.
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Organizations
want:
5. Decrease in total assessment
center costs.
• Participant time
• Assessment center
operations
• Travel, hotels, etc.
• Feedback, follow-up
Costs10 are holding back the use of assessment centers!
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Four Ideas
11
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Idea
#1
Change Assessment
Center Targets
13
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Integrated Talent Management Systems
Built Around Competencies
Promotion/
Placement
Succession
Planning
Selection
Competencies
Organizational
Alignment/
Observation/
Coaching
Performance
Appraisal
Compensation
Training/
Development
14
Career
Planning
Training and
Development Needs
Assessment
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
But now I feel that I
was wrong about
focusing only on
competencies for
assessment and
development…
15
©
© Development
Development Dimensions
Dimensions Int’l,
Int’l, Inc.,
Inc., MMXII.
MMXI. All
All rights
rights reserved.
reserved.
I now believe that we have to go deeper
into our analysis of behavior to…
KEY ACTIONS!
A
competency
is a behavior!
17
A competency
is a collection
of behaviors
that relate to
effective job
performance.
©
© Development
Development Dimensions
Dimensions Int’l,
Int’l, Inc.,
Inc., MMXII.
MMXI. All
All rights
rights reserved.
reserved.
DDI Calls the Behaviors within a Competency
“Key Actions”
Influencing Others
B B
Coaching
B
B
B
B
B
B
B
B
B
B
B
B
Decision Making
B B B
B B
B
B
B
B
B
B = Behaviorally-defined Key Actions
18
Delegating
©
© Development
Development Dimensions
Dimensions Int’l,
Int’l, Inc.,
Inc., MMXII.
MMXI. All
All rights
rights reserved.
reserved.
Advantages of Assessing
at the Key Action Level
More accuracy—higher validity
2. Better understanding of why a person is
recommended or not
3. More precise development insights—
where an individual should focus during
training and development
1.
Actionable!!
19
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
Resolving Workplace Conflict
Key Actions
•
States purpose and importance
• Clarifies the issues
• Stays focused on resolution
• Develops others’ and own ideas
• Remains open to all sides
• Initiates action
• Close – check for understanding and
agreement
20
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
Is Assessment at the Key Action
Level Possible?
Assessment Targets
• 10 target competencies
10
• 5 Key Actions per competency
50
• 5 Observations per Key Action
250
• Key Actions reduced by
overlap of some Key Actions
21
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
120 - 150
Interaction
Essentials
SM
Influencing Others
B B
Coaching
B
B
B
B
B
B
B
B
B
B
B
B
Decision Making
B B B
B B
B
B
B
B
B
B = Behaviorally-defined Key Actions
B = Behaviorally-defined Key Actions which are
Interaction Essentials
22
Delegating
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
Inter-correlation of
leadership and
management
competencies = 0.63
n=38,912
WHY?
Delegating
Decision-Making
Influencing
Coaching
Job Performance
Interaction Essentials
(That Meet Personal and Practical Needs)
Source: DDI Analysis of Cross-Organizational Database of Performance Ratings, 2011; n=19,157
.32
Correlation of Essentials with job
performance—higher than any
single competency.
Source: DDI Analysis of Cross-Organizational Database of Performance
Ratings, 2011; n=19,157
25
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
First- and Second-level Management
Proficiency in Interaction Essentials
SM
n = 4,141 US managers who went through Manager Ready.
26
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
First- and Second-level Management
Proficiency in Interaction Essentials
SM
n = 2,191 Chinese managers who went through Manager Ready.
27
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Recognizing that there is
an overlap in competency
Key Actions, makes it
possible to target Key
Actions rather than
competencies.
28
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Idea
#2
Electronically Capture Assessee
Behavior
•
Audio
• Video
• Computer
responses
Idea
#3
Use Human Assessors Where They
Make the Greatest Contribution –
Use Algorithms Everywhere Else
Assessors
•
Observe behavior
• Rate Key Actions
Algorithms
•
Combine behavior observed in
different simulations into Key Action
ratings
• Key Action ratings into competency
ratings
• Competency ratings into overall
ratings
31
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Algorithms Look for Patterns Missed in Most
Assessment Centers (e.g., Situational Insights)
Competency: Decision-Making; Key Action: Judgment
Key Action Scores on 1 – 5 Scale
Assessor Rating of
Judgment
Situation
Observation 1
2
People Problem
Observation 2
5
Numbers/Facts
Observation 3
1
People Problem
Observation 4
4
Numbers/Facts
Algorithms Have Always Been Accepted by
the International Congress on
Assessment Center Methods
33
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
34
©
© Development
Development Dimensions
Dimensions Int’l,
Int’l, Inc.,
Inc., MMXII.
MMXI. All
All rights
rights reserved.
reserved.
How to Have Very High Reliability at
Key Action Level
•
•
•
•
•
Pinpointed Key Action
assessment targets
Specialization of assessors
Extensive training
Every 10th assessor’s
judgment independently
checked
Assessors get constant indepth feedback
35
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Criterion Validity of Ideas 1, 2, and 3
(Combined)
Study
Validity
AT&T (multiple studies in 1960s)1
.37 (median)
Meta-Analysis (1987)2
.37
Meta-Analysis (2003)3
.36
Assessment Center that
(2010)
Incorporates Ideas 1 & 2
.40
1. Thornton & Byham (1982, Assessment Centers and Managerial Performance)
2. Gaugler, Rosenthal, Thornton, & Benston (1987, Journal of Applied Psychology, 72, 493-511)
3. Arthur, Day, McNelly, & Edens (2003, Personnel Psychology, 56, 125-154)
36
© Development Dimensions Int’l, Inc., MMX. All rights reserved.
Summary: Advantages of Focusing Assessors
and Using Algorithms in Assessment Centers
•
•
•
•
37
Validity increased
Reliability vastly increased
Important situational insights uncovered
that will increase training effectiveness
Decreased costs
© Development Dimensions Int’l, Inc., MMXII. All rights reserved.
Training and
Development
38
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
Idea
#4
Connect Assessment Insights
with Development Activities
Provide a “fix” for things
assessed
• Save money by not
training people on what
they know or are good at
• Take advantage of Key
Actions to speed training
• Provide multiple practice
opportunities where Key
Actions overlap—most
important Key Actions
•
Interaction
Essentials
SM
Influencing Others
B B
Coaching
B
B
B
B
B
B
B
B
B
B
B
B
Decision Making
B B B
B B
B
B
B
B
B
B = Behaviorally-defined Key Actions
B= Behaviorally-defined Key Actions which are
Interaction Essentials
40
Delegating
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
How Development
Components Are Linked
Determination of focus Key Action
development needs based on assessment report and
discussion with manager

Training to learn and practice focus needs “10”

Deliberate Practice with manager or others “20”

Follow-up development and reinforcement to assure
training sticks “70”

Adoption of Key Actions into ongoing personal skill set
41
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Ongoing Practice and
Measurement of Key Actions
•
Reminders of key learnings
and forms provided in
training
• Additional simulations for
more practice
• Games to build skills
• Ways to collect ongoing
feedback (Social Media)
42
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Advantages of Linking Assessment
with Training and Development
•
Focuses training and development efforts
• Guides deliberate practices
• Motivates learners (understand need)
• Speeds training and development
• Decreases training and development
costs
• Makes assessment and training/
development into an integrated system
43
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Is the Effort Worth It?
44
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Proof that Key Actions
Can Be Developed
Percent of trainees
displaying behaviors often
Improvements after Training
n= 2,637 leaders and 4,120 of their managers, colleagues, and direct reports.
45
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
How did I do on the organizational
wants?
Understand bench strength
2. Speed learning and development
3. Link assessment and development
4. Provide fair and reliable judgments
5. Decrease total assessment center costs
1.
46
© Development Dimensions Int’l, Inc., MMXIII. All rights reserved.
Questions
Comments
Concerns
47
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
Thank
you.
48
© Development Dimensions Int’l, Inc., MMXI. All rights reserved.
Manager Ready
®
U.S. Patent Pending
• Identifies candidates for first and secondlevel management positions
• Defines development needs
• Supports Human Capital Management
• Speeds training and learning
Manager Ready
®
U.S. Patent Pending
Continued
• 9 target competencies, 3.5 hours
• Day-in-the-life experience, realistic job preview
• Participants respond by typing memos,
completing forms, making decisions in game-like
simulations involving planning and fact-finding
• Open-ended responses
• Scored by highly-trained assessors