BEHAVIORAL AND SOCIAL SCIENCES
A T T HE N ATIONAL R ESEARCH C OUNCIL
www.nationalacademies.org/bbcss
REPORT BRIEF
M
A A
A
P
A
H
C
B
I
P
R
:
G
M
Selec ng U.S. military enlisted recruits who do
not succeed, for any number of reasons, comes at
great expense in me and money invested in each
recruit, in cri cal job vacancies, and ul mately in
force readiness. The military’s accession system
is a carefully designed, selec ve process in which
poten al recruits are assessed through formal tests
of cogni ve knowledge, skill, and ability; recently
developed tests of personality; and assessments
of moral character and physical and medical readiness to serve. In many aspects, the process of selecon and assignment is highly effec ve, but future
assessments of performance poten al may be
en rely different from those used today, in what
is assessed and in how it is assessed. What if future
assessments could predict individual and group success in the U.S. Army with greater accuracy, allowing
the military to screen in soldiers who will be highly
sa sfied with the Army experience in general and
their occupa onal specialty in par cular? What
if the selec on process could reduce subsequent
challenges to unit success that result from soldiers
with low mo va on or other problem behaviors?
What if a ri on rates before comple on of service commitment could be reduced? While the
military is a unique employer in many ways—for example, in its accession system, occupa onal
and unit assignment process, and service commitment—it also shares many of the same interests as other organiza ons who seek to a ract, select, and retain the best individuals into the
fields and jobs in which they succeed. While criteria for selec on should reflect the par cular
organiza on’s values, the interests underlying these criteria are o en cross-cu ng.
In a new report, Measuring Human CapabiliƟes: An Agenda for Basic Research on the Assessment of Individual and Group Performance PotenƟal for Military Accession (2015), an expert
commi ee of the Na onal Research Council recommends a basic research agenda for the U.S.
Army Research Ins tute for the Behavioral and Social Sciences to undertake to maximize the
efficiency, accuracy, and effec ve use of human capability measurement in the military’s process for selec ng soldiers and assigning occupa onal special es. Much of the commi ee’s data
gathering in support of this report was conducted during a public workshop held in April 2013,
National Academy of Sciences • National Academy of Engineering • Institute of Medicine • National Research Council
the presenta ons and discussions of which are
summarized in a prior publica on, New DirecƟons
in Assessing Performance PotenƟal of Individuals
and Groups: Workshop Summary (2013).
reasoning), the iden fica on and predic on of
new outcomes (e.g., teamwork behavior), and
methods and methodology for assessing individual
differences.
PERSONNEL SELECTION CRITERIA
IDENTIFICATION AND MEASUREMENT
OF NEW PREDICTOR CONSTRUCTS
Measures of individual difference that predict job
performance can be subdivided into “can do” and
“will do” predictors. The Armed Services Vocaonal Ap tude Ba ery (ASVAB) is a cogni ve,
knowledge, skill, and ability ba ery that focuses
on “can do” skills. The ASVAB has documented
accuracy for predic ng enlistees’ technical performance following appropriate job-specific training. Recently, the ASVAB has been supplemented
by the Tailored Adap ve Personality Assessment
System (TAPAS), administered to certain military
service candidates to assess “will do” personality a ributes important for predic ng job performance and risk of a ri on. ASVAB and TAPAS are
strong measures of the ability and personality
domains they measure, but they do not predict
outcomes of interest perfectly, nor do they predict
all outcomes of importance to the military’s organiza onal values and missions. It is possible that
by expanding or modifying its personnel selec on
criteria, the Army could improve the predic ve
accuracy of the selec on process for enlisted service members.
Fluid Intelligence, Working Memory Capacity,
Executive Attention, and Inhibitory Control.
Successful job performance requires abili es in
reasoning and problem solving that are not adequately assessed by crystallized intelligence tests
of learned and acquired skills. The psychological,
cogni ve, and neurobiological mechanisms underlying fluid intelligence, working memory capacity, execu ve a en on, and inhibitory control
contribute to demonstrated emo onal, behavioral, and impulse control important for success
in many jobs. While the importance of individual
differences in these constructs is known to impact
job performance, research is needed to ascertain what the four constructs have in common
and what makes each dis nct from the others.
Results from basic research into the underlying
mechanisms would inform future development
of efficient computer-automated assessment of
relevant cogni ve, personality, and physiological
dimensions.
Cogni ve Biases. Research into cogni ve biases
indicates humans are suscep ble to errors in judgA RESEARCH AGENDA TO IMPROVE
ment or decisions when available informa on and
SELECTION SYSTEMS
choices are not thoroughly considered. However,
In this report, the commi ee recommends a
thinking shortcuts may be necessary and even
basic research agenda to supplement the Army’s
lead to be er decisions in certain circumstances
current enlisted soldier accession system with
that require fast, nonconscious, or automa c
addi onal predictors of individual and collec ve
decisions. To aid understanding of how cogni ve
performance that have the poten al to improve
biases affect decisions, especially considering
the speed at which cri cal
the already-high quality of
decisions must some mes
accession decisions. The
Given the large numbers of
topics included were carebe made in military environpotential soldiers screened
fully considered within a
ments, research should be
each year and the high costs
conducted into stable indinumber of constraints, such
of decision errors, even small
as the need for pre-accession
vidual differences that influincreases in the predictive
mass administra on in a costence a tendency to engage
accuracy of a selection system
in cogni ve biases. Future
effec ve manner. The comresearch should examine
mi ee focused on immedican be of great value.
ate research opportuni es
poten al correlates between
with high near-term payoff, but also considered
suscep bility to cogni ve biases and results of
research areas where future technological develtradi onal measures of personality and cogni ve
opments could significantly change the feasibility
ability tests and informa on-processing factors
of opera onal use in the long term. The research
(such as working memory, execu ve a en on,
agenda includes the iden fica on and measureand inhibitory control). Studies to understand how
ment of new predictor constructs (the a ribute
cogni ve biases affect performance should also
label a ached to a measure, such as arithme c
consider contextual factors; fast thinking might
2
Measuring Human Capabilities
be beneficial in some circumstances and lead to
poor decision making in others.
in situa ons that elicit strong emo ons, such as
fear, anger, or extreme me pressure. Defensive
reac vity, emo onal regula on, and performance
Spa al Abili es. Spa al abili es—the ability
under stress can be collec vely referred to as “hot
to mentally manipulate, understand, and recall
cogni on” constructs. Future research in this
spa al rela onships among
area is needed to explore
objects—impact job perThe ASVAB and TAPAS are strong
measures of improved or
formance in many career
predictors of key outcomes
diminished cognitive and
fields, including expanding
behavioral performance
of
importance
to
the
military,
technology careers that rely
and physiological markers of
but they do not predict these
upon accurate and meaningunconscious effects of emooutcomes perfectly—raising
ful interac ons with images,
on. The predic ve validity
computer graphics, and
the question of whether
of hot cogni on measures
data visualiza on. While the
prediction could be improved
requires an understanding
ASVAB currently includes
through measures of additional
of how hot cogni on relates
one spa al ability measure,
to and differs from other perindividual-difference
constructs.
Assembling Objects, future
sonality constructs (such as
research into the interrelaConscien ousness or Agreeableness) currently
onships among mul ple facets of spa al abilconsidered in the Army’s selec on system.
ity as well as the degree to which sex differences
are mi gated or accentuated by training or life
Adaptability and Inven veness. Measures are
needed to assess individuals’ adaptability and
experiences will ascertain the best measure of
inven veness—the ability to act effec vely in
spa al ability for outcomes important for military
job performance. Addi onally, advances in techchanging, challenging, and unpredictable environnology have expanded available tes ng methods,
ments. One promising line of inquiry is idea-genand future research into the design and adminisera on measures, in which an open-ended task
tra on of spa al abili es tests will benefit from
allows for evalua on of the frequency and qualthese developments.
ity of ideas generated by an individual. Another
promising assessment method is the iden ficaon and development of narrow personality meaIDENTIFICATION AND PREDICTION OF
sures
that are likely to yield stronger correla ons
NEW OUTCOMES
to adaptability and inven veness than are broad
Teamwork Behavior. Soldiers’ ability to contribpersonality variables, such as Openness.
ute to effec ve teamwork is a cri cal component
to the success of the Army small unit. Addi onMETHODS AND METHODOLOGY
ally, the Army’s expanding role in mul na onal
coali ons, joint forces opera ons, and other ad
Psychometrics and Technology. As future research
hoc teams faces soldiers with new challenges and
iden fies addi onal a ributes of interest for selecopportuni es in teamwork. A military selec on
on purposes, it will be necessary to improve psysystem that includes the iden fica on of indichological-assessment measurement methods,
vidual a ributes predic ve of success in a team
emerging assessment technologies, and sta s cal
environment could broadly enhance the collecanalysis approaches in order to increase the preve capacity to perform. While there has been
cision, validity, efficiency, and security of assessprogress in iden fying such a ributes, further
ments without demanding significantly addi onal
research is needed to expand understanding of
tes ng me. Poten al topics of research include,
and assessment metrics for team outcomes and
for example, further development of item generaeffec veness.
on; test assembly; rank, preference, and other
response methods; and detec on of and defense
against test compromise. Future research is also
HYBRID TOPICS WITH JOINT FOCUS
needed on how best to collect and interpret appliON NEW PREDICTOR CONSTRUCTS
cant data from mul ple sources, including data
AND PREDICTION OF NEW OUTCOMES
from experience, interac ve assessments, and
Hot Cogni on: Defensive Reac vity, Emo onal
background informa on.
Regulation, and Performance under Stress.
Success in many military environments and job
Situa ons and Situa onal Judgment Tests. Situoccupa ons requires the ability to func on well
a onal judgment tests offer unique tes ng opporMeasuring Human Capabilities
3
tuni es that require test takers to use judgment to interpret, evaluate, and weight alternate courses of
ac on in response to hypothe cal real-world problems. Innova ve formats for presen ng and experiencing situa ons, such as immersive computer-generated graphics, will offer insight into how a prospec ve
soldier might react to a par cular military environment.
Assessment of Individual Differences through Neuroscience Measures. Current neuroscience measures
may not be op mal for tes ng the performance capability of military recruits, but science-based strategies
for monitoring neural ac vity may be useful for understanding test performance and for improving the
validity of assessments. During assessments, neurophysiological biomarkers—measurements of bodily
func ons that provide insight into psychological state or behavior—can serve as robust and objec ve
measures of test taker experiences such as anxiety, a en on, and mo va on. This informa on may provide insight for the design of test environments that facilitate performance reflec ng actual skill levels
and capacity, thereby increasing tests’ accuracy.
IMPLEMENTATION OF THE RESEARCH AGENDA
The research agenda recommended by the commi ee includes many poten al direc ons in which the
Army may choose to develop its selec on systems, based upon forward-looking considera on of personnel selec on in the future. The research agenda is not priori zed, and the topics should be pursued at
levels commensurate with the outcomes of greatest import to the Army. In a con nued austere budget
environment, implementa on of any por on of the research agenda would reflect progress. However,
quicker progress and poten al to capitalize on synergies across projects would be more likely if mul ple
mul faceted projects were implemented simultaneously. In the commi ee’s opinion, implementa on
of an effec ve and expedi ous research program to enhance soldier selec on would require a supplemental funding commitment to the ARI Founda onal Science Research Unit in the range of $3.5 million
to $7 million per year.
COMMITTEE ON MEASURING HUMAN CAPABILITIES: PERFORMANCE POTENTIAL
OF INDIVIDUALS AND COLLECTIVES
PAUL R. SACKETT (Chair), Department of Psychology, University of Minnesota, Minneapolis; GEORGIA T.
CHAO, Eli Broad Graduate School of Management, Michigan State University; ANN DOUCETTE, The Evaluators’ Ins tute, The George Washington University; RANDALL W. ENGLE, School of Psychology, Georgia Ins tute
of Technology; RICHARD J. GENIK II, Department of Biomedical Engineering, Wayne State University College
of Engineering and School of Medicine, Detroit, MI; LEAETTA HOUGH, Dunne e Group, Ltd. Saint Paul, MN;
PATRICK C. KYLLONEN, Center for Academic and Workforce Readiness and Success, Educa onal Tes ng Service,
Princeton, NJ; JOHN J. MCARDLE, Department of Psychology, University of Southern California; FREDERICK L.
OSWALD, Department of Psychology, Rice University; STEPHEN STARK, Department of Psychology, University
of South Florida; WILLIAM J. STRICKLAND, Human Resources Research Organiza on, Alexandria, VA; CHERIE
CHAUVIN, Study Director
For More Informa on . . . This brief was prepared by the
Board on Behavioral, Cogni ve, and Sensory Sciences
(BBCSS) based on the report Measuring Human CapabilliƟes: An Agenda for Basic Research on the Assessment of
Individual and Group Performance PotenƟal for Military
Accession (Na onal Research Council, 2015). The study
was sponsored by the U.S. Army Research Ins tute for the
Behavioral and Social Sciences (ARI). Any opinions, findings, conclusions, or recommenda ons expressed in this
publica on are those of the authors and do not necessarily reflect those of ARI. Copies of the report are available
from the Na onal Academies Press, 500 Fi h Street, NW,
Washington, DC 20001; (800) 624-6242; http://www.nap.
edu or via the BBCSS web page at http://www.national
academies.org/bbcss.
4
Measuring Human Capabilities
Copyright © 2015 by the Na onal Academy of Sciences.
Permission is granted to reproduce this document in its
en rety, with no addi ons or altera on.