Pilot Candidate Selection
Method (PCSM)
Program Status
16 May 2006
Dr. Thomas R. Carretta
Air Force Research Laboratory
Wright-Patterson AFB, OH
Thomas.Carretta@wpafb.af.mil
1
Pilot Candidate Selection Method
• Psychomotor
• Cognitive
• Attitudes
Air Force Officer
Qualifying Test
+
• PCSM is a regressionweighted pilot
aptitude composite
• Operationally
implemented by
USAF in 1993
+
• Cognitive ability
• Aviation-related
knowledge
Basic Attributes Test
Previous Flying
Experience
2
T-37 Outcome by PCSM Quartile
700
64.1% Grad
89.3% Grad
Frequency (N)
600
500
400
Eliminees
Graduates
300
200
100
0
1-25
26-50
51-75
76-99
PCSM Score
N = 1,268
Grad %: 58.5%
69.7%
81.8%
92.1%
3
PCSM Changes
• All three PSCM components are undergoing change
• Air Force Officer Qualifying Test
– July 2005: Form S implemented; dropped 5 of 16
subtests and created new composites
•
Basic Attributes Test
– July 2006: Scheduled to be phased out and
replaced with the Test of Basic Aviation Skills
(TBAS)
•
Flying Experience
– Flight screening program has undergone several
changes since 1993
4
AFOQT (Forms O, P1/P2, &Q1/Q2)
• Paper-and-pencil test
• Approximately 380 items in 16 subtests are combined
into 5 composites:
–
–
–
–
–
•
•
Verbal
Quantitative
Academic Aptitude (Verbal + Quantitative)
Pilot
Navigator/Technical
1978 norms based on active duty population
New form implemented approximately every 8 years
5
AFOQT (Forms O, P1/P2, & Q1/Q2)
Composite
________________________________________
Verbal Quantitative Academic Pilot NavigatorAptitude
Technical
Test
Abbr.
(V)
(Q)
(AA)
(P)
(N-T)
________________________________________________________________________
Verbal Analogies
VA
X
X
X
Arithmetic Reasoning
AR
X
X
X
Reading Comprehension
RC
X
X
Data Interpretation
DI
X
X
X
Word Knowledge
WK
X
X
Math Knowledge
MK
X
X
X
Mechanical Comprehension
MC
X
X
Electrical Maze
EM
X
X
Scale Reading
SR
X
X
Instrument Comprehension
IC
X
Block Counting
BC
X
X
Table Reading
TR
X
X
Aviation Information
AI
X
Rotated Blocks
RB
X
General Science
GS
X
Hidden Figures
HF
X
________________________________________________________________________
Note. Abbr. means abbreviation.
6
AFOQT: What is Measured?
g
Verbal
VA
RC
WK
Math
GS AR
DI
Spatial
MK
SR
MC
Percep.
Speed
Aviation
EM
BC
RB
HF
IC
AI
TR
7
AFOQT: What is Measured?
•
Confirmatory factor
analysis:
–
–
–
Like other cognitive
tests, AFOQT has a
hierarchical structure
Higher-order factor is
g
4%
9%
4%
4%
12%
5 lower-order factors:
•
•
•
•
•
Verbal
67%
Math
Spatial
Aviation Knowledge
Perceptual Speed
g
Verbal
Math
Spatial
Aviation
Per. Spd.
67% of the common
(explained) variance
is due to g
8
AFOQT: What is Predictive?
• Olea & Ree (1994) examined the predictiveness of g,
specific ability, and specific knowledge from the
AFOQT for pilot trainees (N = 1,867 to 3,942)
– Predictors were unrotated principle components from AFOQT
– Training criteria included academic grades, check flight
grades, an overall performance composite, and a
passing/failing score
– g was the best predictor for all pilot training criteria. The
average validity of g was .31
– Specific knowledge (IC & AI tests) showed incremental validity
beyond g of .098
– Specific ability (verbal, math, spatial, perceptual speed)
showed little increment to g, about .02
9
AFOQT Form S
• Goals:
– Reduce test administration time
– Maintain composite score reliability and validity
– Retain previous factor structure (verbal, quantitative, spatial,
aviation knowledge, perceptual speed)
– No adverse impact due to changes in battery
•
Results:
– Dropped 5 subtests reducing administration time from 4.5 to 3
hours (RC, DI, MC, EM, SR)
– Retained reliability and validity of composites as well as factor
structure
– Added Self-Description Inventory (SDI+) to assess personality
factors (resultant test is 3.5 hours)
10
AFOQT Form S
Reliability & Validity Effects
•
Intercorrelations between old and new Composites
Form Q
•
Form R /S
Verbal
.97
.96
Quant
.97
.98
Acad Apt
.98
.98
Pilot
.97
.92
Nav-Tech
.98
.94
Little reliability or correlation loss caused by removing
RC, DI, MC, EM, and SR
• Validity of new Pilot composite versus T-37
performance was comparable to old Pilot composite
11
AFOQT Form S
Composite
______________________________________
Test
Abbr
V
Q
AA
P
N/T
_________________________________________________________________________
Verbal Analogies
VA
X
X
Arithmetic Reasoning
AR
Word Knowledge
WK
Math Knowledge
MK
Instrument Comprehension
IC
Block Counting
BC
Table Reading
TR
X
Aviation Information
AI
X
General Science
GS
Rotated Blocks
RB
Hidden Figures
HF
Self-Description Inventory
SDI+
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
12
AFOQT Form S
Self-Description Inventory (SDI+)
• Personality inventory based on Goldberg (1990)
adjective clusters
– 220 items consisting of trait descriptions (behavioral
statements)
– Captured Big Five personality dimensions and 2 additional
factors – Service Orientation (SO) & Team Orientation (TO)
• SO – organizational commitment
• TO – preferences for working as member of a group or
working alone
– Subsequent factor analysis confirmed Big Five factors and
Service Orientation factor
• Team-Orientation factor not separate from other factors
•
Proposed to be used for career counseling
13
Basic Attributes Test (BAT)
• Basic Attributes Test (BAT)
measures:
– Psychomotor
– Cognitive
– Personality/Attitudes
• In operational use since 1993
– 386-based computer
– Test hardware can no
longer be maintained
– To be replaced by the Test
of Basic Aviation Skills
(TBAS)
14
Test of Basic Aviation Skills (TBAS)
• Test of Basic Aviation Skills
(TBAS) measures:
–
–
–
–
Psychomotor
Cognitive (spatial)
Multi-task performance
Memory
• Unlike BAT, several TBAS
subtests are related to one
another
• Scheduled for operational
implementation in July 2006
15
TBAS Status
• Preliminary TBAS-based PCSM scoring algorithm completed
– A regression-weighted composite was developed using
AFOQT, flying experience, & TBAS scores
– TBAS scoring algorithm designed to counter test-taking
strategies (e.g., concentrate on part of multiple-task test)
– Predictive validity is comparable to BAT-based PCSM
equation
• Ongoing tasks
– Development of test materials (e.g., operator’s manual,
examinee information pamphlet)
– Compliance with professional test standards (e.g., APA)
– Deployment of test equipment
– Development of long-term sustainment plan
16
Flying Experience
• PCSM Flying Hour Code is an unequal interval scale based on
examination of relation of previous flying hours with UPT
performance
• Code categories:
–
–
–
–
–
–
–
–
–
–
0: 0 flying hours
1: 1 to 5 flying hours
2: 6 to 10 flying hours
3: 11 to 20 flying hours
4: 21 to 40 flying hours
5: 41 to 60 flying hours
6: 61 to 80 flying hours
7: 81 to 100 flying hours
8: 101 to 200 flying hours
9: 201 and Up flying hours
17
Flying Experience
• Prior to 1994: Flight Screening Program
–
–
15 to 20 hours in T-41
Conducted by USAF at USAFA and Hondo, TX
• 1994-1997: Enhanced Flight Screening Program
–
–
–
21 hours in T-3 (Slingsby Firefly) & included more aerobatic training
Conducted by USAF at USAFA and Hondo, TX
Discontinued following several uncommanded engine stoppages
• 1997-1998: No flight screening program
• 1998: Introductory Flight Training Program
–
–
•
Syllabus expanded to 40, then 50 hours
Taught at Civilian training schools
Present: Introductory Flight Screening Program
–
25 hours conducted by contractor instructors
18
Future Directions
• TBAS-based PCSM post-implementation S&T issues
– Subgroup differences
– Test-retest performance
– Extension of PCSM to other career fields (e.g., UAV operators)
•
•
Examine utility of AFOQT SDI+ for career counseling
Collaboration with the US Navy
– TBAS and iTBAS (internet TBAS) comparability
– Examine relations between USAF and USN officer
commissioning & aircrew selection tests
• Development of common test battery
– Computer-administered AFOQT
19
Questions?
20
AFOQT
1. Verbal Analogies
DIRECTIONS: This part of the test measures your ability to reason and
see relationships between words. You are to decide the answer that best
completes the analogy developed at the beginning of each question.
1. FINGER is to HAND as TOOTH is to _____.
a. tongue b. lips c. nose d. mouth e. molar
2. RAQUET is to COURT as __________.
a. tractor is to field b. blossom is to bloom c. stalk is to prey
d. plan is to strategy e. moon is to planet
3. FLY is to AIRPLANE as __________.
a. drive is to stake b. skate is to slide c. push is to fall
d. swim is to float e. rod is to hook
21
AFOQT
2. Arithmetic Reasoning
DIRECTIONS: This part of the test measures mathematical reasoning. It
is concerned with your ability to arrive at solutions to problems. Each
problem is followed by 5 possible answers. Decide which of the 5 answers
is most nearly correct.
1. If an aircraft travels at 564 miles per hour, how many miles did the
aircraft fly in 900 seconds?
a. 141 b. 226 c. 300 d. 451 e. 846
2. A typist took 3 typing tests. The average typing speed on these tests
was 48 words per minute (WPM). If the typists speed on 2 of these tests
was 52 WPM, what was the speed on the third test in WPM?
a. 38 b. 40 c. 42 d. 44 e. 46
22
AFOQT
3. Word Knowledge
DIRECTIONS: This part of the test measures verbal comprehension
involving your ability to understand written language. For each question,
choose the answer that means the same as the capitalized word.
1. CRIMSON
a. crisp b. neatly dressed c. reddish d. colorful e. lively
2. CEASE
a. start b. change c. continue d. stop e. fold
3. FORTNIGHT
a. 2 days b. 1 week c. 2 weeks d. 1 month e. 2 months
4. TERSE
a. pointed b. trivial c. oral d. lengthy e. raggedy
23
AFOQT
4. Math Knowledge
DIRECTIONS: This part of the test measures your ability to use learned
mathematical relationships. Each problem is followed by 5 possible
answers. Decide which one of them is most nearly correct.
1. The first digit of the square root of 59463 is:
a. 1 b. 2 c. 3 d. 4 e. 5
2. The distance in miles around a circular course with a radius of 35
miles is (use  = 22/ 7):
a. 110 b. 156 c. 220 d. 440 e. 880
3. Solve for x: 2x/ 7 = 2x2
a. 1/ 7 b. 2/ 7 c. 2 d. 7 e. 14
24
AFOQT
5. Instrument Comprehension
DIRECTIONS: This test measures your ability to determine the position of
an airplane in flight from reading instruments showing its compass
heading, amount of climb or dive, and degree of bank to right or left.
25
AFOQT
6. Block Counting
DIRECTIONS: This is a test of your ability to “see into” a 3-dimensional
pile of blocks and determine how many pieces are touched by certain
numbered blocks. All of the blocks in a pile are the same size and shape.
26
AFOQT
7. Table Reading
DIRECTIONS: This test measures your ability to read tables quickly and
accurately. In the table below, the X values are indicated at the top and the
Y values at the left. In this test, you are to find the table entry that
corresponds to a given set of row (X) and column (Y) values.
1. For X = 0 and Y = -1:
a. 29 b. 32 c. 33 d. 34 e. 35
2. For X = -3 and Y = -3:
a. 22 b. 23 c. 28 d. 29 e. 31
3. For X = -1 and Y = +2:
a. 25 b. 27 c. 29 d. 30 e. 31
4. For X = +3 and Y = 0:
a. 30 b. 34 c. 35 d. 37 e. 39
27
AFOQT
8. Aviation Information
DIRECTIONS: This part of the test measures your aviation knowledge.
Decide which of the choices best answers the question or completes the
statement.
1.
If the elevator tabs on a plane are lowered, the plane will tend to:
a. nose up b. nose down c. pitch fore and aft d. go into a slow roll
e. wing over
2.
The pilot always advances the throttle during a:
a. nose dive b. landing c. turn d. spin e. climb
3.
The slipstream of an airplane will have the least effect on the plane’s direction
of flight when the plane’s:
a. stick is moved forward b. stick is moved backward c. stick is moved
sideways d. rudder bar is pressed with the left foot e. rudder bar is pressed
28
with the right foot
AFOQT
9. General Science
DIRECTIONS: This part of the test measures your knowledge in the area of
science. Decide which one of the choices best answers the question or
completes the statement.
1. An eclipse of the sun throws the shadow:
a. moon on the earth b. earth on the sun c. sun on the earth
d. earth on the moon e. moon on the sun
2. Lack of iodine is often related to which of the following diseases?
a. Beriberi b. Scurvy c. Rickets d. Goiter e. Asthma
3. After adding salt to water, the freezing point of the water is:
a. variable b. inverted c. unchanged d. raised e. lowered
29
AFOQT
10. Rotated Blocks
DIRECTIONS: This test measures your ability to visualize and manipulate
objects in space. In each item, you will be shown a picture of a block.
The problem is to find a second block that is identical to the first.
30
AFOQT
11. Hidden Figures
DIRECTIONS: This part of the test measures your ability to locate a simple
geometric figure embedded in a complex drawing. Your task is to
determine which lettered figure (A, B, C, D, or E) is contained in each
numbered drawing.
31
AFOQT
12. Self Description Inventory
DIRECTIONS: This inventory measures personal traits and attitudes. The
inventory consists of a list of statements. The task is to read each
statement carefully and decide how well each one describes you.
Look at the sample statement below.
S1. I enjoy reading poetry.
Decide if statement S1 is characteristic of you and indicate your
agreement using the scale below.
A – Strongly Disagree
B – Moderately Disagree
C – Neither Agree nor Disagree
D – Moderately Agree
E – Strongly Agree
32
BAT
1. Two-Hand Coordination
• Rotary pursuit tracking
task
• Target moves in a fixed,
elliptical pattern at varying
speeds
• Task is to keep gunsight
on target using control
sticks
• Test length ~10 minutes.
33
BAT
2. Complex Coordination
• Compensatory tracking
task
• Dual-axis control stick is
used to keep the small
cross centered on the
large cross
• Single-axis control stick
is used to keep the
rudder bar centered at
the bottom of the screen
• Test length ~10 minutes.
34
BAT
4. Time Sharing
• Assesses control precision
and reaction time
• Combines a compensatory
tracking task with a target
identification (digit
canceling) task
• Test length ~30 minutes
35
BAT
3. Item Recognition
• Assesses short-term memory
• A string of 1 to 6 digits is
presented quickly, removed,
and replaced by a single digit
• Examinee must determine if
single digit presented was
also present in the original
string
• Test length ~20 minutes
36
BAT
5. Activities Interest Inventory
• Assesses attitudes toward risktaking
• Examinee is presented with
activity pair and asked to
choose between the activities
• Activity pairs differ in level of
risk
• Test length ~10 minutes
37
TBAS
1. UAV Directional Orientation
• Assesses spatial orientation
• Examinee must determine position of target relative to the
UAV
– The "tracker map” shows the examinee the location and
heading of the UAV
• Example “select the North parking lot”
38
TBAS
2 & 3. 3-Digit and 5-Digit Listening
• A string of letters and numbers is presented over
headphones
•
Task is to squeeze the trigger when any of three (3) or
five (5) pre-specified target numbers are presented
•
Test length = 60 seconds
39
TBAS
4. Horizontal Tracking
• Compensatory
tracking task
• Use rudder pedals to
keep the yellow box
over a moving
airplane target
• Speed of the moving
target increases
during test
• Test length = 60
seconds
40
TBAS
5.Airplane Tracking
• Compensatory tracking
task
• Dual-axis joystick is used
to keep the yellow sight
over a moving airplane
target
• Test length = 60 seconds
41
TBAS
6. Airplane & Horizontal Tracking
• Compensatory tracking task
• Combines Airplane Tracking Task and Horizontal Tracking Task
• Time length = 120 seconds
42
TBAS
7 & 8. Airplane & Horizontal Tracking &
Listening
•
•
Compensatory tracking task and listening task
•
Time length = 180 seconds
Combines Airplane Tracking Task and Horizontal Tracking Task with
either 3-Digit Listening or 5-Digit Listening tasks
43
TBAS
9. Emergency Scenario
• Compensatory tracking
& memorization
• Target digit listening is
replaced with
emergency scenarios
task
– Examinee must
resolve “emergency
situations” by
entering code
memorized earlier
from test
instructions
44