Using a Scored Background Questionnaire for
Selection: Construct and Criterion-related Validity
Frederick P. Morgeson – Michigan State University
Matthew H. Reider – Purdue University
Michael A. Campion – Purdue University
Slides are available at:
http://www.msu.edu/~morgeson/
1
In the Beginning

Collect background information

Resumes and application forms

Huge reduction in number of applicants
2
Our Purpose

Comparatively little research

A gap in the literature

Construct and criterion-related validity

Development and validation of scored background
questionnaire
3
What’s Been Done?

Application forms

Weighted application blanks

Training & Experience Evaluations
4
What We Did

Developed a scored background questionnaire

Keep it simple

Conducted a job analysis

Key job factors assessed
5
The Background Questionnaire

Questions
–
–
–
–
–
Experience in manufacturing and production
Experience with work-related equipment
Experience with physically demanding work
Experience with shift work and overtime
Educational and training background
» Demonstrating ability to learn
» Manufacturing or business courses
» Maintenance or technical courses
– Promotion experiences
– Experience working in a team
– Situations demonstrating creativity or innovation
6
Example Item

What experiences have you had with physically
demanding work (e.g., working with your hands,
standing or sitting for extended periods on the job,
lifting, etc.)?
– Company name, title/job duties, years
– If you have not had previous employment of this type,
have you ever done this type of work in other settings
(e.g., farm, work at home, volunteer work, hobbies,
etc.)?
» Please explain, years
7
How it Was Scored

Supervisors or HR representatives rated the
questions

Used a simple 5-point rating scale

Example scale
(5) Substantial work experience suggesting he/she can
perform physical requirements of jobs.
(3) No relevant work experience, but no indication that
physical requirements would be a problem.
(1) Potential concerns that there may be difficulties
meeting physical requirements.
8
Methods

Setting & Participants
– Steel mini-mill
– 96 incumbents

Study context
– Development of new selection system
– Concurrent validation study
– Different supervisors provided predictor/criteria data
9
Predictor and Criterion Measures

Scored background questionnaire
– 10-item measure, a = .74

Structured Panel Interview
– 14 situational questions; 14 past behavior questions
– Inter-rater reliability = .89; a = .84.

Personal Characteristics Inventory (Barrick & Mount)
– Big 5 Personality


Teamwork-KSA Test (Stevens & Campion)
Job performance
– 10-item measure, a = .97
– Supervisory ratings
– Working safely, working efficiently, etc.
10
Results
Background Questionnaire
Situational Interview
Past Behavior Interview
Teamwork-KSA Test
Conscientiousness
Agreeableness
Emotional Stability
Extroversion
Openness to Experience
Background
Job
Questionnaire Performance
–
.278**
.162
.079
.392**
.259**
.331**
.384**
.279**
.261**
.291**
.151
.185*
.170
.056
.166
.229**
.091
11
Results Summary

What is the background questionnaire measuring?
–
–
–
–

Quality of work experiences
General ability
Dependability
Ability to work well with others
What about predicting job performance?
– Decent validity
– Multiple correlation
12
Conclusions

Good reliability

Related to other predictors

Good validity

Potentially useful screening device
13