Industrial
Reading Test
Technical Manual
Copyright © 2006 NCS Pearson, Inc.
All rights reserved. No part of this publication may be reproduced or transmitted in any form or by
any means, electronic or mechanical, including photocopy, recording, or any information storage
and retrieval system, without permission in writing from the publisher.
ii
Introduction
1
Description of the Test
Uses of the Test
Development of the Test
1
1
1
Directions for Administration and Scoring
Testing Conditions
Preparing for the Administration
Administering the Test
Scoring and Reporting
Test Security
Accommodating Examinees with Disabilities
3
3
3
3
4
4
4
Norms
5
Evidence of Reliability
13
Evidence of Validity
14
Correlations with Other Tests
15
References
27
iii
List of Tables
IRT Scores and Corresponding Percentile Ranks
6
IRT Means and Standard Deviations for Vocational High School Students by Programs
9
IRT Means and Standard Deviations for Various Samples of Adults in Schools and Industry
12
IRT Alternate-Form Reliability Coefficients and Standard Errors of Measurement
13
IRT Split-Half Reliability Coefficients and Standard Errors of Measurement
14
Correlation Coefficients Between the IRT and Grades of High School Students in
Vocational Training Programs
16
Correlation Coefficients Between the IRT and Final Grades of High School Students in
Specific Vocational Training Programs
18
Correlation Coefficients Between the IRT and Criteria for Students in
Special Remedial and Vocational Programs
22
Correlation Coefficients Between the IRT and Criteria for Industrial Groups
23
Correlation Coefficients Between the IRT and Other Tests
24
iv
Introduction
Description of the Test
The Industrial Reading Test (IRT) is a measure of reading ability developed for the use
in industry and vocational schools. The passages on the test deal with such topics as
basic principles of first aid, kinds of clothing and protective devices used by technical
workers, the importance of blueprints in manufacturing, the metric system, and the like.
Two alternate forms of the test are available. The sale of Form A is restricted to
business and industry while Form B is available in paper-and-pencil format and can be
purchased by both schools and businesses. Each form has 9 passages and 38 items.
Some of the passages and items on the IRT are relatively easy while others are more
difficult. The test is designed to be within the reading ability of the typical high school
vocational student or an older person with a high school education or its equivalent. The
test is not recommended for use below the 9th grade.
The IRT is intended to be a power test of reading comprehension rather than a measure
of reading speed. Most examinees will find the time limit of 40 minutes adequate for
completing the test. The IRT yields one score—the total number of items answered
correctly. The paper-and-pencil version can be scored either by hand or by machine,
and the online version of Form A is automatically scored.
Uses of the Test
The IRT measures an individual’s ability to comprehend written technical materials. The
test can be used to determine the ability of applicants for jobs in industrial settings, or
trainees in technical or vocational training programs, to read instructional and jobrelated materials.
The IRT is designed so that applicants or trainees will find the material relevant and
meaningful. Good performance on the test is not dependent upon previous knowledge
of the subject matter. The test simply shows whether or not an individual has the
necessary reading ability to perform adequately on the job or to make satisfactory
progress in technical training. Companies that provide remedial reading instruction can
use the IRT to identify employees or trainees who would benefit from such instruction.
Development of the Test
The idea of developing an Industrial Reading Test grew out of discussions between staff
members of Pearson TalentLens and managerial personnel in various industries
that train and employ people for skilled technical jobs.
Many of the applicants selected by these organizations find the training programs
difficult. The problem appears to be not so much one of poor mechanical skills or lack of
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
1
motor coordination, but rather one of inadequate reading ability. Some training
programs are rigorous, particularly three- and four-year apprentice programs in
construction, mechanical maintenance, electrical installation and repair, and the like.
Individuals selected for these programs must be able to read and master difficult
technical material in areas such as the design and interpretation of blueprints, basic and
applied physics, and industrial electronics. The numerous rules and regulations set forth
in company safety manuals must also be read and understood. A readability analysis of
representative textbooks and training manuals used in these programs indicated that
they were written at a 9th or 10th grade level. An appropriate, professionally developed
reading test can provide useful information about the reading level of applicants for such
programs.
Development of the Industrial Reading Test began with careful consideration of the
characteristics of a suitable instrument. First, it was decided that the IRT should be
relatively short, and easy to administer and score. A second consideration related to the
difficulty level of the passages and items; the reading level of the IRT should be
consistent with that of written material the examinees will need to read and comprehend
in the course of their training programs. A third consideration was that the content of the
test should appear relevant and reasonable. The passages should look like, and read
like, excerpts from a training manual or company memorandum.
The Occupational Outlook Handbook (U.S. Dept. of Labor, 1978) and Dictionary of
Occupational Titles (U.S. Dept. of Labor, 1977), job descriptions, education and training
requirements, and other relevant materials were used to determine the content areas,
specific subject matter, and the appropriate level of reading difficulty for each topic in
the test. Textbooks and manuals used in training programs for technical jobs were also
used in determining content and reading difficulty.
Each passage written for the IRT, as well as a representative sample of the reference
material on which it was based, was evaluated for reading difficulty. Readability
estimates were obtained by means of the Dale-Chall formula for estimating readability
(Dale & Chall, 1948; Powers, Sumner, & Kearl, 1958). The analyses indicated that all
but one of the passages had Dale-Chall readability indexes at the 7th–8th grade level or
the 9th–10th grade level. One passage (Form B, page 11) had a Dale-Chall readability
index at the 11th–12th grade level.
A pool of items was then written for each passage, and the passages and items were
tried out on samples of employees and students. Although the IRT was originally
designed for industrial use, the preliminary item statistics obtained from a sample of
vocational high school students suggested that the test was also appropriate, in content
and difficulty level, for vocational students. Consequently, normative and validity data
were collected for vocational students as well as for employees in industry.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
2
Directions for Administration and Scoring
The computer-based Industrial Reading Test is administered through the
TalentLens platform, an Internet-based testing system designed by Pearson TalentLens
for the administration, scoring, and reporting of professional assessments.
Instructions for administrators on how to order and access the test online
are provided at TalentLens.com. Instructions for accessing the IRT interpretive
reports are provided on the website. After a candidate has taken the IRT on the
TalentLens platform, the test administrator can review the candidate’s results in an
interpretive report.
Testing Conditions
It is important to ensure that the test is administered in a quiet, well-lit room. The
following conditions are necessary for accurate scores and for maintaining the
cooperation of the examinee: good lighting, comfortable seating, adequate desk or table
space, comfortable positioning of the computer screen, keyboard, and mouse, and
freedom from noise and other distractions.
Preparing for Administration
Being thoroughly prepared before the examinee’s arrival will result in a more efficient
online administration session. It is recommended for test administrators to take the
computer-based IRT prior to administering the test, being sure to comply with the
directions and the time requirement. Examinees will not need pencils or scratch paper
for this computer-based test. In addition, examinees should not have access to any
reference materials (e.g., dictionaries or calculators).
Administering the Test
After the initial instruction screen for the IRT has been accessed and the examinee is
seated at the computer, say,
The on-screen directions will take you through the entire process that begins with
some demographic questions. After you have completed these questions, the test
will begin. You will have 40 minutes to work on this test. The test ends with a few
additional demographic questions. Do you have any questions before starting the
test?
Answer any questions and say, Please begin the test.
After the examinee clicks Start Your Test, the sample passage and sample items
appear, followed by the actual test passages and items. During the test, examinees may
skip items and return to them later. The examinee also may review test items at the end
of the test. Examinees have 40 minutes to complete the test.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
3
If an examinee’s computer develops technical problems during testing, the test
administrator should move the examinee to another suitable computer location. If the
technical problems cannot be solved by moving to another computer location, the
administrator should contact Pearson TalentLens Technical Support for
assistance. The contact information, including phone and fax numbers, can be found at
the TalentLens.com website.
Scoring and Reporting
Scoring is automatic, and the report is available a few seconds after the test is
completed. Adobe® Acrobat Reader® is necessary to open the report. The test administrator
may view, print, or save his or her candidate’s report.
Test Security
Industrial Reading Test scores are confidential and should be stored in a secure
location accessible only to authorized individuals. It is unethical and poor test practice to
allow individuals, who do not have a legitimate need for the information, access to testscores. Storing test scores in a locked cabinet or password protected file that can only
be accessed by designated test administrators will help ensure their security. The
security of testing materials (e.g., access to online tests) and protection of copyright
must also be maintained by authorized individuals. Avoid disclosure of test access
information such as usernames or passwords and only administer the test in proctored
environments. All the computer stations used in administering the IRT must be in
locations that can be easily supervised with the same level of security as the paper-andpencil administration.
Accommodating Examinees with Disability
The test administrator should provide reasonable accommodations to enable
candidates with special needs to take the test comfortably. Reasonable
accommodations may include, but are not limited to, modifications to the test
environment (e.g., desk height) and medium (e.g., having a reader read questions to the
examinee, or increasing the font size of questions) (Society for Industrial and
Organizational Psychology, 2003). In situations where an examinee’s disability is not
likely to impair his or her job performance, but may hinder the examinee’s performance
on the IRT, the organization may want to consider waiving the test or de-emphasizing
the score in lieu of other application criteria. Interpretive data as to whether scores on
the IRT are comparable for examinees who are provided reasonable accommodations
are not available at this time due to the small number of examinees who have requested
such accommodations.
If, due to some particular impairment, a candidate cannot take the computeradministered test but can take the test on paper, the administrator could provide
reasonable accommodation for the candidate to take the test on paper, and then have
the candidate’s responses entered into the computer system. The Americans with
Disabilities Act (ADA) of 1990 requires an employer to make reasonable
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
4
accommodations for a qualified applicant with a known disability, which would not cause
an “undue hardship” to the operation of the employer’s business.
Norms
The value of a test is based on the extent to which the normative data it is based upon
is accurate and relevant to the user’s own population. Locally developed norms are the
most useful because they pertain specifically to the group that is of immediate concern.
Where such information is not available, the user can be guided by the norms published
in the test manual for similar groups. The norms tables are arranged according to the
nature of the population tested: vocational high school students; adults in remedial,
technical, or vocational training programs; and industrial applicants, apprentices, and
employees.
Data for the high school samples are grouped according to vocational programs. for
students enrolled in comparable programs, average scores did not improve from grade
10 to grade 12. Similarly, the average scores of males and females enrolled in
comparable programs or applying for similar kinds of work were not significantly
different.
Table 1 presents Industrial Reading Test percentile ranks for groups of at least 100
cases. The percentile norms have been devised to encourage realistic interpretation of
test scores.
The norms in Table 1 include three samples from vocational high schools grades 10–
12, three adult student samples, and four industrial samples. The high school students
are enrolled in various programs within three broad occupation categories: service
occupations, machine trades, and structural work. The samples within these three
groups are quite diverse.
Each percentile in the table represents a band or zone of ability for which the indicated
percentile is approximately the midpoint. In most cases, each band is five percentile
units; however, the zones differ somewhat at the extremes. Thus, a percentile of 5
includes 4 through 7; 3 includes 2 and 3; 1 stands for the first percentile only. Similarly,
the 95th percentile includes 93 through 96; 97 includes 97 and 98; 99 stands for only
the 99th percentile.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
5
Table 1
IRT Raw Scores and Corresponding Percentile Ranks
Vocational High School Students
In Programs for
Service
a
a
In Programs for
Machine Trades
b
Adult Students
In Programs for
Structural Work
Percentile
Occupations
Occupations
Occupations
99
36–38
37–38
97
95
34–35
33
90
85
80
c
In Remedial
Mathematics at a
Vocational
Institute in New
d
At a Skills Center
e
At a Vocational
Institute in
f
Mexico
in Tennessee
Florida
Percentile
37–38
37–38
36–38
37–38
99
35–36
34
35–36
34
35–36
33–34
34–35
32–33
36
35
97
95
32
31
30
33
32
31
32–33
31
—
32
31
30
31
29–30
28
—
34
—
90
85
80
75
70
65
60
29
28
27
26
—
30
29
—
30
29
28
27
29
28
27
26
27
26
25
—
33
32
—
31
75
70
65
60
55
25
28
26
25
24
30
55
50
24
27
25
24
23
29
50
45
23
26
24
23
22
—
45
40
35
30
22
21
20
25
24
23
23
22
21
22
21
20
21
20
19
28
—
27
40
35
30
25
19
22
19–20
19
17–18
25–26
25
20
15
18
16–17
21
19–20
17–18
15–16
17–18
15–16
15–16
13–14
22–24
20–21
20
15
10
5
3
13–15
11–12
9–10
15–18
13–14
11–12
13–14
11–12
9–10
13–14
10–12
6–9
12
10–11
8–9
18–19
14–17
12–13
10
5
3
1
0–8
0–10
0–8
0–5
0–7
0–11
N
Mean
SD
362
23.9
6.9
437
26.2
6.2
467
24.5
7.0
193
24.1
7.2
188
22.4
7.0
140
28.4
6.3
1
N
Mean
SD
Includes students in grades 10–12 in food service and cosmetology programs at 4 schools (3 in PA, 1 in NY).
b
Includes students in grades 10–12 in machine shop; auto mechanics; diesel mechanics; engine rebuilding; and heating, ventilation, and air conditioning programs at 12
schools (7 in PA, 2 in NY, 1 in MA, 1 in TN, 1 in W VA).
c
Includes students in grades 10–12 in auto body, welding, electricity or electronics, and carpentry programs at 12 schools (4 in NY, 4 in PA, 1 in FL, 1 in MA, 1 in TN, 1 in
W VA).
d
e
f
Includes 114 males and 79 females; 60% from minority (Black/African-American and Hispanic/Latino[a]) groups.
Includes 99 males and 89 females; 100% from minority (Black/African-American and Hispanic/Latino[a]) groups.
Mostly male; mostly White (non-Hispanic).
Copyright © 2006 by NCS Pearson, Inc.
All rights reserved.
6
Table 1
IRT Raw Scores and Corresponding Percentile Ranks
(continued)
Applicants for
Apprentice Training
at a Steel Plant in
g
h
I
j
Percentile
Illinois
99
97
95
90
85
80
g
Millwright and Motor
Inspector
Apprentices at a Steel
h
Utility Operator and
Technician Trainees
i
Service Technician
Trainees at a
Technical Training
j
Plant in Illinois
at a Southern Utility
Center in Virginia
38
37
36
35
34
—
37–38
—
36
—
—
35
37–38
—
—
36
—
—
38
—
37
—
—
36
99
97
95
90
85
80
75
70
65
60
55
33
32
31
30
29
34
—
33
—
—
—
35
—
—
—
—
—
35
—
—
75
70
65
60
55
50
28
32
34
34
50
45
40
35
30
25
—
27
26
24–25
22–23
—
—
31
30
29
—
33
—
—
32
—
33
—
—
32
45
40
35
30
25
20
15
10
5
3
1
N
Mean
SD
20–21
17–19
14–16
12–13
10–11
0–9
145
27.3
7.4
28
26–27
24–25
21–23
14–20
0–13
115
31.3
5.1
—
31
30
28–29
27
0–26
100
33.9
2.5
—
31
29–30
25–28
20–24
0–19
380
33.6
3.9
20
15
10
5
3
1
N
Mean
SD
Mostly male; mostly minority.
Mostly male; includes 66 White (non-Hispanic) and 49 minority (Black/African-American and Hispanic/Latino[a]).
Mostly male; includes 85 White (non-Hispanic) and 15 minority (Black/African-American and Hispanic/Latino[a]).
Mostly male; mostly White (non-Hispanic).
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
7
Percentile
Table 2 presents means and standard deviations for additional groups of varying size.
Some of the student groups are subgroups of the large samples that were the basis for
the percentile norms in Table 1. In Table 2, the data are reorganized to show
comparative performance on the IRT by specific vocational program rather than by the
broad occupational categories used in Table 1.
The means and standard deviations in Table 2 are shown, when appropriate, for
combined groups of students in similar programs at different schools. In order for these
subsamples to be combined, the groups had to be closely comparable in terms of
program, grade level(s), IRT scores (mean scores no more than 3 points apart), and sex
and racial composition.
Table 3 presents means and standard deviations for various adult student and industrial
samples of less than 100 cases. The data for adult students and trainees are organized
according to type of program. The data for industrial groups are listed separately for
each organization and job category.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
8
Table 2
IRT Means and Standard Deviations for Vocational High School Students
by Programs
Grade
Level
Sex
Race/Ethnicitya
N
Mean
SD
10–12
Mostly
Male
Mostly white
40
31.8
4.6
Civil construction and drafting programs in
Pennsylvania
11–12
Mostly
male
Mostly white
66
30.1
4.5
Medicine and health programs in New York
and Pennsylvania
11–12
Mostly
female
Mostly white
59
30.7
3.8
Medical lab program in Pennsylvania
10–12
Mostly
female
Not reported
27
30.5
3.8
Medical and dental programs in
Pennsylvania
10–11
Female
Not reported
31
21.0
6.0
Graphic arts program in Pennsylvania
11–12
Male
White
24
25.2
6.1
Commercial art program in New York
10–12
8 male,
25 female
Not reported
33
24.4
7.2
Commercial art program in Pennsylvania
11–12
15 male,
10 female
Not reported
25
22.0
8.5
10–12
15 male,
16 female
Not reported
31
30.3
4.7
10
Mostly
female
13 white,
64 minority
77
17.7
6.0
Data processing program in Pennsylvania
11–12
21 male,
50 female
Mostly white
71
28.8
4.9
Data processing program in Pennsylvania
10–12
21 male,
51 female
Not reported
72
25.3
5.5
Accounting and data processing programs
in Pennsylvania
11–12
Mostly
female
Mostly minority
30
20.7
6.9
Marketing programs in Pennsylvania
11–12
10 male,
65 female
Mostly white
75
25.2
5.5
Programs for Service Occupations
Food service programs in Pennsylvania
and West Virginia
10–12
7 male,
18 female
14 white
11 minority
25
22.1
7.7
10–12
64 male,
60 female
Not reported
124
20.8
7.1
Cosmetology program in Pennsylvania
12
Female
White
32
30.4
3.0
Cosmetology programs in Pennsylvania
11
Female
Mostly white
111
28.1
4.9
Program
Programs in Professional, Technical, and
Managerial Work
Electronic technology programs in New
York and West Virginia
Clerical and Sales Programs
Typing and shorthand classes in Florida
Business education program in
Pennsylvania
Food preparation and service programs in
New York and Pennsylvania
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
9
Table 2 IRT Means and Standard Deviations for Vocational High School Students by Programs (continued)
Grade
Program
Sex
Race/Ethnicitya
Mean
N
SD
Level
Programs for Service Occupations
(continued)
Cosmetology programs in New York
and Pennsylvania
10–12
Female
Not reported
127
23.3
6.6
10–11
Female
Mostly minority
25
16.3
5.8
10–12
32 male,
6 female
Not reported
38
23.8
7.5
10–11
Male
Not reported
54
22.7
5.5
Machine shop and mill and cabinet
programs in West Virginia
11–12
Male
White
34
22.3
6.9
Machine shop and metal fabrication
programs in Massachusetts
10
Male
Not reported
40
25.2
5.9
10–12
Male
Mostly white
24
25.0
7.4
Aircraft mechanics program in
Pennsylvania
12
Mostly male
Mostly white
40
30.7
4.4
Diesel and engine repair programs in
Pennsylvania and Tennessee
12
Male
Mostly white
27
27.3
5.4
Auto mechanics programs in New
York, Pennsylvania, Tennessee, and
West Virginia
10–12
Male
Mostly white
205
26.6
6.1
Heating and ventilation programs in
Pennsylvania
10–12
Male
Mostly white
35
25.6
5.4
Air conditioning and auto mechanics
programs in New York
11–12
Male
Not reported
121
25.3
7.1
Agricultural mechanics programs in
New York
11–12
Male
White
31
24.5
7.4
Auto mechanics programs in
Massachusetts
10–12
Male
Not reported
46
24.1
6.5
10
Male
Not reported
119
23.6
6.7
10–12
Male
Not reported
29
28.1
4.6
Cosmetology program in
Pennsylvania
Agricultural, Fishing, and Forestry
Programs
Conservation program in New York
Machine Trades Programs
Machine shop programs in
Pennsylvania
General metals trades program in
Tennessee
Various mechanical repairing
programs in Massachusetts and
Pennsylvania
Bench Work Programs
Electrical shop program in
Massachusetts
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
10
Table 2 IRT Means and Standard Deviations for Vocational High School Students by Programs (continued)
Race/Ethnicitya
N
Mean
SD
10–12
Mostly
male
39 white,
11 minority
50
22.5
7.0
10
Male
Not reported
28
26.1
4.2
Auto body programs in New York,
Pennsylvania, and West Virginia
10–12
Male
Mostly white
107
21.4
7.1
Welding programs in New York,
Pennsylvania, and West Virginia
10–12
Male
Mostly white
121
24.1
7.4
Electromechanics and electronics
programs in Massachusetts and
Pennsylvania
10–12
Male
Mostly white
89
29.6
5.2
Electricity and electronics classes in
New York, Pennsylvania, Tennessee, and
West Virginia
10–12
Male
Mostly white
114
28.1
5.9
Basic electricity and electronics courses in
New York and Pennsylvania
10–11
Male
Not reported
91
27.9
5.6
11
Male
Mostly white
29
27.3
5.1
Trade electricity program in New York
10–12
Male
Not reported
25
25.9
6.1
Carpentry programs in New York and
Pennsylvania
10–12
Male
White
70
27.7
5.7
Masonry programs in Florida,
Massachusetts, Pennsylvania, and West
Virginia
11–12
Male
Not reported
74
25.1
6.3
Carpentry programs in Florida,
Massachusetts, New York, and
Pennsylvania
11–12
Male
Mostly white
125
24.0
6.0
Carpentry and masonry programs in New
York and Pennsylvania
10
Male
Not reported
69
21.8
7.4
Plumbing programs in Massachusetts and
Pennsylvania
10–11
Male
Not reported
55
21.2
6.8
Carpentry and building maintenance
programs in Pennsylvania and West
Virginia
10–11
Male
17 white,
9 minority
26
17.7
8.0
Bench Work Programs (continued)
Radio/TV and small appliance repair
programs in Pennsylvania, Tennessee, and
West Virginia
Programs in Structural Work
Auto body programs in Massachusetts and
Pennsylvania
Industrial electricity programs in
Pennsylvania and West Virginia
a
Grade
Level
Sex
Program
Minority includes Black/African-American and Hispanic/Latino(a)
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
11
Table 3 IRT Means and Standard Deviations for Various Samples of Adults in Schools and Industry
Sex
Race/Ethnicitya
N
Mean
SD
7 male
30 female
30 white,
7 minority
37
27.2
5.8
Adults in clerical programs at a skills center in Tennessee
Mostly female
Minority
35
23.8
5.7
Adults in mechanical repairing programs at skills centers in
New York and Tennessee
Mostly male
13 white
24
24.3
6.9
Adults in an auto shop program at a vocational center in
California
Mostly male
White
23
21.1
8.6
Male
Minority
29
17.8
8.3
Adults in audio, video, and small appliance repair programs
at a skills center in Tennessee
Mostly Male
Minority
29
24.1
6.6
Adults in auto body and welding programs at a skills center
in Tennessee
Mostly male
Minority
25
22.9
6.6
Adults in electricity and electronics programs at skills
centers in Florida and New York
Mostly male
Mostly white
36
32.6
3.5
29 male
Mostly white
64
26.4
7.0
70 male
70 white,
83
29.5
5.4
27
28.7
7.6
Sample
Adults in a clerical program at a vocational institute in
Florida
Adults in CETA (Comprehensive Employment Training Act)
programs at a skills center in New York
Technical workers at an oil refinery in the South
11 minority
35 female
13 female
13 minority
Mill repairmen and motor inspectors at a steel plant in
Illinois
Male
21 white,
Apprentices at a manufacturing company in the South
Male
Not reported
30
28.4
5.5
Apprentice applicants at a steel plant in Indiana
Male
42 white,
72
28.0
6.3
56
28.0
5.7
68
23.1
8.1
6 minority
30 minority
Machine operators at a cereal company in the Midwest
Male
54 white,
2 minority
Technical trainees at a utility in the Northeast
a
Mostly male
Minority includes Black/African-American and Hispanic/Latino(a)
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
12
Mostly minority
Evidence of Reliability
Reliability or consistency of measurement is an important characteristic of any test. The
reliability of a test refers to the accuracy, consistency, and stability of test scores across
situations (Anastasi & Urbina, 1997). A reliable test has a relatively small measurement
error, which is the difference between the hypothetical true score and an individual’s
obtained test score. Evidence of reliability is often demonstrated as a correlation
coefficient or the standard error of measurement (SEM).
Evidence of reliability is demonstrated statistically in several ways. Alternate-form
reliability provides an estimate of the degree of relationship between test scores
obtained on alternate forms of the test that have been administered on different
occasions to the same individuals. Split-half reliability provides an estimate of the test’s
internal consistency. Reliability coefficients can vary between .00 and 1.00. The closer
the coefficient is to 1, the greater the test’s reliability.
The standard error of measurement (SEM) provides a different type of information. This
statistic has the advantage of being relatively independent of the spread or variability of
groups. It reflects the extent to which a given individual’s test scores would be expected
to vary if he or she were administered the test many times (discounting the effects of
fatigue and practice). The SEM represents the difference between an individual’s
obtained test score and the hypothetical true score. For example, a standard error of
measurement of 3 points indicates that, in about two out of three cases, an individual’s
obtained score may be expected to lie within 3 points ( + or –) of the “true” score.
Table 4 presents alternate-form reliability coefficients and standard errors of
measurement for the Industrial Reading Test, derived from the counter-balanced
administration of Forms A and B to a sample of 61 adult students at a vocational skills
center in Tennessee. The time interval between testings was three months.
Table 4
IRT Alternate-Form Reliability Coefficients and Standard Errors of Measurement
Sample
N
r12
Mean
Form A
SD
SEM
Mean
Form B
SD
SEM
Adult students at a
skills center in
Tennessee
61
.75
22.3
7.3
3.7
21.9
7.6
3.8
Note. Form A and Form B were administered in counterbalanced order. The time interval between testings was three months.
Table 5 presents split-half reliability coefficients and standard errors of measurement for
students at a vocational high school in Massachusetts and for applicants for craft
apprenticeship programs at a steel plant in Illinois. In computing the split-half reliability
coefficients, the halves of the test were formed by assigning alternate passages, rather
than alternate items, to each of the halves. This is a more rigorous check of the test’s
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
13
reliability. The reliability coefficients are generally high for a power test of reading
comprehension.
Table 5
IRT Split-Half Reliability Coefficients and Standard Errors of Measurement
Sample
N
Mean
Form A
SD
ra
SEM
N
Mean
Form B
SD
ra
SEM
Students at a vocational high school in
Massachusetts
Grade 9
111
23.5
7.0
.84
2.8
103
21.8
7.2
.83
3.0
Grade 10
106
25.6
6.1
.80
2.7
101
25.1
6.0
.79
2.7
Grade 11
70
26.1
6.3
.81
2.7
60
25.2
6.8
.86
2.5
Grade 12
44
25.8
7.6
.90
2.4
37
24.8
6.6
.82
2.8
58
28.7
6.3
.88
2.2
70
28.3
7.2
.92
2.0
Apprentice applicants at a steel plant in
Illinois
a
Split-half coefficients corrected by the Spearman-Brown formula.
Evidence of Validity
The validity of a test refers to the degree to which evidence supports the interpretation
of test scores for their intended purpose. There are three major types of evidence of
validity. For the Industrial Reading Test, all three types—content, criterion-related, and
convergent/discriminant validity are presented.
Evidence of content validity is based on the degree to which the test items adequately
represent and relate to the trait or function being measured. Content validity requires
systematic examination of the intended subject area and the preparation of exercises
and items so that an appropriate sample of this intended content is incorporated into the
test. The construction of all IRT passages and items was preceded by an examination
of required reading material in selected jobs and training programs. Textbooks and
training manuals from actual industrial training programs were used to establish an
appropriate content outline and appropriate difficulty levels for passages.
Evidence of criterion-related validity is obtained from data that show the statistical
relationship between IRT test scores and other measures of performance, such as
course grades, training program grades, supervisory evaluations, or job performance
measures. This relationship is typically expressed as a correlation coefficient.
Table 6 presents correlation coefficients between the IRT and final grades for high
school students in vocational training programs. The IRT was administered in early
spring and final grades were given in June.
Table 7 presents correlation coefficients between the IRT and course grades for high
school students in specific vocational training programs. The IRT was administered in
early spring and final grades were given in June.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
14
Table 8 presents correlation coefficients between the IRT and various criteria for
students in remedial education and vocational skills programs. Table 9 presents
correlation coefficients between the IRT and various criteria for industrial groups.
In Tables 6 through 9, samples are described in terms of sex and racial composition of
the groups, when available.
Evidence based on the correlation coefficients support that the IRT is a reasonable
predictor of success in both school-based and industrial training programs.
Correlations with Other Tests
Table 10 presents correlation coefficients between the IRT and various other tests. The
data show that the highest correlations tend to be between the IRT and other
standardized tests of reading ability, providing evidence of convergent validity for the
IRT. Significant correlations were also obtained between the IRT and a variety of other
ability measures.
The pattern of correlations suggests that, although there is some overlap with other
measures of ability such as the Bennett Mechanical Comprehension Test (Bennett,
2006) and the various subtests of the Differential Aptitude Tests for Personnel and
Career Assessment (Bennett, Seashore, & Wesman, 1991), there is evidence of
discriminant validity.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
15
Table 6 Correlation Coefficients Between the IRT and Final Grades of High School Students in Vocational Training Programs
Grade
_____IRT____
_____________Criterion______________
Sample
Sex
Race/Ethnicitya Level
Mean
Descriptionb
Mean
N
SD
r
SD
Students in various technical training
programs at three vocational high
schools in eastern Pennsylvaniac
Mostly male
Mostly male
Mostly male
Not reported
Not reported
Not reported
10
11
12
302
241
76
23.6
23.5
25.6
6.9
7.2
6.9
.21**
.11
.15
Final grade
Final grade
Final grade
3.3
0.9
3.4
0.9
3.8
0.9
Students in various technical training
programs at three vocational high
schools in central Pennsylvaniac
Male
Male
Male
White
Mostly white
White
10
11
12
67
148
82
25.6
27.5
25.5
6.3
6.0
6.7
.40**
.33**
.39**
Final grade
Final grade
Final grade
3.1
1.0
3.5
0.9
3.4
0.8
Students in various technical training
programs at a city vocational high
school in central Pennsylvania
Mostly male
Minority
10
64
17.8
7.0
.38
.32**
Final grades:
English
Vocational course
2.9
1.1
2.6
1.1
.30
.50**
Final grades
English
Vocational course
2.9
1.1
**
Male
14 white,
23 minority
11
21.2
8.6
28.2
29.6
5.3
4.9
.30**
.23*
Final grade d
Final grade
3.4
2.3
3.6
1.1
0.9
0.9
268
24.4
7.3
.34**
Final gradef
7.3
7.0
10–
g
12
62
27.6
6.4
.34**
Final grade
3.3
1.0
White
11
96
26.3
6.8
.31**
Final examination grade
74.5
12.4
Male
White
12
84
25.5
6.7
.39**
Final examination grade
68.9
11.8
Male
White
10–
12g
78
25.6
6.6
.53**
.50**
Final examination grade
Final course grade
72.1
79.0
12.2
6.9
Students in various technical training
programs at a vocational high school
in western Pennsylvania
Mostly male
Mostly male
Mostly white
Mostly white
11
12
Students in various technical training
programs at a vocational high school
in West Virginia
Mostly male
Mostly white
10–
12e
Students in various technical training
programs at a vocational school on
the Gulf Coast of Florida
Mostly male
Not reported
First-year students in various twoyear technical training programs at
two vocational high schools in
upstate New Yorkc
Male
Second-year students in various twoyear technical training programs at
two vocational high schools in
c
upstate New York
Students in various technical training
programs at a third vocational high
school in upstate New York
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
37
16
101
86
Table 6 Correlation Coefficients Between the IRT and Final Grades of High School Students in Vocational Training Programs (continued)
Grade
Sample
a
Sex
Students in various technical training
programs at a cooperative vocational high
school in New York State
Students in various technical training
programs at a vocational high school in
Massachusetts
_____IRT____
____________Criterion_____________
Race/Ethnicity
Mostly male
Mostly male
Not reported
Not reported
Level
N
10
140
11
139
Mostly male
Not reported
9
Mostly male
Not reported
10–12
*
e
Descriptionb
Mean
SD
**
Final grade
3.2
1.1
**
.22
Final grade
3.4
1.0
.12
.24**
Final grades:f
Shop
Related instruction
8.1
8.2
2.1
2.6
Final grades:f
Shop
Related instruction
8.6
7.2
2.2
2.9
Mean
SD
r
24.5
7.5
.25
26.5
6.8
195
22.5
7.4
432
25.3
6.6
**
.29
.43**
Significant at p <.05 level.
Significant at p <.01 level.
a
Minority includes Black/African-American and Hispanic/Latino(a).
b
Unless otherwise indicated, the criterion descriptions of “Final grade” and “Final examination grade” refer specifically to vocational courses (and not to
other subjects) and are for the last term completed. Unless otherwise indicated, when letter grades were provided, they were converted to a numerical
scale ranging from 1 (F) to
5 (A).
c
The schools belong to the same school district and use the same grading system. Similar distributions of IRT scores for comparable groups of students
at each school permitted the presentation of combined validity data.
d
Letter grades were converted to a numerical scale ranging from 1 (D) to 4 (A).
e
Preliminary statistical analysis showed no significant increase in mean IRT score from grade 10 to grade 12, and a similar relationship between IRT
scores and course grades across grade levels. Therefore, validity data were combined across grade levels.
f
Letter grades were converted to a numerical scale ranging from 1 (F) to 13 (A+).
g
Due to the small sample size, analysis of the data by grade level was not technically feasible.
**
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
17
Table 7 Correlation Coefficients Between the IRT and Final Grades of High School Students in Specific Vocational Training Programs
______________Criterion_____________
Grade
____IRT____
Program/Sample
Sex
Race/Ethnicitya Level
Mean
Descriptionb
Mean
N
SD
r
SD
Metal-Machining Programs
Machine Shop
Students at three vocational high
c
schools in central Pennsylvania
Students at a vocational high
school in Massachusetts
Male
Mostly white
10–12d
66
27.5
5.1
Male
Not reported
10–12
d
52
26.9
5.6
.29*
3.2
1.0
Shop
9.0
2.4
Related instruction
7.6
2.6
3.2
0.8
.40
.61**
Final grades:
Shop
Related instruction
7.7
5.4
2.7
3.4
Final grades:e
.20
**
.37
Students at two vocational high
schools in eastern Pennsylvaniac
Metalworking Programs
Metal Fabrication
Students at a vocational high
school in Massachusetts
Mechanical Repairing Programs
Aircraft Mechanics
Students at a vocational high
school in western Pennsylvania
Auto Mechanics
Students at a vocational high
school in western Pennsylvania
Agricultural Mechanics, Auto
Mechanics
Students at a vocational high
school in upstate New York
Agricultural Mechanics, Auto
Mechanics, Machine Trades
Students at a vocational high
school in upstate New York
Air Conditioning, Heating and
Refrigeration, Auto Mechanics,
Aviation Maintenance
Students at a cooperative
vocational high school in New
York State
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
Male
Not reported
10
33
22.9
5.3
Male
Not reported
10–12d
37
24.4
7.8
Final grade
.40*
Final grade
e
*
Male
Mostly white
12
40
30.7
4.4
.44**
Final grade
3.4
0.9
Mostly
male
Mostly white
11
54
27.4
5.2
.46**
Final gradef
1.9
0.7
Male
White
11
30
26.3
7.3
.67**
Final examination grade
78.6
11.5
Male
White
10–12d
33
26.0
7.1
.69**
**
.64
Final examination grade
Final course grade
70.0
81.4
14.5
7.7
Male
Male
Not reported
Not reported
10
11
60
61
24.3
27.5
7.4
6.2
.12
.34**
Final grade
Final grade
2.8
3.4
1.0
1.1
18
Table 7 Correlation Coefficients between the IRT and Grades in Specific Vocational Training Programs for High School Students (continued)
Grade
___ IRT____
______________Criterion_____________
Program/Sample
Sex
Race/Ethnicitya Level
Mean
Descriptionb
Mean
N
SD
r
SD
.
Mechanical Repairing Programs
(continued)
Auto Mechanics, Heating and
Ventilation
Male
Mostly white
10–12d
60
25.6
6.4
.55** Final grade
3.5
0.8
Students at three vocational high
c
schools in central Pennsylvania
Auto Mechanics; Diesel Mechanics;
Heating, Ventilation, and Air
Conditioning; Recreational Vehicles
Students at a vocational high
school in Massachusetts
Auto Mechanics, Engine Rebuilding,
Mechanical Maintenance
Students at two vocational high
c
schools in eastern Pennsylvania
Metal Fabrication And Welding
Programs
Welding
Students at two vocational high
schools in upstate New Yorkc
Male
Not reported
Male
Male
Not reported
Not reported
Male
Body and Fender, Welding
Students at a vocational high
school in West Virginia
Auto Body, Sheet Metal, Welding
Students at a vocational high
school in eastern Pennsylvania
Electrical Assembly, Installing, and
Repairing Programs
Electronics
Students at a vocational high
school in western Pennsylvania
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
10–12
g
116
23.7
e
6.9
.30
**
.40
Final grades:
Shop
Related instruction
8.4
7.2
1.9
2.3
3.5
3.6
1.0
0.9
70.3
10.0
**
10
11
90
76
23.0
23.3
6.5
7.7
.15
.03
Final grade
Final grade
White
11–12d
33
25.5
7.6
.38*
Final examination grade
Male
Mostly white
10–12
d
75
22.8
7.5
.13
Final gradee
7.4
2.2
Male
Not reported
11
40
22.0
7.0
.16
Final grade
3.2
0.9
Male
Mostly white
11
32
29.1
5.5
.01
Final grade
f
3.2
0.8
19
Table 7 Correlation Coefficients Between the IRT and Grades in Specific Vocational Training Programs for High School Students (continued)
Grade
____IRT____
______________Criterion_____________
Program/Sample
Sex
Race/Ethnicitya Level
Mean
Descriptionb
Mean
N
SD
r
SD
Electrical Assembly, Installing, and
Repairing Programs (continued)
Electronics, Industrial Electricity
Students at a vocational high
school in West Virginia
Male
Mostly white
10–12
d
45
29.9
5.3
Electronics, Electromechanics
Students a vocational high
school in Massachusetts
Male
Not reported
10–12d
43
29.7
5.1
Basic Electricity, Basic Electronics
Students at three vocational high
schools in eastern Pennsylvaniac
Male
Not reported
10
58
28.2
Male
Not reported
11–12d
42
Male
Not reported
10–12d
Male
White
Male
Not reported
Male
Construction Electricity, Industrial
Electricity
Students at three vocational high
c
schools in eastern Pennsylvania
Electricity, Trade Electricity
Students at a cooperative
vocational high school in New
York State
Construction Trade Programs
Carpentry
Students at three vocational high
schools in central Pennsylvaniac
Students at a cooperative
vocational high school in New
York State
Carpentry, Masonry
Students at a vocational high
school in West Virginia
Carpentry, Masonry, Plumbing
Students at a vocational high
school in Massachusetts
Copyright © 2006 by NCS Pearson, Inc.
All rights reserved.
Final gradee
7.9
2.5
.39
**
.49
Final grades:e
Shop
Related instruction
8.7
7.7
2.3
2.6
5.7
.13
Final grade
3.5
0.8
27.3
5.5
.22
Final grade
3.5
0.8
45
26.6
5.5
.32*
Final grade
3.4
0.9
10–12d
57
27.6
6.0
.47**
Final grade
3.6
0.9
10–12
d
33
22.2
7.7
.27
Final grade
3.5
1.1
Mostly white
10–12d
50
22.7
7.1
.42**
Final gradee
7.3
2.1
Mostly
male
Not reported
10
35
22.7
7.2
.35*
.50**
Final grades:
Shop
Related instruction
8.7
6.6
1.5
3.7
Male
Not reported
.17
.53**
Final grades:
Shop
Related Instruction
9.7
7.9
1.4
3.1
20
11
30
25.1
.55**
**
e
e
5.3
Table 7 Correlation Coefficients Between the IRT and Grades in Specific Vocational Training Programs for High School Students (continued)
Grade
_____IRT____
____________Criterion_____________
Program/Sample
Sex
Race/Ethnicitya Level
Mean
Descriptionb
Mean
N
SD
r
SD
Construction Trade Programs
(continued)
Building Trades, Carpentry, Masonry,
Plumbing
Students at two vocational high
schools in eastern Pennsylvaniac
Male
Male
Not reported
Not reported
*
10
11
77
41
20.6
21.8
7.1
7.1
.08
.14
Final grade
Final grade
3.0
3.2
0.9
1.0
Significant at p <.05 level.
Significant at p <.01 level.
a
Minority includes blacks Black/African-American and Hispanic/Latino(a).
b
Unless otherwise indicated, when letter grades were provided, they were converted to a numerical scale ranging from 1 (F) to 5 (A).
c
The schools belong to the same school district and use the same grading system. Similar distributions of IRT scores for comparable groups of students
at each school permitted the presentation of combined validity data.
d
Due to the small sample size, analysis of the data by grade level was not technically feasible.
e
Letter grades were converted to a numerical scale ranging from 1 (F) to 13 (A+).
f
Letter grades were converted to a numerical scale ranging from 1 (D) to 4 (A).
g
Preliminary statistical analysis showed no significant increase in mean IRT score from grade 10 to grade 12, and a similar relationship between IRT
scores and course grades across grade levels. Therefore, validity data were combined across grade levels.
**
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
21
Table 8 Correlation Coefficients Between the IRT and Criteria for Students in Special Remedial and Vocational Programs
_____IRT____
___________Criterion_
Mean
Descriptionb
Program/Sample
Sex
Race/Ethnicitya
N
SD
r
Adult students in an individualized
instruction program at a vocational
center in California
Adult students in technical
programs at a skills center in
Tennessee
Adult students in technical
programs at a vocational institute
in Florida
Adult students in electrical
assembling, installing, and
repairing programs at a vocational
institute in Florida
Students in CETA
(Comprehensive Employment
Training Act) programs in upstate
New York
69 male, 18 female
41 white, 46 minority
87
21.9
8.5
.30
Course gradesb
3.0
0.9
22 male, 17 female
19 white, 20 minority
39
24.8
7.1
.30
Teachers’ evaluationsc
3.8
1.4
4/5 male,
1/5 female
Minority
113
22.8
6.9
.14
Teacher’s evaluation of
learning rated
3.2
1.0
4/5 male,
1/5 female
Minority
101
22.9
7.0
.18
Class rank
2.6
1.0
Mostly male
Mostly white
140
28.4
6.3
.20*f
Completion of course vs.
withdrawal or failure
—
—
Mostly male
Mostly white
108
28.9
6.1
.24
Final gradeg
3.8
0.8
g
3.7
1.0
5.8
5.4
5.5
5.6
2.1
2.1
2.2
2.3
h
Mostly male
Mostly white
29
31.0
5.8
45 male, 50 female
Mostly white
95
23.6
8.0
**
___________
Mean
SD
*
**
.66
*
.24
.22*
*
.24
.24*
*
e
Final grade
Instructors’ ratings:i
Ability to learn
Communication skills
Progress in training
Likelihood of job
success
Significant at p <.05 level.
Significant at p <.01 level.
a
Minority includes Black/African-American and Hispanic/Latino(a).
b
Letter grades were converted to a numerical scale ranging from 1 (D) to 4 (A).
c
Teachers’ evaluations were coded on a 5-point scale ranging from 1 (poor) to 5 (excellent).
d
Learning rate was coded on a 5-point scale ranging from 1 (very slow) to 5 (very fast).
e
Class rank was coded on a 4-point scale corresponding to the particular quartile indicated for a student. Top (or first) quartile was coded 4, second
quartile was coded 3, third quartile was coded 2, and bottom (fourth) quartile was coded 1.
f
Point-biserial r.
g
Letter grades were converted to a numerical scale ranging from a (F) to 5 (A).
h
A subgroup of the sample of 108.
I
Students were rated on each factor separately on a scale of 1–10.
**
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
22
Table 9 Correlation Coefficients Between the IRT and Criteria for Industrial Groups
____IRT____
Mean
Program/Sample
Sex
Race/Ethnicitya
N
SD
Service technician trainees at a
technical training center in
Virginia
Utility operator trainees at a
southern utility
Mostly male
Mostly male
Mostly white
Mostly white
133
203
33.9
34.0
3.6
3.4
Mostly male
Mostly white
72
34.1
2.4
Mostly male
Mostly white
50
34.8
r
.11
.16*
.33**
.38**
.27*
1.6
.06
.02
Health physics technicians in a
training program at a southern
utility
Male
Apprentices in a construction
union in the Northeast
Male
Mostly white
½ white, ½ minority
24
56
33.2
30.6
2.6
5.3
.70**
.53**
.24
.31
.50*
*
.27
.52**
*
_ _________ __Criterion_
Descriptionb
________
Mean
SD
Supervisory ratingsb
Productivity of machine between
service calls
Course grades:
Systems and components-general
Systems and components-specific
Mathematics
Course grades:
Nuclear preparation
Nuclear fundamentals
Final grades obtained in four
training modules:
Introduction to the company
Systems and components
Introduction to health physics
Basic health physics
Average grade – all four modules
Percentile rank of ratings on a
work sample
Job knowledge test
3.5
88.2
0.5
44.3
94.8
84.5
89.6
4.1
6.3
6.9
85.6
85.3
5.6
5.7
94.1
89.3
78.3
85.5
86.8
50.1
4.0
8.4
9.2
6.2
5.7
28.8
17.7
4.8
Significant at p <.05 level.
**
Significant at p <.01 level.
a
Minority includes Black/African-American and Hispanic/Latino(a).
b
Mean of last three performance appraisals. Performance was rated on a scale from 1 (low) to 5 (high).
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
23
Table 10 Correlation Coefficients Between the IRT and Other Tests
Test
Sample
N
_____IRT_____
Mean
SD
__Other Test__
Mean
SD
r
Bennett Mechanical Comprehension Test, Form S
Apprentice applicants at a steel plant in Illinois
Apprentice applicants at a steel plant in Indiana
143
33
27.4
30.5
7.5
4.8
.67
.37*
**
40.9
46.1
11.9
8.2
Bennett Mechanical Comprehension Test, Form T
Technical workers at an oil refinery in the South
83
29.5
5.4
.38**
46.4
9.1
Differential Aptitude Tests, Form S
Verbal Reasoning (VR)
Numerical ability (NA)
VR + NA
Abstract Reasoning
Clerical Speed and Accuracy
Mechanical Reasoning
Space Relations
Spelling
Language Usage
Students in grades 10–12 at a cooperative
vocational high school in New York
159
22.8
7.1
.55
.51**
.59**
.40**
–.03
**
.34
.34**
.47**
**
.54
**
20.3
16.8
37.1
29.8
47.1
42.9
25.9
58.4
25.4
10.5
7.8
16.6
9.9
17.2
10.3
10.6
16.8
10.0
Differential Aptitude Testsb
Verbal Reasoning (VR)
Numerical ability (NA)
VR + NA
Abstract Reasoning
Clerical Speed and Accuracy
Mechanical Reasoning
Space Relations
Spelling
Language Usage
Students in grades 10–12 at three cooperative
vocational high school in eastern Pennsylvania
Differential Aptitude Tests, Form T
Numerical Ability
Apprentice applicants at a steel plant in Illinois
Applicants for apprentice training at a
construction union in the Northeast
Mechanical Reasoning
Space Relations
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
24
908
908
908
485
764
850
843
768
764
23.7
23.7
23.7
23.5
23.7
23.7
23.8
23.7
23.7
7.0
7.0
7.0
7.1
7.0
7.0
7.0
7.1
7.0
.48**
.39**
.50**
.33**
.12**
.28**
.24**
.32**
.41**
20.9
17.3
38.2
31.6
42.0
42.3
27.4
54.9
23.5
8.4
6.4
13.0
8.8
13.5
9.8
10.6
16.3
8.2
143
56
27.4
30.6
7.5
5.3
.73**
20.1
10.8
.16
.30**
59.7
43.3
6.6
10.7
Table 10 Correlation Coefficients Between the IRT and Other Tests (continued)
Test
Differential Aptitude Tests, Form A
Numerical Ability
Minnesota Clerical Test
Names
Numbers
Personnel Tests for Industry, Form A
Verbal
Numerical
Revised Minnesota Paper Form Board Test
Sample
N
_____IRT_____
Mean
SD
Utility operator and technician trainees at a
southern utility
75
34.1
2.4
Utility operator and technician trainees at a
southern utility
75
34.1
2.4
Utility operator and technician trainees at a
southern utility
100
33.9
143
27.4
7.5
Basic Occupational Literacy Test
Reading Vocabulary
Reading Comprehension
Arithmetic Computation
Arithmetic Reasoning
Adult students at a skills center in Tennessee
116
22.4
6.7
California Achievement Tests, Form A and B
Reading
Mathematics
Language
California Test of Mental Maturity
Students in CETA (Comprehensive Employment
Training Act) programs in upstate New York
Gates–MacGinitie Reading Tests:
Survey F, Form 2M
Comprehension
Short Form Test of Academic Aptitude
Language
Nonlanguage
Total
Reading for Understanding Placement Test
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
24.6
329
25.4
6.8
Adult students at a vocational institute in Florida
Adult students at a vocational center in California
Students in CETA (Comprehensive Employment
Training Act) programs in upstate New York
168
87
90
27.8
21.6
24.7
6.5
8.4
7.7
Adult student at a skills center in Tennessee
182
25
31.1
5.0
.12
.03
117.8
115.7
19.9
24.3
.16
**
.28
.58**
42.0
23.8
37.1
4.3
3.0
11.6
.56**
.57**
**
.30
**
.46
108.9c
121.0 c
c
107.0
c
108.3
16.0 c
16.4 c
c
15.7
c
18.4
.83**
.42**
.70**
.58**
10.4d
9.3 d
9.2 d
41.6
2.9 d
3.4 d
3.0 d
10.6
.79**
.70**
10.9 d
9.1 d
2.7 d
4.1 d
**
100.6e
94.1 e
97.5 e
88.3
15.2 e
e
14.5
14.6 e
21.7
8.0
Students at three vocational high schools in
upstate New York
22.3
.16
2.5
Apprentice applicants at a steel plant in Illinois
94
__Other Test__
Mean
SD
r
7.0
.73
**
.69
.78**
.65**
Table 10 Correlation Coefficients Between the IRT and Other Tests (continued)
Test
Wide Range Achievement Test, Level 2
Arithmetic Computation
Ohio Trade Tests
Sample
Adult students at a vocational institute in Florida
_____IRT_____
Mean
SD
r
168
27.8
6.5
.58**
40
49
45
28.9
25.8
22.9
7.1
6.2
7.0
.60**
.67**
.63**
_ Other Test__
Mean
SD
7.6d
2.2d
Students at three vocational high schools in
Pennsylvania and one in West Virginia
Basic Electronics Achievement
Machine Trades Achievement
Welding Achievement
*
Significant at p <.05 level.
Significant at p <.01 level.
a
Means and standard deviations are expressed in raw scores, unless otherwise indicated.
b
Form unknown.
c
Standard scores.
d
Grade equivalents.
e
IQ scores.
**
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
N
26
126.2
139.9
109.9
37.5
39.0
25.9
References
Anastasi, A. & Urbina, S. (1997). Psychological testing (7th ed.). Saddle River, NJ: Prentice Hall.
Bennett, G. K. (2006). Bennett Mechanical Comprehension Test manual. San Antonio, TX: Harcourt
Assessment, Inc.
Bennett, G. K., Seashore, H. G., & Wesman, A. G. (1991). Differential Aptitude Tests for Personnel
and Career Assessment technical manual. San Antonio, TX: Harcourt Assessment, Inc.
Dale, E., & Chall, J. S. (1948). A formula for predicting readability. Educational Research Bulletin,
27, 11–20, 37–54.
Powers, R. D., Sumner, W. A., & Kearl, B. E. (1958). A recalculation of four readability formulas.
Journal of Educational Psychology, 49, 99–105.
Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of
personnel selection procedures (4th ed.). Bowling Green, OH: Author.
U.S. Department of Labor. (1977). Dictionary of occupational titles (4th ed.). Washington, D.C.: U.S.
Government Printing Office.
U.S. Department of Labor. (1978). Occupational outlook handbook. Washington, D.C.: U.S.
Government Printing Office.
Copyright © 2006 NCS Pearson, Inc.
All rights reserved.
27