Elise Cronin & Kaleen Schultz
TORC-4
What it is:
 The TORC is a measure of silent reading comprehension. The test consists of
five subtests.
What it is used for:
 The TORC-4 is appropriate for individuals who vary widely in the type of English
they speak.
 TORC is intended for identifying children who score significantly below their
peers. This gives us a better understanding if they need help in improving
reading skills. TORC is also used for documenting student progress.
 The TORC is meant to measure word identification and contextual meaning. The
overall score from the subtest will give an appropriate approximation of students
reading comprehension abilities.
Who it is for:
 The TORC-4 should only be given to individuals between ages 7-0 and 17-11
o Individuals must be capable of understanding the tests directions and
formulate responses prior to taking the test
Norming:
"The TORC was normed on a sample of 1,942 persons in 14 states." For each major
standardized site, the site coordinator hired experienced personnel to administer the
test. The standardization sites were chosen to ensure representation in the four regions
of the U.S. The results gave a sample that is to represent the nation.
 The characteristics of the sample are reported in regards to gender, race,
geographic region, Hispanic status, family income, exceptionality status, and
educational level of parents.
Drawbacks: The drawbacks are listed under each subtest section.
Subtest 1
Relational Vocabulary
 The student reads 3 words that are related to each other, and then the student
reads a different list of four words and chooses 2 words that are related to the
original three words.
 This measures the readers understanding of sets of vocabulary items that are
associated with the same general concept.
Example:
Red
Green
A.
B.
C.
D.
Blue
Yellow
Circle
Orange
Light
The answers are A. and C.
 Student is not timed; rather the test administrator observes the student's
responses and notes if they're correct or incorrect. Once the student has met the
subtest's ceiling, the test is stopped.
Materials Student booklet and answer sheet, examiner book and record sheet,
pencils
Basal
Begins at item 1.
Ceiling
When 3 in any 5 consecutive items are scored incorrect
Scoring




Correct item scored as 1
Incorrect item scored as 0
Student must circle BOTH items correctly to receive score of 1. If
BOTH items are not circled then they receive a 0
The raw score is the number of correct items including the last
correct item in the last five consecutive items
 Validity: Having a vocabulary subtest contributes to the validity of this test
because vocabulary knowledge is critical to reading comprehension for three
reasons:
1. We read and understand text through our knowledge of word meanings as
they stand alone as well as in context.
2. Vocabulary is embedded in specific topics and knowledge of a topic helps
with comprehension.
3. Vocabulary, reading and general intellectual ability are highly correlated.
 Limitations:
o It is difficult to watch the student mark their responses while looking to see
if they have reached the ceiling.
Subtest 2
Sentence Completion
 The student reads a sentence that is missing two words. The student then reads
a list of word pairs and chooses the best fit for the sentence.
 This subtest measures the students “understanding of interaction with sentence
structure, vocabulary and implied context.”
Example:
A ____ can____.
A.
B.
C.
D.
E.
Hill-play
Wet-swim
Dog-run
Cow-read
Car-talk
The answer is C.
 The student is not timed. The instructions are read to the student, the instructor
goes through the examples with the student. If the student cannot get the
examples correct, subtest two is skipped. If the student can correctly complete
the examples the subtest is given. The test is stopped once the ceiling is
reached.
 This subtest uses the cloze procedure which has the student fill in missing words
in the passage.
Materials Student booklet and answer sheet, examiner book and record sheet, pencils
Basal
Begins at item 1.
Ceiling
When 3 in any 5 consecutive items are scored incorrect
Scoring


Correct item scored as 1, incorrect item scored as 0
The raw score is the number of correct items up to and including the
last item passed in the five-item sequence.
 Validity: This subtest contributes to the validity of the TORC by incorporating
sentence completion. Good readers know which words make sense within
context and which words do not make sense. In addition, good readers are able
to predict which word will make sense based on context. Lastly, readers should
be able to identify which words will complete a sentence.
 Limitations:
o It can be difficult to look at the student’s sheet from a comfortable distance
while watching for the ceiling to be reached. This needs to be done
without interfering with the student; however, it is important we see when
they meet the ceiling.
Subtest 3
Paragraph Construction
 The student reads a list of sentences which are in the wrong order. The student
must think about how the order should go. The student will then be asked to
rearrange the sentences into an order that makes sense.
 This allows us to measure the reader’s ability to construct meaningful
paragraphs.
Example:
A. Then I will eat dinner.
B. I eat in the morning.
C. Next I will eat lunch.
The answer is B, C, A.
 The examiner does not time the student. The instructions are read to the student.
Then, the examiner and student go through the examples. The subtest is then
started. There are 3 to 5 sentences the student must put into order. The subtest
is over when the ceiling is reached.
Materials
Student booklet and answer sheet, examiner book and record sheet,
pencils
Basal/Ceiling When less than three points on any two consecutive items is scored,
beginning with item four.
Scoring



A score of 0, 2, 3, 4 or 5 is possible for each passage.
A student receives 2 points if two sentences have been
sequentially ordered.
3 points are given if 3 sentences have been sequentially



ordered.
4 points are given if a student successfully orders 4 sentences in
a row or has 2 sets of 2 sequentially ordered sentences.
5 points are given when the student successfully orders all 5
sentences, or, the student sequentially orders 2 sentences and
3 sentences.
The raw score is the total number of points the student receives
up to and including the ceiling item.
 Validity: This subtest shows validity by measuring the ability to order sentences
into plausible paragraphs. Reading comprehension requires us to understand
how sentences relate to each other and how those sentences create a short
story without a model. Ordering sentences in a logically constructed paragraph
has readers demonstrating their ability to create a coherent cognitive framework
in the absence of a model or title.
 Limitations:
o This subtest is really hard to score while the student is taking the test.
o The scoring instructions are complex. The student can score 0,2,3,4 or 5
points for each passage. It is difficult to see the order of each response
and calculate the appropriate points earned while the student is working.
Subtest 4
Text Comprehension
 The student will be read the instructions by the examiner. The examiner will go
through the first example with the student. The subtest requires the student to
answer questions based on a short story.
 This subtest measures the reader’s ability to answer questions about the content
of stories that become progressively more difficult.
 The test is not timed. If the student has trouble correctly answering the example
questions, the examiner needs to go over the example again. The test is then
allowed to start. The subtest is done when the ceiling is reached.
Example:
Story:
The girl plays with her dog.
Her dog is brown.
She likes to play ball with her dog.
Her mom gave her the dog.
Question:
What color is her dog?
A.
B.
C.
D.
Black
Green
Brown
Purple
The answer is C.
Materials Student booklet and answer sheet, examiner book and record sheet,
pencils
Basal
Begins at 1
Ceiling
When any three questions are missed (scored 0) in each of three
consecutive stories.
Scoring


Correct answer is 1.
Incorrect answer is 0.

The raw score is the total number of points the student receives.
 Validity: This subtest shows validity by measuring the reader’s ability to answer
questions about the content of stories in passages. The passages include testing
in the areas of vocabulary, grammar and content. The method in subtest 4 is one
of the most widely used formats for assessing reading comprehension. The
approaches taken in the subtest such as reviewing the questions before the story
is read, and skimming an article or story are relevant to real life and therefore
reflect a valid approach to reading comprehension assessments.
 Limitations:
o The paragraphs start out short and progressively get long. The tests
become very long; therefore, if a student is a slow reader, the test could
take longer than anticipated to complete.
o
Subtest 5
Contextual Fluency
 This subtest is designed to measure the speed that students can recognize
words without spacing. Students draw a line to separate one word from the next.
 Subtest 5 measures the student’s ability to single out words in a continuous
string of letters that can be sorted into words.
Example:
IHAVEATOY
The answer should look like:
I/HAVE/A/TOY
 The items are printed in the student answer booklet rather than the student
question booklet. The examiner will read the manual and work through the
example with the student. This subtest is timed. The test stops at exactly 3
minutes.
Materials Student answer booklet, examiner book and record sheet, pencils, stop
watch
Basal
Ceiling
Scoring
Begins at 1
Three Minutes







Do not score unless the student successfully completes the
examples and shows understanding of the task on the practice item.
Begin scoring with the last row that the student attempted.
Each correctly identified word is worth 1 point.
Score backward until you reach the spot at which the student has
correctly identified all the words in a passage or until all words have
been scored.
Scoring key is in Appendix E.
Raw score is the total number of correctly identified words.
Scoring concerns are listed in the limitations section of this board.
 Validity: The passages used on this subtest are the same as those on the Text
Comprehension subtest; therefore, the student is more apt to have familiarity with
the content of the passages in this subtest. This allows consistency with the idea
that familiarity of context plays an ideal role in reading. Fluency is a main and
fundamental aspect of reading comprehension.
 Limitations:
o The directions for this subtest are somewhat unclear. The administer is
supposed to write on the students answer sheet when going through the
examples. The directions state the administer of the test is to teach the
task to the student if the student does not understand the example. There
is an example script for teaching the task to the student, located at the
bottom of the directions. However, If the administer does not explain the
task well to a confused student, it may affect the performance of the
student.
Test Interpretation
Derived Scores: The TORC-4 examiner record form has section two set aside for
recording. Raw scores, age equivalents, grade equivalents, and percentile ranks are the
derived scores found in this test. A derived score is a group of scores that allows us to
compare between test scores.
o Age: Age equivalents are derived from the average scores of all students
who took the exam in the norm sample. The raw scores that relate to the
50th percentile for each age was then determined. Those raw scores were
plotted on a graph, lines connected the average scores. It was then
determined the age that corresponded to each possible raw score.
o Grade: Found similar to age equivalents, the average raw score at four
intervals for each grade was computed and graphed. Lines connecting the
average scores had to be smoothed which then allowed the grade level to
correlate to each possible raw score.
o Percentile rank: These indicate the percentage of the distribution that is
at or below a certain score.
Scoring: The results are interpreted using the Appendix in the back of the
examiners manual.
o Age and Grade Equivalence: Age and grade equivalencies for raw
scores are found using Appendix C and D
 In each subtest, a raw score is documented. The raw score
corresponds to an age and grade for each subtest in the appendix.
 For example: a student’s raw score in the subtest of Sentence
Completion gave a 21. This shows an age equivalency of 10-3.
This means the student performed consistent with the students in
the normative sample who were 10 years and 3 months old. The
grade equivalency for a raw score of 21 in the same subtest gave
5.0. This means the student performed close to that of students in
the norm sample who were in the first month of fifth grade.
o Percentile Rank: The table in appendix A and B that corresponds to the
student’s age is used to interpret the percentile rank.
 Say we want to find a student’s percentile rank for the same subtest
above, the raw score of 21 shows a percentile rank of 50. This
means that 50% of the norm sample scored at or below the
students score.
o Scaled Scores: Standard scores for the subtests are called scaled
scores.
 To find the scaled score for each subtest, the examiner consults
Appendix A. The tables are based on the age of the student.
 For a student at the age of 10-7, the examiner would use table A.8
in the back of the examiners manual. The raw score from each
subtest corresponds with a scaled score.
 The TORC-4 scaled scores have a mean of 10 and a standard
deviation of 3.
Results: The Reading Comprehension Index (RCI) is a composite of the scaled
scores from the 5 individual subtests. The RCI is found by combining the results of the
five subtests. The subtests measure related but different areas of reading
comprehension.
o There are descriptive terms that correspond to specific score ranges. The
terms are as follows:
 Very superior (scaled score of 17-20) (RCI >130)
 Superior (scaled score of 15-16) (RCI 121-130)
 Above average (scaled score of 13-14) (RCI 111-120)
 Average (scaled score of 8-12) (RCI 90-110)
 Below average (scaled score of 6-7) (RCI 80-89)
 Poor (scaled score of 4-5) (RCI 70-79)
 Very poor (scaled score of 1-3) (RCI <70)
o Test results are observations, not a diagnosis for a student.
o The results specify performance level for a certain time, under a certain
situation.
Psychometrics
Reliability: refers to the consistency with which a test measures ability.

o
o
The purpose of a test is to measure a certain characteristic, ability,
or content. The more the items relate to each other, the smaller the
error in the test.
 Reversely, if test questions are unrelated to each other, the amount
of test error will be large.
 Based on the Standard errors of Measurements, we know that the
TORC-4 contains little or no bias relative to the subgroups in the
normative sample and therefore the general population.
Coefficient Alpha:
 For a test to be reliable it should have a coefficient of .9 or higher.
 The TORC-4 has 5 subtests, but one of the subtests is timed and
therefore does not get a coefficient. The rest of the subtests
obtained a .9 or higher indicating that the test is highly reliable.
Standard Error of Measurement (SEM):
 SEM estimates the amount of error in an individual’s test score due
to the less-than-perfect reliability of the test. In other words, it
establishes a zone within which an individual’s true score probably
lies.
 Why is this important?
 One way for a score interpreter, the examiner, to control
error.
 Examiners should know that a test score is only an estimate
of the students test performance.
 By adding and subtracting the SEM to and from a student’s
score, the examiner can determine the student’s limits.
o Test Re-test:
 Error due to time sampling means that a student's testing
performance may change as a result of the passage of time
between tests.
 Time sampling error is usually estimated by the test-retest method.
 When the two tests are administered at different times and the
results are compared, the degree of similarity between the two tests
shows the amount of stable reliability the test has.
o How the TORC-4 test/retests:
 The TORC-4 assessed a group of 68 regular class public school
students in Austin, TX within the same age range as the tests
requirements.
 Time lapse varied from 1 to 2 weeks.
 The coefficients for the combined group equal or exceed .8 (.8 is
considered minimally reliable and .9 is considered most desirable).
 The coefficients for the Reading comprehension are .94.
 Meaning if a student retakes this test, we can be very confident that
the student will obtain a similar score.
o Scorer Differences: Refers to the amount of test error due to the
examiners scoring difference.
 The TORC-4 tested the consistency of their scoring procedures
among 50 protocols drawn from students in the normative sample.
 The scorers who were not language or assessment specialists,
learned to administer the TORC-4 by reading the Examiners
Manual, the Student Questions Booklet and the Student Answer
Booklet.
 The results of the scorings were correlated with a resulting
coefficient of .90 or higher.
Validity: The validity of the TORC-4 is confirmed using criterion-prediction
validity and construct-identification validity.
 Criterion-prediction validity is the effectiveness of a test in
predicting an individual's performance in specific activities. In other
words, it compares the test with other outcomes (the criteria)
already validated. For example, IQ tests are often validated against
measures of academic performance (the criterion) with criterionprediction we can predict that a high IQ is likely to yield high
academic performance.
 The relationship of the TORC-4 and four other literacy tests were
studied. They correlated the TORC-4's scores with the criterion
measures of reading, compared their means and standard
deviations and finally performed a series of positive predictive
analyses pertaining to the tests sensitivity, specificity and positive
predictive value. They compared their samples to several validated
intelligence and reading tests. They found that the TORC-4 highly
correlates with the valid tests compared.

Construct-identification validity relates to the degree to which
underlying traits of a test can be identified and to which these traits
reflect the assumptions on which the test is based. There is a three
step process to determine this kind of validity written out in the
manual. Using this method, they found the basic constructs that
underlie the TORC-4 were: relationship to age, relationship among
subtests, and relationship to intelligence, and differences among
groups (bias).
Conclusion
Thoughts: Overall the TORC-4 was easy to give as far as instructions went.
o The biggest downfall of the test was scoring as the student is responding
to each subtest.
o The TORC is a comprehensive reading assessment that proves to be
valid, reliable and fairly stress-free to administer.
Unique Features: After reviewing all of the information, it is important to note
the test suggests when it is appropriate to test in group vs. testing a student
individually.
o Do not test in a group when:
 Testing for placement purposes
 Testing students who you know are poor readers
 Testing students with disabilities
o You may test in a group when:
 Testing for research purposes
 Screening an entire classroom for low performance in reading
List of Instruments: Below is a list of other tests that measure similar areas
in reading as the Torc-4 does.
o Subtest 1: Relational Vocabulary
 Gray Diagnostic Reading Test-second edition
 Woodcock Reading Mastery Tests (WRMT-R)
 Group Reading Assessments and Diagnostic Evaluation (GRADE)
o Subtest 2: Sentence Completion
 Workbook for Synonyms, Homonyms, and Antonyms
 Lincoln Library for Essential Information
 The Writers Digest Flip Dictionary
 GDRT-2
 GRADE
 WRMT-R
o Subtest 3: Paragraph Construction

This subtest does not compare to another test but teachers are
showing students to build overall meaning as they read and
understand the text as a whole
o Subtest 4: Text Comprehension
 GDRT-2
 WJ III DRB
 SRI-2
 GSRT-2
o Subtest 5: Contextual Fluency
 WJ III Diagnostic Reading Battery
 GORT-4
 TOSWRF
 TOSCRF