Sterling Practices in Designing and
Scoring Performance-Based Exams
Ronald C. Rodgers, Ph.D.
Continental Testing Services, Inc./
Employment Research Institute
Presented at the 2005 CLEAR Annual Conference
September 15-17
Phoenix, Arizona
Design Principles
• Select work products that can be tested
uniformly for all candidates (materials, etc.)
• Blind Scoring: If possible, have examiners
score work products anonymously
• Observed Scoring: Checklists of observable
steps may limit bias if blind scoring not feasible
• Apply industry standards in scoring criteria
• Special security steps may be required
Sample 1: Plumbing Practical Exam
Residential Drawing
Three-story apartment building with fixtures
Finish plumbing specifications for all vents and
drains in compliance with Plumbing Code
May use isometric or diagrammatic response
Copper Pipe Assembly
Bring specified 5′ & 2′ copper tubing & fittings
Measure, cut, assemble and solder pipe left- or
right-hand rolling offset around obstruction
Sample Copper Pipe Specifications
Blind Scoring: Plumbing
Residential Drawing (Did candidate follow Code?)
 Bathroom soil stacks, drains, vents, sizing (50%)
 Kitchen waste stack, drains, vents, sizing (24%)
 Basement ejector, laundry, sewer connection (26%)
Copper Pipe Assembly (Quality of workmanship)
 Measurement, Assembly & Alignment (53%)
(deduct for each ¼″ difference over ±¼″)
 Pressure Test at 100 psi under water (30%)
 Quality of soldering & reaming (17%)
Plumbing Practical Exam Security
Two building drawings with same fixtures;
candidates seated with alternate drawings
Two copper projects; no two candidates with
same project placed together in shop
Pipe dimensions change for every test
Must show calculations for 4 pipes not given
Repeat candidates receive different copper
pipe project and drawing for any retesting
Sample 2: Court Reporters
• Professionally recorded 5-minute opening
or closing argument at 200 wpm
• Professionally recorded 5-minute testimony
in two contrasting voices at 225 wpm
• Two-minute practice test in same voice(s)
• Word & syllabic counts controlled for each
15-second interval throughout dictations
• CD controls playback speed & intervals
Blind Scoring: Court Reporters
Define scoring criteria in candidate study guide
 Wrong, omitted, added or misspelled words
 Wrong or missing dates or numbers
 Omitted capital letters for specified words
 Also specify what is NOT scored as an error
(misplaced commas & periods, spacing, etc.)
Passing score = 95% accuracy in transcript*
 50 errors or less in 200 wpm transcription
 56 errors or less in 225 wpm transcription
* National Court Reporters Association standard
Court Reporters Exam Security
Exam CDs repeat in two-year rotation
Proctors have directions to erase digital
memory from transcription machines
All cell phones and recording devices must
remain outside room during testing
Paper-less transcription machines prohibited
(proctors would need to erase exam from
three digital memory sites in machine)
Sample 3: Carpentry/Rafters Test
Quality and Safety Requirements
• Layout and cutting ± 1/16" of specifications
• Layout must be performed in required
measurement and marking sequence
• Layout/cutting must be done within time limits
• The test must be stopped and candidate scored as
failed if a candidate removes safety glasses while
working on project and/or fails to follow any other
OSHA safety procedures
Observed Scoring: Carpentry
Cutting
All common rafter cuts ± 1/16" of plans or
specifications
Tracing Second Rafter
Traced common rafter marks ± 1/16" of plans or
specifications
Followed Measurement and Marking Sequence
Finished Cutting & Tracing within Time Limit
Contact Information
Ronald C. Rodgers, Ph.D.
Director of Measurement Services,
Continental Testing Services
& President, Employment Research Institute
809 Ridge Road, Suite 201
Wilmette, IL 60091
847-256-5240
Fax 847-256-5241
rrodgers@qmail.com