January 2012 Regents Scanning
Training for CFN 201 Assessment Liaisons
Version 2.02 – December 2011
1
Objectives
The purpose of this presentation is to educate you on the
Regents scanning procedures, and empower you to turn-key the
process with your school.
After viewing the presentation, you will understand:
> The overall scanning process and step-by-step procedures for each
stage;
> How your school should plan for Regents exam week, and special
situations to prepare for;
> What documents and resources are available for training and
support, including use of the demo IT environment for school
training;
> Best practices for turn-keying the scanning process at schools; and
> Process updates for the January 2012 administration
2
Process Changes for Jan. 2012 Regents Scanning
> Key differences for January 2012 Regents Exams include:
 A new exam code deck has been released in STARS student
scheduling. (Slide 9)
 Addition of a “Did not meet lab requirement” bubble for students who
were scheduled in STARS, but failed to meet lab requirements prior to
the exam, or who were mistakenly allowed to sit for the exam despite
not having met lab requirements. (Slide 20)
 No final score will be calculated until an answer document is fully
scanned. (Slide 24)
 Changes to the RGTU screen in ATS – principals can correct scanning
errors on fully scanned answer documents. Any changes that result in a
score moving from fail to pass will be held for DAPS approval and will
lock subsequent rescanning for that document. (Slides 29-31)
 Elimination of the centrally reported data validation process. (Slide 32)
 Addition of a misadministration field for scanned documents in RGTU to
eliminate score reporting to the transcript and report card systems
(Slides 20, 21, 31)
3
SED Regents Requirement & DOE Solution
SED Requirement: Beginning in June 2011, the SED requires all districts
to collect and submit item-level data for the following Regents exams:
> Comprehensive English
> Integrated Algebra
> Earth Science
> Living Environment
> Global Studies & Geography
> U.S. History & Government
Beginning in June 2012, all ten Regents exams will be scanned, including
Algebra 2/Trig, Geometry, Chemistry, and Physics.*
DOE Solution: Leverage existing technology and systems currently used
by schools to capture and report item-level data. Benefits of the
scanning process include:
>
>
>
>
>
>
Answer documents: student-specific documents are printed at school level
Scoring: teachers mark Constructive Response items only
Data capture: scan answer docs with increased accuracy
Reporting results: real-time results available in ATS after scanning
Improved Instruction: item level data easily available to schools
Audit Trail: improved audit trail will aid in special inquiries.
*RCTs will continue to use the traditional administration and scoring process and will not be scanned.
4
Regents Scanning from A-Z: Process Overview
PrePlanning
Overnight Data
Transfer
STARS
1. Schools receive training and
begin preparations for Regents
Scanning process.
2. Schedule students for Regents and
produce exam invitations in STARS.
Exam
Week &
Scoring
3. STARS data flows to ATS.
Schools test scanner and
generate and print pre-slugged
answer docs in advance.
Logistics
Planning
5. Schools administer exams. Answer documents
for walk-ins are printed on-demand. Scoring
committee marks Constructive Response items
and bubbles in scores on answer documents.
4. Schools conduct Regents week planning activities for
logistics, including assignment of roles, and performing
checklist items in the week before exams are administered.
Overnight Data
Transfer
Attendance
Scanner
6. Schools scan answer docs
for upload to ATS.
ATS &
Scanner
ATS
7. ATS instantly applies scoring key &
conversion chart for scale score.
Results available immediately.
STARS
(& Beyond)
8. ATS passes data nightly to STARS
for report cards and transcripts,
and for reporting to NYSED.
6a. Re-scan docs.
Re-Scanning (Only As Necessary)
5
Timeline of Key Steps at School Level
Dates
Key Step
ASAP
Assessment and/or Data/IT Liaisons: schedule and
conduct training sessions as needed.
ASAP
Data/IT Liaisons: consult schools to ensure appropriate
staff members have access to ATS.
ASAP
Schools clean/test scanner, and ensure adequate supplies
(toner, paper, etc.).
Dec. 21, 2011
Schools have begun to register students for Regents in
STARS.
Jan. 9, 2012
Schools begin to print answer documents in ATS
Jan. 16 – 20, 2012
Regents logistics planning & final preparation
Jan. 23 – 27, 2012
Regents administration
Jan. 24 – Feb. 3
Regents scanning window
6
Step 2: STARS
> Main changes for January 2012:
 New January Regents and RCT codes have been added to all high
school code decks.
> Main tasks:
 Register students for Regents exam, and generate exam invitations.
> Best practices:
 Complete registration process for majority of student before printing
answer documents.
 Only register students expected to take exams (though when in
doubt, err in favor of registration).
 Use “Master Schedule” function for creating a single section, and
“Master Schedule Builder” for multiple sections.
 Networks monitor scheduling progress using STARS Completion
Reports and follow-up with schools requiring attention.
> Middle Schools:
 If a middle school wishes to offer the January Regents to accelerated
students, network data points should contact their cluster academic
policy/data point.
7
Step 2: STARS New Exam Code Deck
> The January 2012 Regents and RCT codes for High School students are
below. Note these codes also indicate the month of exam administration.
Exam
Old
Code
New
Code
New Course Name
Integrated Algebra
MXRE
MXRER
ALGEBRA REGENTS JAN
Comprehensive English
EXRL
EXRLR
ELA REGENTS JAN
Global History & Geography
HXR$
HXRGR
GLOBAL REGENTS JAN
US History and Government
HXRA
HXRUR
US HIST REGENTS JAN
Living Environment
SXRK
SXRKR
LIVINGENV REGENTS JAN
Physical Setting/ Earth Science
SXRU
SXRUR
EARTHSCI REGENTS JAN
8
Step 3: Printing Answer Docs in ATS
> There are four main options for printing answer docs:
1. Not Previously Printed (batch printing for registered students)
Prints batch of answer documents that have not been printed
(RGPR)
2. Print by Student ID (for DOE walk-in student)
Prints an answer sheet for any student, one at a time. This option
may be used for another DOE student taking the exam at your
school. (Use SBIO screen to look up student ID.) (RGPR)
3. Print by Name (non-public) (for non-DOE walk-in student)
Prints an answer sheet for a non-DOE student who is taking the
exam at your school. (RGPR)
4. Reprint Answer Documents
Reprint previously printed docs by student ID or answerdocument page number. (RGRR)
9
Step 3: Printing Answer Docs in ATS (RGPR)
> Use RGPR screen for printing options 1, 2, and 3.
Sorting Options
10
Step 3: Printing Answer Docs in ATS (RGRR)
> Use RGRR screen for printing option 4.
Sorting Options
11
ATS-Generated Answer Documents
Regents Scannable
Old Answer Document
Individualized
bar code
Pre-populated
student info
Student MC
answers
CR teachers’
grades
Scorers’
info
Alt. language &
accommodations
12
Step 3: Printing Answer Docs in ATS
> Best practices for printing answer documents
 Answer documents must be printed on white paper only. Use the
same stock paper as is normally used for attendance forms.
Using colored paper or heavy stock may result in scanning errors.
 After printing batches of answer documents, a “test page” will
print. Scan the test page to confirm the ink quality for a batch.
Trailing test pages are specific to each exam to better diagnose
scanning issues ahead of testing period.
 Do not use incompatible printers (e.g., Lexmark T630). Print
answer docs only on supported printers (Lexmark T640 or later).
 Finalize section/room data in STARS before printing answer docs.
 You may sort print jobs by teacher, section, or room number.
 Remind school team members on when to use RGPR versus
RGRR.
 Answer documents are no longer included in the Regents exam
booklet to be torn out. No generic answer documents may be
used.
13
Step 4: Logistics Planning for Regents Week
> Main tasks:
 Distribute test rosters and page one of answer docs to
appropriate staff in advance.
 Train exam proctors on new procedures and policies.
 Ensure adequate supply of toner, paper, pens/pencils.
 Establish process and task owners for dealing with walk-ins.
(See slide 17)
> Best practices:
 Roles and responsibilities are clear and communicated to all
relevant staff before Regents Week.
 Perform all checklist items.
> Supporting materials: Regents Scanning – Implementation
Checklist
> Support: CFN Assessment Liaison - Amy Reynolds areynolds@schools.nyc.gov 917-9407319, Jerry Lin jlin6@schools.nyc.gov 718-281-7561
14
Step 5: Exam Week & Scoring
> Main tasks:
 Generate on-demand answer documents for “walk-in” testtakers.
 Proctors emphasize proper technique • completely fill in bubbles and erase stray marks;
• remind students to sign student declaration when finished.
> Best practices:
 Students use PENCIL for Multiple Choice, PEN for essays,
DBQ questions, and student attestation.
 Teachers carefully fill in bubbles (using only pencil) when
scoring Constructive Response questions.
 Good preparation will lead to smooth exam administration
> Support: CFN Assessment Liaison - Amy Reynolds areynolds@schools.nyc.gov 917-9407319, Jerry Lin jlin6@schools.nyc.gov 718-281-7561
15
Step 5: Instructions for “Walk-Ins”
Schools should make preparations and assign roles to handle walk-in
students prior to Regents week.
> Inform appropriate staff how to direct walk-ins upon arrival, and appoint staff
to print answer documents using ATS.
> Print answer documents by following the instructions below:
Step
Confirm student’s ID
School notification
Print answer doc in ATS
Enter exam code
DOE Student
Non-DOE Student
Check photo ID
Check photo ID
Check student record against
notification sent from her home
school
Must present signed letter on
school letterhead with exam
specified
“Print by Student ID”* to connect
to student’s DOE record
“Print by Student ID”* if student has
NYC ID from DOE summer school.
“Print by Name” in other cases.
Use appropriate exam code
Use appropriate exam code
Detailed instructions on handling walk-ins is included in the Handbook and FAQ.
*If Student ID is unknown, use ATS SBIO menu to look it up.
16
Step 5: Exam Week & Scoring
> SED Rescoring Policy:
 Beginning in June 2011, schools are no longer permitted to
rescore any of the open-ended questions on any Regents
exam after each question has been rated the required
number of times (as specified in the rating guide), regardless
of the final exam score.
 Specifically, exams receiving a final score of 60-64 or 50-54
are not permitted to be re-scored.
 Principals and other administrative staff in a school or
district do not have the authority to set aside the scores
arrived at by the teacher scoring committee and rescore
student exams.
17
Step 5: Exam Week & Scoring
> Teacher scoring of their own students’ exams:
 For January 2012 exams, teachers can score their own
students’ exams, although NYSED requests principals should
make every effort to prevent teachers from rating their own
students’ responses, where possible.
 Under guidance from the New York State Education
Department, starting in the 2012-13 school year, teachers will
no longer be able to score their own students’ responses.
 Non-electronic distributed scoring pilots will take place in
January and June 2012 to inform the DOE on Regents
scoring approaches that meet the new NYSED requirements.
Contact your Cluster Assessment Point for more information.
18
Step 6 and 6a: Scanning Answer Documents
> Main tasks:
 After notification by DOE, you may use the attendance scanner
to begin scanning answer documents. Notification will be
communicated to CFN Assessment and Data/IT Liaisons, on the
intranet Regents page, and through the ATS login screen.
Documents scanned before notification will not be processed and
must be rescanned.
 Documents with errors in reading bar codes will be reprinted by
the scanner with an error message. Documents with bubble read
errors will be indicated on confirmation sheet(s) and may be
investigated in ATS. Re-scan affected documents as necessary.
 Ensure the scanner is clean; a dirty scanner may result in
inaccurate results. See the Tips for Scanning appendix of the
Regents Scanning Handbook* for instructions on cleaning.
 Review REDS reports for “warnings” noted in Teacher section.
 Print SCRN report, which provides information on scanning status.
> Support: Data/IT Liaison Jerry Lin jlin6@schools.nyc.gov 718-281-7561
* The Handbook is available in ARIS
19
Step 6 and 6a: Scanning Answer Documents
> Main changes for January 2012:
 Addition of a “Did not meet lab requirements” bubble on Earth Science
and Living Environment Regents answer documents. If this bubble is
filled when scanned, it will remove the score; no grade will appear on
report cards or transcripts. The score will not be reported to the State.
• This designation is used only for students who sat (but were ineligible)
for either exam, and should be used only on completed and scanned
documents. Un-schedule students who did not sit for the exam in
STARS and do not scan these documents.
 Updated NYSED policy guidance - in order to preserve answer sheet
quality for audit purposes, answer documents should be scanned only
once except if there is missing or incorrect teacher scores on the
document, or if the “did not meet lab requirements” bubble is filled.
 No score will be generated until the document is completely scanned.
20
Step 6 and 6a: Scanning Answer Documents
Code
Use Case
Method of Entry
Effect on exam
record
Absent (ABS)
Student is absent
Scanned answer
Score is ABS
document or RGTU
Did not
meet lab
requirements
Student mistakenly
allowed to sit for exam
but was not able to meet
Science lab requirements
in time for the exam
Re-scanned
answer document
Score not
reported on
transcript or
report card (data
sent to SIF)
Invalidated
(INV)
Confirmed case of
student cheating
Scanned answer
document (with
principal initials)
Score of INV
Misadministration
(MIS)
School error (ex. missing
accommodations,
document was created for
the wrong student)
RGTU
Score not
reported on
transcript or
report card
Do not scan completely blank (unused) answer documents
21
Step 6 and 6a: Scanning Best Practices
> When scanning 2-page documents, it is recommended to batch all
page 1 documents and scan separately from page 2 documents.
> Schools may scan the first page of two-page documents
immediately, before scoring is complete for page two.
> If only one page of a 2-page document is scanned, it will register in
ATS as an incomplete scan until the second page is processed.
No score will be calculated until both pages of a 2-page
document are scanned.
> The scanner will print a confirmation page indicating pages with
scanning errors, and will reprint pages for which the bar code
could not be read.
> File the scanned batches together with the DFS confirmation page.
This will assist in retrieving papers as needed in the future.
> Scanning is open from 6 a.m. – 9 p.m. on weekdays, and from
8:30 a.m. – 4 p.m. on weekends. Please allow time for the whole
batch to process, and do not attempt to scan large quantities of
documents too close to the daily cutoff for overnight data transfer
to STARS (9 p.m. on weekdays and 4 p.m. on weekends)
22
Step 6 and 6a: Scanned Images
> Scanning captures
an item-level data
string as well as
images of answer
documents (and any
ancillary re-scans)
for an audit trail.
> Red bubbles
indicate responses
recorded by the
scanner.
> Central users have
access to scanned
image capture.
23
Step 7: ATS Reports: REDS (Status Report)
> REDS reports (available immediately after scanning)*:
 Use Document Status Report to ensure that all answer docs
have been properly scanned and to flag warnings.
 Use Item Report to see students’ scores of individual students.
Review report for “warnings/multiples/omits” by using “Include
only W/M/O” option.
 Use Item Distribution Report to inform decisions on present and
future instruction.
> REDS report changes:
• Scores of Misadministration, Did not meet lab requirements,
Exam invalidated, and Absent will all override scaled score
from responses.
• Score of “INC” will be shown when only one page (of 2-page
answer doc) has been scanned.
> Given the length (and width) of the REDS report, it is best viewed
when printed, as all data may not be visible on the ATS screen.
Schools may download ATS reports into Excel for further analysis.
* Note: REDS reports populate in real-time. Other ATS reports (e.g., RCOS) are not real-time.
24
Step 7: ATS Reports: REDS (Status Report)
> Report uses:
 Displays score, accommodations, warnings, for each student.
 Shows summary of documents scanned, outstanding, and warnings.
 Use report to track outstanding unscanned and incompletely scanned
document.
Scores
Scan-specific info
Accom.
Document statuses
25
Step 7: ATS Reports: REDS (Item Report)
> Report uses:
 Displays line-item data for each student.
 Shows specific questions to review (warnings).
 Use to flag Teacher section questions to investigate (e.g. omits, multiples).
STUDENT SECTION
TEACHER
SECTION
Correct answers
Incorrect answers
Multiple answers
CR raw scores
26
Step 7: ATS Reports: SCNS (Scan Statistics Report)
> Report uses:
 This report provides information on scanning status including the number of
pages scanned, accepted, and rejected, and
 the total number of times scanned, with and without errors.
27
Step 7: ATS Reports: RADI (Item Dist. Report)
> Report uses:
 Shows school-level aggregated data for each question.
 Use data to flag specific subject matter and question types for instruction.
‘*’ denotes correct answer
28
Step 7: ATS Reports: RGTU (Update Regents Answers)
> The RGTU screen is used to update student and teacher responses not
accurately picked up by the scanner, including incomplete erasures,
selections lightly bubbled in, or stray marks.
 Process:
• After a school confirms a scanner error, the school will use RGTU to update
individual student or teacher section responses.
• Changes made in RGTU that do not cause a record to move from a failing score
to a passing score will be processed automatically (e.g. an 80  82 or 49  52).
» Requested changes that cause a record to move from a failing score to a
passing score will be held for review and approval by DAPS, and will cause
the record to be locked until the document status is resolved.
• If a school attempts to rescan a locked record, the scan sheet will be
successfully scanned and sent to ATS—but ATS will reject the record. The error
will appear on the SCNS report, and the record will not be updated in ATS
• Principals will be able to review the status of requests in ATS. Most requests will
be reviewed within 48 hours unless further information is needed.
 Appeal process
• Principals (and Data/IT Liaisons) will be directly notified via email if a change
requested has been denied, or if further information is needed to justify the
change. Assessment and/or Data/IT Liaisons will work with DAPS to clarify any
such requests.
29
Step 7: ATS Reports: RGTU (Update Regents Answers)
> Best practices for use of the RGTU screen:
 Do not attempt to use the RGTU screen on an incompletely scanned
answer document.
 Use RGTU for scanning errors, such as stray marks or lightly bubbled
responses. Do not erase or re-bubble and then rescan any student
responses.
 Under no circumstances may teachers or administrators mark, erase, or
otherwise modify any part of the student section of the answer document.
 If a teacher fails to properly record a score in the Teacher section, then the
RGTU screen should be used to correct the error. If a Teacher section
answer is omitted, the answer document should be bubbled in with the
correct response and then rescanned.
 In the event a document cannot be scanned, schools will need to recreate
the student answer document and scan (and retain the original) in order to
access the RGTU screen.
 Schools may mark an exam record as misadministered on this screen.
 Each school must retain a written explanation of each change in RGTU for
audit and review purposes.
30
Step 7: ATS Reports: RGTU (Update Regents Answers)
> Note corrections to Student and Teacher sections are made
separately in the RGTU screen
31
Step 7: Score Verification
> Main tasks:
 Although optional, schools should verify a sample of scanned
results for accuracy - approximately 5% of answer documents.
 Pursuant to guidance from NYSED, there is no re-scoring of
any scanned Regents exam.
> Best practices:
 Two people at school level perform data verification
 The Manual Scoring Data Validation Form is no longer used.
> Support: CFN Assessment Liaison
32
Step 8: Accessing Results in STARS
> Main changes from June/August 2011:
 Student records with either misadministration or the “Did not meet lab
requirements” answer document bubble filled in will not have scores
reported to STARS, the transcript system, or SED.
> Main tasks:
 On the day after answer documents are scanned, results will be
available in STARS (for non walk-in students) and on transcripts (for
all students), provided that scanning was completed by 9 pm the
previous day.
> Best practices:
 Any future score changes (through re-scanning, DOE answer key
alteration, etc.) will be applied automatically.
 Data flows from ATS to STARS automatically; no manual data entry
is required. Results will be automatically applied to transcripts and
report cards from STARS.
> Support: CFN Data & IT Liaison
33
Turn-Key Training: CFN & School Checklist*
Main Action Items
CFNs coordinate trainings with schools
Schools begin to schedule students for Regents in STARS
Timeline
ASAP
By Dec. 21
Schools utilize demo environment to run mock exam process
Dec. 12 – Jan. 6
Schools make preliminary preparations (check toner, paper supplies)
Jan. 2 – 23, 2012
Schools print answer documents for majority of students
By Jan. 16
Schools establish & communicate Regents week plan (roles, logistics)
and notify and train relevant staff.
Jan. 16 – 20
Schools administer exams, scan documents, and conduct data
validation.
Jan. 23 – 27
*Refer to Implementation Checklist for a complete list of tasks to be performed by schools.
34
Turn-Keying: Demo Environment
A demo environment is available for training purposes. Schools
are encouraged to use this to prepare for Regents Week and to
gain an understanding of the required processes.
> The demo environment is available from December 12, 2011 –
January 6, 2012. After January 6, the demo period will end and all
demo data will be removed.
> Detailed instructions on how to access and use the demo
environment are available through a demonstration script. You will
use your existing ATS username and password.
> Demo scope:
ATS
STARS
STARS
ATS
(Scheduling)
(Scanning &
Reporting)
(Printing)
(Report Card &
Transcripts)
DEMO
> We encourage schools to take advantage of the demo
environment to gain familiarity with the new processes and to help
ensure smooth Regents administration.
35
On-Going Support & Resources
The following resources are available to support implementation
and execution of Regents scanning.
> Documentation and updates:
 ARIS: or: http://www.bit.ly/regentsscanning
> Technical support (STARS & ATS):
 CFN Data & IT Liaison
> Scanner and printer:
 DIIT Help Desk, 718-935-5100
> Overall process support:
 Borough Assessment Implementation Directors (BAID)
> Regents week emergency inquiries from CFNs*:
 Christopher Sandor (csandor2@schools.nyc.gov)
 Patrick Purcell (ppurcell@schools.nyc.gov)
> Kirby Fields – Office of School Support (kfields6@schools.nyc.gov)
*Please do not refer school users directly to Central without first attempting to resolve the issue.
36
Any Questions?
CFN 201 Support:
CFN Assessment Liaison - Amy Reynolds
areynolds@schools.nyc.gov
917-940-7319
Data/IT Systems - Jerry Lin
jlin6@schools.nyc.gov
718-281-7561
37