Overview of the 2012-2013
Alternate Proficiency Assessment
Administrator/Test Coordinator Training
2012-2013 NJ APA Administrator Training
1
Administrator and Teacher Training
This session is only for APA administrators.
 Districts are responsible for teacher training.

◦ Teacher training modules are available on the Pearson website
http://pem.ncspearson.com/nj/apa
◦ An administrator must ensure that all teachers participate in the
training sessions.
◦ Teachers may be trained immediately, but every effort must be taken
to ensure all teachers are trained prior to October 1, 2012.
◦ If a student enrolls after October 1 and a teacher needs training, train
the teacher as quickly as possible. The modules will be available all
year, though it is recommended that you download these modules and
retain them in your district.
2012-2013 NJ APA Administrator Training
2
Agenda














APA Participation pages 4 – 22
Student Transfers, Medical Leave, and Excessive Absenteeism pages 23 – 31
Administrator Responsibilities pages 32 – 34
Material Survey, Pre-ID data, record changes, etc., pages 36 – 45
APA Test Design pages 46 – 80
Common Errors and APA Revisions pages 81 – 84
Scoring Rubric Information pages 85– 131
Sample Entry from 2011-2012 pages 132 – 140
Professional and Ethical Responsibility When Compiling and Submitting APA
page 141 – 146
Score Reports and Accountability pages 147 – 156
Administrator APA Review and Other Tasks pages 157 – 167
Request for Teacher /District Assistance pages 168 - 169
Contact Information pages 170 – 174
Winter Training Materials page 175
2012-2013 NJ APA Administrator Training
3
What is the Purpose of the APA?

To measure performance of students with the most
significant cognitive disabilities on the CCCS
– the knowledge and skills that ALL New Jersey students are
learning

To ensure that ALL New Jersey students are
included in the accountability system that is
required by the No Child Left Behind Act
2012-2013 NJ APA Administrator Training
4
What is the APA?


An alternate assessment that measures
achievement of CCCS specifically designed for
students with the most significant cognitive
disabilities.
A portfolio of student work that demonstrates a
student’s performance of knowledge and skills
that are LINKED to grade level knowledge and
skills.
◦ This is a requirement of the USDOE.
2012-2013 NJ APA Administrator Training
5
Who are the Students who Participate
in the APA?


Students with the most significant cognitive disabilities
Decisions about who participates in the APA are not
based on:
◦ Student’s disability category
◦ Student’s educational placement
◦ On which assessment the student is most likely to
score highest
2012-2013 NJ APA Administrator Training
6
Participation Decisions

Students with the most significant
cognitive disabilities will participate in:
◦ The APA in one/some content area(s)
and the general assessment with
accommodations in the other content
area(s) or
◦ The APA in all content areas (LAL,
Mathematics, and Science).
Note a student may not take the general
education assessment and the APA
assessment in the same content area.
2012-2013 NJ APA Administrator Training
7
Determining Who Participates
In The APA

Decisions are made:
◦ By the IEP team
◦ Separately for each content area assessed by the
general statewide assessment
◦ Using information about the student and the
assessments
 Test Specifications/Sample Test forms
 Present Levels of Academic Achievement and Functional
Performance/Goals and Objectives
 Student Work Samples
2012-2013 NJ APA Administrator Training
8
Participation Guidelines



Information on APA participation guidelines is
provided in your training packet.
This checklist must be completed in order to
determine which students are eligible for the APA.
An administrator must sign the form and include
it in the APA binder.
◦ It is usually the sending school administrator, but if both
sending and receiving school agree on the answers to the
questions, then a receiving school administrator may sign
the form. A teacher may not sign the form.

Access to the Directory of Test Specifications and
Sample Test for the general assessment is
necessary in order to complete the checklist
correctly.
2012-2013 NJ APA Administrator Training
9
Participation Criteria Checklist
1.
Is the nature of the student’s cognitive
disability severe?
◦ The student’s general level of cognitive functioning is
significantly below the grade level expectations for his or
her same aged peers and the student requires
substantial modifications to the grade level curriculum.
◦ The student requires more intensive direct instruction
than his or her peers to acquire knowledge and skills in
the grade level curriculum.
◦ The student requires more extensive systems of support
in order to participate in the grade level curriculum.
2012-2013 NJ APA Administrator Training
10
Participation Criteria Checklist
2.
Is the student’s cognitive disability so severe that the
student is not receiving instruction in any of the
knowledge and skills measured by the general
statewide assessment?
◦ While all students are to be instructed in the CCCS
for their grade level, students with more significant
cognitive disabilities typically receive instruction in
the knowledge and skills at a less complex level than
that measured by the general statewide assessment.
◦ Upon reviewing the sample test forms, if the student
is NOT receiving ANY instruction at the same skill
level, then the answer to this question is YES.
2012-2013 NJ APA Administrator Training
11
Participation Criteria Checklist
3.
Is the student’s cognitive disability so severe that the
student cannot complete any of the types of questions
on the general statewide assessment in the content
area, even with accommodations and modifications?
◦ Upon reviewing the sample test forms and questions
for each content area of the general assessment,
along with the state approved testing
accommodations and modifications, if it is determined
that the student is NOT able to complete any types
of questions even with accommodations and
supports, then the answer to this question is YES.
2012-2013 NJ APA Administrator Training
12
Participation Criteria Checklist
4.
Is the student’s Individualized Education
Program (IEP) aligned with grade level CCCS
through modified expectations?
◦ The student is receiving grade level instruction at a
significantly modified and less complex level than their
grade level peers.
◦ The student may need more extensive systems of
support, as identified in the IEP, in order to access the
curriculum.
2012-2013 NJ APA Administrator Training
13
Participation Criteria Checklist
(continued)



The answer to all four questions
must be YES in order to assess the
APA for a content area.
A signed copy of the checklist must be
placed in the inside front pocket of the
APA binder.
Usually the Sending district administrator
signs the form. However, if the Sending
and Receiving district agree on the APA
status, the Receiving district administrator
may sign the form.
2012-2013 NJ APA Administrator Training
14
What Must Be Written in Each
Student’s IEP?
If the student is taking the general assessment, list the
assessment and the accommodations/modifications (from
approved list) that will be provided.
 If the student is not taking the general assessment in one
or more content areas, include a rationale for this
decision.
 If the student is not taking the general assessment for
LAL, Math, and/or Science, list that he/she will be
participating in the APA.
 IEP model forms and information can be found on the
DOE website
http://nj.gov/education/specialed/iep_form_ann.pdf

2012-2013 NJ APA Administrator Training
15
Assessment and CCCS Materials


Additional copies of all APA materials, including the
teacher training materials, may be obtained on the
Pearson website: http://pem.ncspearson.com/nj/apa
Copies of the Core Curriculum Content Standards,
test specifications, and sample tests for the general
assessments can be obtained on the DOE website:
www.state.nj.us/education
◦ Copies can be ordered for a fee by calling the DOE
publications office at 609-984-0905.
2012-2013 NJ APA Administrator Training
16
What Grades and Subjects Are Assessed?



Students in grades 3 – 8 and 11 must be assessed
in language arts literacy and mathematics.
Students in grades 4 and 8 must also be assessed
in science.
Students in high school must be assessed in
science the year they receive instruction in biology
(grade 9, 10, 11, or 12).
◦ High school students may have an APA submitted for
science in a different year than the year the APA is
submitted for language arts and math.
2012-2013 NJ APA Administrator Training
17
Grade 11 and 12 Students



If a senior is new to the state this year and has not
participated in either the APA or the HSPA, the IEP
team must determine which assessment is
appropriate and the student must participate in this
assessment.
Students who were juniors last year and should
have participated in the APA, but did not, must
participate in the APA this year whether they are
still in 11th grade or are now in 12th grade.
Students in high school do not need to retake the
APA if the test result was Partially Proficient. Score
report results on the high school APA do not affect
the ability to graduate.
2012-2013 NJ APA Administrator Training
18
Retained Students

If a student has been retained in a grade due
to poor academic performance, the student
must be reassessed in that grade.
◦ Students who remain in the same assigned grade twice
due to reasons not related to academic performance
must be assessed only once in that assigned grade,
except for the circumstances described on the next
slide.
 Example of reason not related to academic performance: student is kept in
the middle school an extra year because it was determined that the student
was not yet ready for the high school environment (socially, structure of new
large building, etc.)
2012-2013 NJ APA Administrator Training
19
Retained Students

If a student received a score report with a Void Code
rather than a Proficiency Level in a content area, and
the student is retained in that grade, the student must
participate in the APA again in the same grade.

Also, if a student in 11th grade received a Void Code in
language arts literacy and/or mathematics, the student
must be assessed again in these content areas, either in
11th grade if they remain in that grade, or as a 12th
grader.
2012-2013 NJ APA Administrator Training
20
Assigned Grade Level

The grade level of the student must be marked on the
Demographic Scan Sheet that is included in the portfolio.
This is the assigned grade level used during scoring of the
APA.
◦ If a record change for the student’s grade level is submitted in April
2013, the APA assigned grade level becomes the grade level provided
during record changes.

The assigned grade level will be matched against the grade
level of the CPI Links selected for assessment during the
scoring process.
◦ If the grade level of the CPI Links does not match the assigned grade
level provided during data collection, the entry will receive a zero
score.
 Please note that students in high school (grade 9, 10, 11, or 12) are tested
on the grade 12 CPI Links.
2012-2013 NJ APA Administrator Training
21
LEP LAL Exemption

If a student has entered the United States
after July 1, 2012, AND is currently enrolled
in a language assistance program, this student
is EXEMPT from the LAL assessment.
◦ This exemption is not allowable for high school
students.

The APA scan sheet delivered to districts in
February 2013 must indicate the LEP LAL
exemption.
2012-2013 NJ APA Administrator Training
22
Student Transferred to NJ District from
Out of State after October 31, 2012
◦ Students who require an APA, and enter your
school after October 31, 2012, transferring from
out of state, have missed the cut-off date for the
development of the APA assessment.
 Submit a binder, a completed scan sheet, and a letter from an
administrator that indicates from which state the student
transferred, the date of the transfer, and noting this as the
justification for missing APA evidence.
 The score report will indicate a Void 4, no portfolio evidence, when
the student is a late registrant.
◦ Please note that instruction related to the CCCS must still
occur throughout the year, regardless of the enrollment date.
2012-2013 NJ APA Administrator Training
23
Student Transferred from
One NJ District to Another NJ District

The district that has the student during any part
of a collection period must compile the
portfolio. Once the student transfers within NJ,
the district must send the portfolio to the new
district. The new district must complete the
portfolio and submit it for scoring.
◦ If a student is enrolled in a school or district after July
1, 2012, the administrator may bubble on the scan
sheet Time In School less than one year. This will
remove the student’s results from the Accountability
report.
 Note this is different than last year and is based on the
accountability waiver calculation.
2012-2013 NJ APA Administrator Training
24
Student Transferred from
One NJ District to Another NJ District



If the student moved to another district you must send
the portfolio to the district where the student is newly
enrolled.
As long as a student is in your district, even for a limited
time, you must begin instruction of the CPI Links, and
begin collecting APA evidence. Be sure to transfer the
APA materials to the “new” district in a timely manner.
You, the “former” district, do not need to submit a binder
or letter to the APA program since the “new” district will
send in the portfolio.
.
2012-2013 NJ APA Administrator Training
25
Students Leaving the State
Please note that if a student transfers out
of New Jersey prior to the end of the
second collection period (February 15,
2013), the portfolio does not need to
be submitted for scoring.
 It may be sent to the out-of-state school
for use when reviewing the student’s
educational program, or given to the
parent

2012-2013 NJ APA Administrator Training
26
Medical Emergency
• Medical emergency is the occurrence of a severe medical
or psychiatric condition or episode which requires
medical attention or supervision during which time the
student is not able to participate in state assessments.
• APA students who are not receiving instruction
due to a medical emergency may be eligible for a
Void Code due to Medical Emergency when there
is not enough APA evidence.
• APA uses the term extensive sick leave/hospitalization
interchangeably with the term medical emergency. Note
that no other definition may be used when determining a
void due to medical emergency.
• See the requirements for this void on the next slide.
2012-2013 NJ APA Administrator Training
27
Medical Emergency
• When a student is out of school for an extended amount of time
and not receiving instruction due to extensive sick leave or
hospitalization, the portfolio may be eligible to receive a Void 1
(medical emergency code). Eligibility is based only on the
information below.
 If the student is receiving instruction for 10 days or less during a
collection period, and
 The student has an extended hospitalization or leave due to illness
and is not receiving instruction, and
 An official record documenting the student absences is available, then
you must:
 include a letter on school letterhead from an administrator documenting
the medical emergency.
 include a formal district/school record of the student’s attendance and
absences.
 If all of these criteria are met then the portfolio will be voided due to
extended illness during one or both collection period(s).
2012-2013 NJ APA Administrator Training
28
Record of Absences
The record of absences must be a formal
attendance record produced from the district’s
record system. It must indicate all days school
is open and which days school is closed for
holidays, etc. This record must be marked with
the student’s attendance – present or absent.
 You may not submit an informal record of
attendance such as a memo indicating the
student was absent during the year.

2012-2013 NJ APA Administrator Training
29
Medical Emergency Paperwork
If the paperwork submitted in the APA
binder does not include all necessary
components as defined on the previous
slides, a Void for Medical Emergency/Illness
will not be granted.
 An administrator may contact the Office of
Assessments with any questions on this
topic.

2012-2013 NJ APA Administrator Training
30
Absenteeism – Non Medical Emergency
If a student is receiving instruction for 10 days or less in a
collection period (excessive absenteeism), and the reason for
absence is unrelated to a medical emergency/extensive illness,
then the school administrator should contact the Office of
Assessments.
 If it is determined that a Void for No Evidence is appropriate,
two things must be submitted by an administrator in the APA
binder:
◦ a letter on letterhead indicating the reason for the excessive
absenteeism (parent took student out of country, etc.)
◦ a formal record of the student’s attendance/absences
produced from the district’s record system.

2012-2013 NJ APA Administrator Training
31
Test Coordinator’s Main
Responsibilities

Ensure APA participation guidelines are used correctly by IEP
teams.
◦ Ensure IEP teams have all of the necessary information about the
general assessments and the APA participation guidelines clarifying
questions to ensure the decision is made based on the requirements.

Ensure all APA teachers complete the teacher training
modules and review all training materials including the
Procedures Manual and Content Guide Five Item Samples
document.
◦ Have the teachers sign in to your district training and keep a record of
attendance.
◦ Ensure all teachers view all modules.
◦ Ensure all teachers utilize the critical additional training materials listed
above as the materials as a whole deliver the information necessary to
successfully implement the APA.
2012-2013 NJ APA Administrator Training
32
Test Coordinator’s Main
Responsibilities

Periodically review teacher’s APA process, use of
test specifications and selected CPI Links,
instructional lessons, etc.
◦ This review must occur at the beginning of each collection
period, and at least once prior to the end of each
collection period. Reviewing the materials before the cut
off date allows time if necessary for production of
additional/different APA evidence.
◦ Ensure that only the 2012-2013 CPI Link documents are
used. The APA continues to use the older standards until
the other assessments finalize the alignment to the
Common Core Standards and the APA specifications are
revised.
2012-2013 NJ APA Administrator Training
33
Test Coordinator’s Main
Responsibilities





Ensure registration of APA students, review and
update district contact info, and submission of
Student Pre-Identification file prior to the
deadline.
Complete Demographic Scan sheet providing
accurate data
Complete final portfolio review prior to shipping
Ensure timely shipment of portfolios
Review APA record change rosters and
correct/revise any inaccurate data
2012-2013 NJ APA Administrator Training
34
Vendor Transition
The APA program is transitioning to a
new vendor, Questar Assessment.
 This transition will begin in September.
 Details and deadlines related to student
registration, Pre-ID submission, the new
Online Entry Cover Sheet, etc., will be
forthcoming.
 We appreciate your patience and will
send details on these topics regularly as
they become available.

2012-2013 NJ APA Administrator Training
35
Materials Survey – Registration of APA
Students and District Information

Register the number of APA students in your district
no later than (date to be determined).
◦ Failure to register your students will result in a failure of
delivery of APA materials to your district.
◦ Survey should be completed online (see memo).
◦ Do not register out-of-district students that are assessed in
New Jersey. The receiving district will register these children.
◦ Register students that are sent by your district to an out-ofstate placement as if they are in their home school.
◦ If no APA students this year, then indicate 0 (zero) for the
applicable grade level.
◦ Confirm/revise district address, emails, and contact
information to ensure accurate deliver of information.
2012-2013 NJ APA Administrator Training
36
Online Registration – Material Survey
The online material survey is not yet available. A
memo will be sent to districts when the website is
ready.
 https://nj-servicepoint.questarai.com/Login.aspx

is the Questar website address for use in the future.

Log on information including User IDs and Passwords
will be provided to your Chief School Administrator
(CSA)

If your CSA does not receive this information by
October 30, call Questar customer service at 1866-690-4733 or send an email to
NJAPACustomerSupport@questarai.com
2012-2013 NJ APA Administrator Training
37
Students Sent to an Out-of-State School

The Sending District is responsible for communication to
the out-of-state placement.
◦ This includes providing training materials to the school and ensuring
that the CCCS is instructed and assessed.
◦ This includes ensuring that the APA is in process, evidence is being
collected, and that the APA will be completed and submitted to you
prior to the return portfolio shipment deadline.


Portfolios must be sent from the out-of-state school to the
sending district prior to February 15, 2013. The sending
district includes these portfolios in the district’s shipment.
Registration of these students, completion of scan sheets,
record changes, etc., is the responsibility of the sending
district.
◦ Include these students in the sending school APA enrollment counts.
2012-2013 NJ APA Administrator Training
38
Electronic Pre-ID File Submission

Pre-ID File
◦ This is the first opportunity to submit accurate
student information/data.
◦ Administrators will need to provide such info as
student birth date, SE classification code, grade level,
etc.
◦ Student demographic information must be submitted
electronically on the Questar website. Please be
patient while the website is developed.
 Pre-ID Instructions will be sent to your Chief School
Administrator.
 Pre-ID information will be printed on the demographic scan
sheet in lieu of gridding the information.
2012-2013 NJ APA Administrator Training
39
Electronic Pre-ID File

Last date to submit pre-ID files is: Date to
be determined.
 It is not advisable to wait until the last day since you
may find that the file is not accepted due to data
errors.
 Within the submission window you may submit a
revised file when necessary.
 Pre-ID information will be printed on scan sheets for
those participating districts and sent with the Return
Materials shipment in February 2013. Blank scan sheets
will also be provided.
 Errors/omissions on the Pre-ID label must be corrected
by gridding the info on the scan sheet.
2012-2013 NJ APA Administrator Training
40
Scan Sheet



Student demographic scan sheets, along with directions for
the completion, are shipped to districts in February.
This is the second opportunity for districts to submit
accurate student information/data.
A county-district-school code (CDS) must be provided for
the school the student is attending. This is either a Home
(neighborhood) school or a Receiving school.
◦ If the student is in a receiving school, a CDS code for the sending
school must also be provided. The sending school must provide this
code to the receiving school for inclusion on the scan sheet.
◦ The home or receiving school code is pre-gridded on the scan
sheet.
 APA will begin to use the most up-to-date CDS codes for the private and
specialized, separate schools beginning this year. Please ensure you know your
current code which is also being used for NJASK and HSPA. We will no longer
be using CDS codes that start with “90”.
2012-2013 NJ APA Administrator Training
41
Scan Sheet

If a Pre-ID file was submitted, check the
preprinted information on the scan sheet for
accuracy.
◦ You may correct information submitted in the Pre-ID
file by gridding the correct information on the scan
sheet. The gridded information will override the
preprinted information.
 The only exception to this is changes to the student name,
which may only be corrected during the record change
process.
 We must maintain the same exact student name on the file in order to
process the information until record changes. Only at that point can we
make changes to the spelling of the student’s name.
2012-2013 NJ APA Administrator Training
42
Taking General Assessment Field


Detailed instructions are provided in the use
of this field in the Pre-ID directions and the
Scan Sheet directions.
Only code this field when the student is NOT
taking the APA, but is taking the general
assessment (NJASK, HSPA, NJBCT) in a
content area.
◦ Remember, a student may only take the APA
OR the general assessment, not both, for a
content area.
◦ Marking the student as Taking the General
Assessment will Void all results for the APA.
2012-2013 NJ APA Administrator Training
43
Record Change Process
A roster of APA student demographic information is
shipped to all districts in April 2013. (Will verify date with
new vendor and confirm in Record Change Memo.)
 For student’s who remain in their Home (neighborhood)
school, the “Home” district must review and submit roster
changes.
 For student’s who have both a receiving and sending school,
both districts must review the record change rosters for
accuracy.
 Beginning this administration, the Sending School, not the
Receiving School will submit the record changes to the
vendor.

2012-2013 NJ APA Administrator Training
44
Record Change Process

The Sending School/Home School is responsible for
correcting all student information that was provided on
the scan sheet.
◦ The Receiving School must also review the roster, as errors (e.g. ,
CDS code, demographic information) may impact the student score
reports and accountability reports.
 Corrections and revisions to the data must be submitted to the
student’s Sending School at least several days prior to the record
change deadline. The Sending School must submit these changes to
Questar during the record change timeframe.

This is your final opportunity to correct/revise the
data. This includes the field Taking the General
Assessment. The final record change roster data will
be used for the accountability report. No changes
will be made after this time.
2012-2013 NJ APA Administrator Training
45
APA
What will this portfolio look like?
What are the required components?
2012-2013 NJ APA Administrator Training
46
What is the Format of the APA?

Portfolio: Paper based, submit in a 3-ring binder
◦ Binders are mailed to districts in October, based on the
APA registration counts the district administrator
submitted.

Content Areas: Mathematics, Language Arts
Literacy, and Science
◦ Science is included only in grade 4, 8, and high school.
2012-2013 NJ APA Administrator Training
47
What is the Format of the APA?
Entries relate to content standards, grade-level
CPIs, and CPI Links. Each content area will have
four entries, each one reflecting the assessment of
one CPI Link.
 Evidence documents educational instruction and
student performance of skills. The instructional
activity used at the beginning and at the end of the
CPI Link instruction is submitted as the
assessment evidence.

2012-2013 NJ APA Administrator Training
48
Test Design
APA Portfolio
Table of Contents
Language Arts
4 Entries
Mathematics
4 Entries
Science
4 Entries
NOTE: NO entry should contain more than four pieces of evidence.
This graphic represents a student who is being assessed in all three subjects with
the APA. Not all portfolios will include all three subjects.
2012-2013 NJ APA Administrator Training
49
Test Design Example – Language Arts
(Same Structure for Each Content Area)
APA Portfolio
Table of Contents
Language Arts
Entry 1
Language Arts
Entry 2
Language Arts
Entry 3
Language Arts
Entry 4
Entry Cover Sheet
reflecting 1st Standard,
Strand, CPI, and CPI Link
Entry Cover Sheet
reflecting 2nd Standard,
Strand, CPI, and CPI Link
Entry Cover Sheet
reflecting 3rd Standard,
Strand, CPI, and CPI Link
Entry Cover Sheet
reflecting 4th Standard,
Strand, CPI, and CPI Link
2 pieces of evidence
2 pieces of evidence
2 pieces of evidence
2 pieces of evidence
NOTE: NO entry should contain more than four pieces of evidence. An entry
containing more than four pieces of evidence will result in zero scores for all
dimensions.
2012-2013 NJ APA Administrator Training
50
When Do I Collect Evidence?
Activity
Collection Period
Initial Activity
September 4 - November 16, 2012
Final Activity
December 10, 2012 - February 15, 2013
 Dates on the evidence must include month, day, and year.
 Be sure to plan early in the year when you will instruct and
assess each CPI Link and content area.
 All students will need to be assessed on an initial activity within the
correct time frame above, even if the majority of instruction will not occur
until the winter.
2012-2013 NJ APA Administrator Training
51
Four Entries per Content Area
Entry One
Based on
Standard and
Strand 1
Entry Two
Entry Three
Entry Four
Based on
Standard and
Strand 2
Based on
Standard and
Strand 3
Based on
Standard and
Strand 4
Select CPI &
CPI Link
Select CPI &
CPI Link
Select CPI &
CPI Link
Select CPI &
CPI Link
Provide instruction
& document
activities and results
Provide instruction
& document
activities and results
One Piece of
Evidence
from
September 4 November 16, 2012
One Piece of
Evidence
from
September 4 November 16, 2012
One Piece of
Evidence
from
December 10, 2012February 15, 2013
One Piece of
Evidence
from
December 10, 2012February 15, 2013
2012-2013 NJ APA Administrator Training
Provide instruction
& document
activities and results
Provide instruction
& document
activities and results
One Piece of
Evidence
from
September 4 November 16, 2012
One Piece of
Evidence
from
September 4 November 16, 2012
One Piece of
Evidence
from
December 10, 2012February 15, 2013
One Piece of
Evidence
from
December 10, 2012February 15, 2013
52
Page Numbering


A table of contents that references page
numbers in the portfolio must be included.
Page numbering for evidence that is longer than
one page should be numbered using a combination
of page number and letters (i.e., 2, 2a, 2b, etc.)
 For example, a student was given a reading
comprehension test (matches the CPI Link) with
8 test items. The test was two pages long, but it is
one activity/piece of evidence. The pages might be
numbered page 3 and 3a.

Entry Cover sheets get their own page numbers.
2012-2013 NJ APA Administrator Training
53
Entry Cover Sheet

The vendor will build an electronic entry
cover sheet that will allow teachers to use
drop-down boxes to populate most of the
fields on the form. This form will then be
printed from the web.
◦ Teachers must still type/write in their activity
descriptions on the form.
◦ This form will not be ready until later in the fall.
2012-2013 NJ APA Administrator Training
54
Entry Cover Sheet
Teachers should use/complete the old
version of the cover sheet as a place
holder until the new form is available.
 The Procedures Manual and Pearson’s
website has the old style Entry Cover
Sheet for 2012-2013.
 Once the new online form is available
teachers should use the old form to assist
in transferring the information to produce
the new Entry Cover Sheet.

2012-2013 NJ APA Administrator Training
55
2012-2013 NJ APA Administrator Training
56
Contents of the Entry
Each entry requires documentation (evidence) of
student performance of the skill stated in the CPI
Link.
 Two pieces of evidence is the requirement for an
entry.
 Includes two different activities.

◦ One piece of evidence from the initial activity, and one
piece of evidence from a different, “final” activity.

Evidence may come from classroom work,
community-based settings, or other places where
student is working on grade-level CPI Links.
2012-2013 NJ APA Administrator Training
57
Contents of the Entry (continued)
 The first piece of evidence documents the initial, or very
early, performance of the CPI Link. This evidence must
capture what the student does/does not know related to
the CPI Link.
 This evidence must reflect the entire CPI Link in one
activity, and be collected between September 4 and
November 16, 2012.
 In order for the CPI Link to be eligible for the student’s APA
entry, the accuracy score on the activity can not be above
39%.

Additional information on scoring the evidence is included later
in the training.
2012-2013 NJ APA Administrator Training
58
Contents of the Entry (continued)
 The second piece of evidence documents final
performance of the CPI Link. This evidence should
capture what the student has learned related to the CPI
Link.
 This evidence should assess the same CPI Link using a
different instructional activity, and be collected between
December 10, 2012, and February 15, 2013.
 More information on what makes a different activity is
found in the Teacher Training Modules.
2012-2013 NJ APA Administrator Training
59
Evidence Requirements


Evidence must document the entire CPI Link
noted on the Entry Cover Sheet.
Evidence should relate only to the CPI Link, not
assess other skills as well.
◦

Providing evidence that documents multiple skills
(skills not part of the selected CPI Link), or less than
the entire CPI Link, will result in a low scoring entry.
Have at least 5 items/questions/task elements
included in each activity
2012-2013 NJ APA Administrator Training
60
How are CPI Links Related
to the CCCS?


Grade-level CPIs are the building block.
Content Centrality establishes what concept must be
assessed.
◦ Intent/Essence of the CPI

Performance Centrality directs the level of skill
statement.
◦ Complexity/difficulty
 APA CPI Link can be less complex than the grade-level CPI
and/or less difficult
 Three types of links–Matched, Near, and Far–reflect the
degree of complexity/difficulty when compared to the
grade-level CPI.
2012-2013 NJ APA Administrator Training
61
CPI Link
•Grade Level
CPI
•Far Links
•Links
•Links
•Links
•Essence
Links
CPI 3.1.5G13
Recognize
language
in text (e.g., simile, metaphor, personification,
•Matched
Links figurative •Near
alliteration)
Essence of the CPI: Identify figurative language
Matched Link
Near Link
Far Link
 Find examples of three
 Match at least two
 Identify examples of similes
different types of
different types of
 Identify examples of
figurative language in a
figurative language to
personification
text*
its type*
 Identify examples of
alliteration
 Identify examples of
metaphors
2012-2013 NJ APA Administrator Training
62
Using the CPI Links for Instruction
CPI Links should not be used in isolation.
Read the Standard and Strand for the overall
concept.
 Read the CPI and Essence.
 Make sure instruction is within the Essence of
the Standard, Strand, and CPI.


◦ Evidence of instruction on CPI Link will be scored not only on the
discrete Link but also within the Essence of the standard, strand,
and CPI
◦ For example, 3.1.3G3 is a reading standard with a Matched Link
that states “Interpret information from graphs, charts,
and/or diagrams found in text.” The evidence must assess the
student’s interpretation of graphs/charts/diagrams within the
context of reading text and not within the mathematical context
of data analysis.
2012-2013 NJ APA Administrator Training
63
Evidence Must

Assess ONLY one CPI Link
◦ Remember, when reviewing the CPI Links in the APA
Procedures Manual, a CPI Link is only one bullet.
◦ Once the type of link to assess a student is determined
(Matched, Near, or Far Link), then select only one bullet
in that column.
Assess the entire CPI Link providing the
necessary systems of support
 Include the student’s response to the test item

◦ Must be completed in the student’s mode of
communication
◦ Ensures that the student can review his own work
 This is a good way to “test” if the evidence documentation
includes the necessary components, but it is not a requirement
that you document the student’s actual review of the work.
2012-2013 NJ APA Administrator Training
64
Acceptable Format for the Evidence
• Evidence must show the test items and
student responses. Acceptable evidence
includes
– Samples of graded student work
– Series of captioned photographs
• Must evidence the student actually performing the
skill
– Snapshot of completed student work that
documents student performance of the skills
contained within the CPI Link
• Must be a photo of a student work product that is
too large or bulky to include in the portfolio (i.e., a
poster, a written piece using objects or textured
cues, etc.)
2012-2013 NJ APA Administrator Training
65
Acceptable Evidence
What else is needed to make evidence
acceptable?
All evidence must include:
• Student’s Name
• Date of Activity Completion using Month, Day, and Year
(must be within the collection windows to be valid)
• The accuracy of each student response
• Use + (plus) for correct and – (minus) for incorrect responses
• The independence level of each response
• I=Independent performance, V=Verbal prompt, G=Gestural
prompt, M=Model prompt, P=Physical prompt (presented from
least to most intrusive)
• More information on use of prompting appears later in the
presentation
2012-2013 NJ APA Administrator Training
66
Acceptable Evidence
What else is needed to make evidence
acceptable?
All evidence must include:
 A minimum of 5 items that assess the CPI link
◦ A piece of evidence must include at least 5 test items that
assess the skills contained within the CPI Link.
 For example
 Ordering decimals would require 5 sets of numbers with which
to order—not just that 5 decimals were ordered.

More information is found in Modules IV, V, and VI of the
online training, in the 2012-2013 Procedures Manual, and in
the CPI Content Guide and Five Items Resource.
2012-2013 NJ APA Administrator Training
67
A Minimum of 5 Test Items

For CPI Links, an activity must include at least 5 test
items.
◦ For example
 A student will answer 5 comprehension questions about a
completed job application.
 A student will use multiplication to check the answers to 10
division problems.
 NOTE via an example: If a student is given 7 pictures, and is
asked to circle the plants, they must also be asked to cross out
the pictures that are not plants. They may not have items that
do not require a response and then score it as if the item was
correct. Also, there must be enough plant pictures to test the
skill, and a balance of right to wrong options.
2012-2013 NJ APA Administrator Training
68
Rule 5 : Example Five Items

When the Link has an “and” in it, 5 items across the
skills are required rather than 5 items for each part
of the Link.
◦ For instance, 4.2.7E1 Near Link “Calculate the area of a
triangle, rectangle and square”
2012-2013 NJ APA Administrator Training
69
Acceptable Evidence

Student work samples serve as the best evidence to
document all of the necessary components of the
assessment activity.
◦ This type of evidence includes the test
items/questions, response options (if applicable), and
student responses.
◦ Teachers may need to scribe the student’s responses
when necessary.

The use of videotape, audiotape, data charts, and
most photographs are not acceptable evidence.
2012-2013 NJ APA Administrator Training
70
Student Work Samples Used as
Evidence May Be

Computer screen printouts of student’s test item and
response after hitting a switch.
◦ Summary printouts where each response is not evidenced is
unacceptable.
An essay written using picture symbols and scored using
a rubric.
 A worksheet completed by the student.
 Drawings to extend patterns.
 Completed job application using name, address, and
other stamps.
 A graph made from ordered pairs.

2012-2013 NJ APA Administrator Training
71
Student Work Samples May Be



A paragraph written using sentence strips
A printout of a sentence written using a
picture writer (i.e., Writing with Symbols,
PicWriter, etc.)
A snapshot of work too large to fit into the
portfolio (a poster, a model, etc.)
2012-2013 NJ APA Administrator Training
72
Acceptable Photographic Evidence

A series of captioned photographs that document the
student performing the skills included in the CPI Link
will be accepted only if:
◦
Each item and the student’s response is clearly present
in each photograph




Must include a description for each photo
Must include running record of the items performed
Must include accuracy and independence information for
each item and include total percent scores for each
dimension
Must meet the requirements for the Universal Scoring
Rules
2012-2013 NJ APA Administrator Training
73
Acceptable Photographic Evidence
(continued)



A photograph of a large or bulky completed project
will be accepted only if the picture captures the
evidence requirements for the activity.
A description of the activity must be included along
with the documentation of the scoring, but the
actual work must be clearly shown in the picture.
The Universal Scoring Rules requirements must be
met.
2012-2013 NJ APA Administrator Training
74
Scribing Work Samples
Acceptable Scribing Technique
Unacceptable Scribing Technique
Uses the student’s communication
mode to document the student’s
answer
Does not use the student’s
communication mode
Allows the student to review her/his
work
Does not allow the student to review
her/his work
Is used to record the student’s
response on a work sample from
eye gaze, a voice-output device
(VOD), Augmentative
Communication Device, reach and
grasp, etc.
Does not show the student
performance of the skill
(observational data)
2012-2013 NJ APA Administrator Training
75
Unacceptable Evidence




Data charts
Single photographs of student working on an
activity
Photographs of student work that could have been
included
Any work that is not completed by the student
◦ Note, the student responses must be marked at the time
of the activity, not at a later point in time, in order to be
acceptable evidence.
2012-2013 NJ APA Administrator Training
76
Unacceptable Evidence
 Has
fewer than 5 items/questions/task elements
 Has no name, date, or incomplete date
 Is scored using anything other than percentages
◦ e.g., ratios (4/5), percent ranges (90-100%), letter
grades (A, B+)
 Has
percent scores that can not be replicated by
scoring staff
2012-2013 NJ APA Administrator Training
77
Evidence
• Universal Scoring Rules must be met on two pieces of
evidence for the entry to be scored based on the
rubric. Otherwise, the entry will receive a score of
zero for all dimensions.
• If any entry does not have at least two pieces of
evidence OR has more than four pieces of evidence
per entry, it will receive a score of zero.
 Two pieces of evidence are all that is required for APA.
2012-2013 NJ APA Administrator Training
78
Universal Scoring Rules
1. The evidence must include the student’s name.
2. The evidence must include the complete date (month/day/year).
 Verify that the dates fall within the appropriate collection period:
 September 4, 2012 - November 16, 2012 for the first piece of evidence
 December 10, 2012 - February 15, 2013 for the second piece of evidence
3. The evidence must be presented in the appropriate format.

The evidence must include a writing rubric when specified in a Writing CPI Link.
4.
5.
6.
7.
The evidence must reflect the student’s mode of communication.
The evidence must include at least 5 items.
The evidence must show student responses for at least 5 items..
The evidence must assess the entire link while connecting to the
essence of the CPI, standard, and strand.
8. Both pieces of evidence must assess the same CPI Link and skills.
9. The evidence must not include more than the skills contained
within the CPI Link.
Note: The USRs pertain to both student work samples, as well as the writing rubric
when required. If the entry does not adhere to all rules it will receive zero scores.
2012-2013 NJ APA Administrator Training
79
Activities – Errors to Avoid
The first piece of evidence should not include a more difficult
application/context of the skill than the final activity/evidence.
Examples of this type of error (a more difficult initial activity) are below:
•
•
•
•
•
•
•
The initial activity has longer test items with more complex steps (e.g.,
open-ended essay questions vs. multiple-choice) than the final activity.
The initial activity has multiple-choice test items where no correct
answers are offered as options.
Fewer additional supports (steps, formulas, pictures, application, calculators,
etc.) are given in the initial activity than the final activity.
Regular grade-level text is used in the initial activity, but summarized text
with picture cues and key details highlighted are used in the final activity.
No explanation of what to do is given for the initial activity, but directions
are given for the final activity. Note: directions should ALWAYS be given
during an activity.
No prompts on the first piece of evidence with no answers written, or all
answers are clearly off topic (answers are not at all related to concept).
The initial activity is not in the student’s mode of communication, but the
final activity is.
2012-2013 NJ APA Administrator Training
80
Common Mistakes found in
2011 - 2012 Assessments
Assessing the skills contained in the CPI Link without
connection to the critical essence, intent or big idea of
the content and concepts of CPI, Strand, and/or
Standard
 Assessing the skills contained within the CPI Link
without correct understanding of the Content or
concepts of CPI, Strand, and/or Standard
 Assessing more than one CPI Link in an entry
 Assessing different parts of a CPI Link in each piece of
evidence
 Assessing only part of the CPI Link

2012-2013 NJ APA Administrator Training
81
Common Mistakes found in
2011 - 2012 Assessments
Failure to mark all test items with accuracy and/or
independence/prompt information
 Failure to give supports on the first activity
 Failure to give prompts, instead marking student
response as “Don’t Know” or “No Response”

◦ For assessment, an item may not be marked as “don’t know” or
“no response.” Teachers must utilize a prompt(s) in order to
assist the student in generating a response.
2012-2013 NJ APA Administrator Training
82
Changes for 2012-2013

Districts must use the 2012-2013 updated
CPI Links when assessing a standard, strand,
and CPI for the APA.
◦ A few CPI Links were revised to make them more clear.
◦ The asterisk following a link is a reminder that there is a
sample test item available in the Five Items document.

Important CPI Link clarifications are
included in the new Content Resource
Guide and Five Items document. Be sure to
read all of the relevant grade-level materials
thoroughly.
2012-2013 NJ APA Administrator Training
83
Content Guide Revision

Some curriculum issues have been identified and
must be avoided in the future. Some evidence
contained errors in the instruction of the skills.
◦ Example: Our Solar System only contains one star, the
Sun. Therefore, stars may not be used as a correct
answer when asking what is in our Solar System.
◦ Example: Teachers are submitting math Symmetry work
sheets that don’t correctly assess the CPI Link that was
selected.
These topics were addressed in last year’s
document.
 These errors will result in zero scores for 20122013.

2012-2013 NJ APA Administrator Training
84
Developing Standards-Based Entries
2012-2013 NJ APA Administrator Training
85
Developing an Entry








Step 1: Select a CPI and one related CPI Link to be assessed.
Step 2: Plan instruction and assessment relating to the CPI and CPI Link.
Step 3: Design activities that will be used to assess the CPI Link.
Step 4: Assess the student to get an initial piece of evidence for APA
purposes.
Step 5: Implement instruction.
◦ Ensure instruction reflects the essence of the CPI, strand, and standard.
◦ Ensure the instruction is age- and grade-level appropriate.
◦ Retain a working folder of instructional activities and classroom-based
assessments implemented between the activities that generate the initial
and final pieces of APA assessment evidence.
Step 6: Determine when evidence can be collected to document the final
instructional assessment of the CPI Link for APA purposes.
Step 7: Based on the student’s accuracy score and level of prompt
information on the “final” activity, determine if additional instruction and
collection of evidence needs to occur for the entry.
Step 8: Review APA evidence to ensure that all information related to test
design requirements are included.
2012-2013 NJ APA Administrator Training
86
How is the APA Scored?


Each portfolio entry is scored based on specific
scoring criteria described in the APA scoring
rubric found in the Procedures Manual.
The criteria are grouped into three categories
called dimensions.
2012-2013 NJ APA Administrator Training
87
APA Scoring Rubric

Scoring Dimensions
◦ Complexity Link
 Match, near, or far link selection
◦ Performance
 Percentage of accuracy when performing skill
◦ Independence
 Percentage of time skill was performed independently
2012-2013 NJ APA Administrator Training
88
Scoring Dimensions
Dimension
0
1
2
3
4
Complexity
Evidence
provided is
unscorable; all
dimensions will
receive a score
of zero
CPI Link was
assessed, but
there are major
flaws in the
evidence
CPI Link is a
Far Link to the
grade-level
indicator
CPI Link is a
Near Link to
the grade-level
indicator
CPI Link is a
Matched Link
to the gradelevel indicator
Performance
Evidence is not
clear or all items
are not marked as
correct/incorrect
Accuracy of
work is
40-59% based
on the final
activity
Accuracy of
work is
60-80% based
on the final
activity
Accuracy of
work is
81-100% based
on the final
activity
Independence
Evidence
is not clear or all
items are not
marked for
Independence/
prompt level
Accuracy of
work is
0-39% based on
the final activity
OR
Second activity
includes more
intrusive prompt
Student completed
items/tasks
independently
0-39% of the time
based on the final
activity
Student
completed
items/tasks
independently
40-59% of the
time based on
the final activity
Student
completed
items/tasks
independently
60-80% of the
time based on
the final activity
Student
completed
items/tasks
independently
81-100% of the
time based on
the final activity
2012-2013 NJ APA Administrator Training
89
Scoring Rubric and APA Forms

The APA Scoring Rubric found on the
previous slide is also found in the
2012- 2013 APA Procedures Manual.
Don’t forget to review the Procedures
Manual, which includes the APA
Scoring Rubric, scoring clarifications,
and other topics.
2012-2013 NJ APA Administrator Training
90
Complexity Dimension
Evaluates the CPI Link assessed and how closely the
complexity and difficulty (Matched, Near, Far) links to
the Core Curriculum Content Standards (CCCS) and
grade-level cumulative progress indicators (CPI).
2012-2013 NJ APA Administrator Training
91
Complexity Dimension
0
1
2
3
4
Evidence
provided is
unscorable; all
dimensions
will receive a
score of zero
CPI Link was
assessed, but
there are major
flaws in the
evidence
CPI Link is a
Far Link to the
grade-level
indicator
CPI Link is a
Near Link to
the grade-level
indicator
CPI Link is a
Matched Link
to the gradelevel indicator
2012-2013 NJ APA Administrator Training
92
Complexity Score of 2, 3, or 4

A score of 2, 3, or 4 is obtained if the CPI
Link is assessed correctly and a different
activity is used for the initial and final piece
of evidence.
◦ A score of 2 is obtained when a Far Link is
assessed correctly.
◦ A score of 3 is obtained when a Near Link is
assessed correctly.
◦ A score of 4 is obtained when a Matched Link
is assessed correctly.
2012-2013 NJ APA Administrator Training
93
What Makes an Activity Different?

Two activities are required when submitting
evidence for an entry.
◦ Both of these activities must assess the same CPI
Link.
◦ The two activities must be different activities in order
to address the Universal Scoring Rules.
An activity is considered to be different than
another when the context of the instruction of
the CPI Link is different.
 An activity is also considered to be different
from another when the application of the skill
in the CPI Link is different.

2012-2013 NJ APA Administrator Training
94
What Makes an Activity Different?
While the skills assessed and the supports
needed to perform the skills remain the
same, an activity is considered different:
◦ With a different context in which the
framework or events within which the student
is demonstrating the skill changes
◦ With a different application in which the
types of questions used, or the way in which the
student demonstrates the skill changes
2012-2013 NJ APA Administrator Training
95
What Makes an Activity Different?
Examples of two activities with different context:
 The student is working on following vertex-edge graphs
1. after whole class introduction and demonstration of vertex-edge graphs, the
student completes a worksheet
2. after having a lesson on flight plans, the student uses a vertex edge graph to
follow flight paths
 The student is working on adding two matrices
1. by completing a worksheet
2. by completing a different worksheet with different problems than used in
the initial activity
 The student is working on writing a cover letter
1. after researching a job of a particular interest, write a cover letter that
would be used when applying for the job
2. after listening to speakers about different careers, write a cover letter for a
job of interest
 The student is working on identifying theme in a given work of fiction
1. by reading The Outsiders and answering multiple-choice questions
2. by reading Flowers for Algernon and answering multiple-choice questions
2012-2013 NJ APA Administrator Training
96
What Makes an Activity Different?
Examples of two activities with different applications:
 The student is working on classifying types of planets
1. by using a graphic organizer
2. by matching
 The student is working on addition and subtraction
1. by completing a worksheet with problems on it
2. by using a Smart Board to complete math problems
 The student is working on writing an essay
1. by using a computer or word processor
2. by using paper and pencil
 The student is working on identifying theme in a given work of fiction
1. by reading a passage of The Outsiders and then answering multiplechoice questions
2. by reading a passage of The Outsiders and then answering short answer
questions
2012-2013 NJ APA Administrator Training
97
Teachers Must Think about the Two
Activities in Conjunction
As you plan for instruction, you should not think
about the initial and final activity in isolation. Things to
think about:



What type of activities might be easier and harder for the
student? Do not use the easier activity as the final activity.
What type of prompt, if any, might the student need on the
final activity? Do not use a more intrusive prompt level on
the final activity than was used on the initial activity.
How can I keep the student engaged in the various
activities? How will the initial activity be different than the
final activity and still engage the student?
2012-2013 NJ APA Administrator Training
98
Complexity Score of 1: Major Flaws
 An entry that demonstrates work in a CPI Link
but has a major flaw will score a 1 in
Complexity.
• A major flaw includes
1. Assessing the same activity for both pieces of
evidence
2. Assessing only part of the CPI Link
– This may be found in the work sample, and both pieces
of evidence must be assessing the same part of the link.
– This may also be found in the writing rubric.
2012-2013 NJ APA Administrator Training
99
Complexity Major Flaw Example (Same Activity)
•9/10/12
•2/11/13
The directions given to the student were “Match herbivores, carnivores, and omnivores
to the type of food they eat.”
2012-2013 NJ APA Administrator Training
100
Complexity Major Flaw Example
(Activity assesses only part of a link)
3.1.8G1 Matched Link: “After reading
newspapers, magazines, or electronic texts,
identify the type of propaganda used and
explain its purpose”
1/10/13
10/1/12
Complexity score would be 1 for only
assessing part of the link.
2012-2013 NJ APA Administrator Training
101
Complexity Score of Zero
When the Universal Scoring Rules are not
adhered to, the entry will receive a score
of zero for Complexity, Performance, and
Independence.
 Please review Teacher Training Module VII:
Universal Scoring Rules for more
information.

102
2012-2013 NJ APA Administrator Training
Performance Dimension
This dimension measures the student’s
accuracy performing the skills
represented in the CPI Links identified
within the portfolio.
2012-2013 NJ APA Administrator Training
103
Performance Dimension
• Each item must be marked as correct (+) or incorrect (–).
• The score for accuracy should be expressed as a percentage.
• Any item for which the student received a physical prompt
should be marked as incorrect (–P).
• The second piece of evidence should not include a more
intrusive prompt than was used on the first piece of evidence.
• The student’s accuracy score on the first piece of evidence should
not exceed 39%.
• The first piece of evidence should not be more difficult than the
second piece of evidence.
• When assessing a 3.2 Writing Link that requires a writing rubric,
there should be feedback on the student’s work explaining
how/why the work was scored in relation to the writing rubric.
2012-2013 NJ APA Administrator Training
104
Performance Dimension
0
1
2
3
4
Evidence is not
clear or all
items are not
marked as
correct/incorrect
Accuracy of work is
0-39% based on the
last activity
OR
Second activity
includes more
intrusive prompt
Accuracy of
work is
40-59% based
on the last
activity
Accuracy of
work is
60-80% based
on the last
activity
Accuracy of
work is
81-100%
based on the
last activity
2012-2013 NJ APA Administrator Training
105
Scoring Student Evidence for
Accuracy

Each item must be marked as correct (+) or incorrect (–).
◦ The score for accuracy should be expressed as a percentage.

Scoring the activity for accuracy requires a consistent
understanding of when to mark an answer right or wrong.
◦ Certainly, if the student performed the skill independently, the answer
is either correct or incorrect.
◦ But what about when the student receives a prompt? How do you
score the item correct or incorrect?
2012-2013 NJ APA Administrator Training
106
Scoring Student Evidence for
Accuracy
•
Understanding the difference between the
following provisions will help ensure that the
scoring information on the evidence is
accurate:
• providing task directions
• providing supports
• providing indirect prompts (verbal, model, and
gestural)
• providing physical prompts
• providing the answer (directly prompting the
student using a verbal, model, or gestural prompt
to correctly answer the question)
2012-2013 NJ APA Administrator Training
107
Scoring Student Evidence for
Accuracy
•
If the student performs the item independently,
then
• Score as correct (+) any item the student
performs correctly.
• Score as incorrect (–) any item the student
performs incorrectly.
• Any system other than +/– must include a key.
• If you used a different scoring system, such as
✗ equals incorrect and ✓ equals correct, you
must explain that clearly on the evidence.
2012-2013 NJ APA Administrator Training
108
Scoring Student Evidence for
Accuracy
•
If the student received an indirect verbal,
gestural, or model prompt when performing
the skill, then
• Score as correct (+) any item the student
answered correctly.
• Score as incorrect (–) any item the student
answered incorrectly.
• Remember to mark next to each item the
prompt level used: V, G, or M (e.g., +V means
that an indirect verbal prompt was given and
that the student answered the item correctly.)
2012-2013 NJ APA Administrator Training
109
Scoring Student Evidence for
Accuracy
•
Any item for which the student received a
physical prompt should be marked as
incorrect (–P).
• A physical prompt is a direct prompt
whereby the teacher physically guides
the student to the correct answer (e.g.,
hand over hand).
• However, if the student receives a physical
prompt when performing the skill, then
• Score the item as incorrect (–P).
2012-2013 NJ APA Administrator Training
110
Scoring Student Evidence for Accuracy
•
Total accuracy score must be in the
form of a percent.
• The number of items correct divided by the
total number of items multiplied by 100 equals
the total accuracy score.
Accuracy
# items correct divided by # of total items times 100 equals %
Correct
____/_____ x 100 = ______%
2012-2013 NJ APA Administrator Training
111
Performance
• The second piece of evidence should not include a
more intrusive prompt than was used on the first
piece of evidence.
Prompting can affect the reliability of the
performance score if not implemented
correctly.
• Therefore, it is important to adhere to the prompt
level hierarchy described in your 2012-2013
Procedures Manual.
• If the second piece of evidence includes a more
intrusive prompt level than the first piece of
evidence, a score of 1 will be assigned for the
Performance dimension.
2012-2013 NJ APA Administrator Training
112
Scoring Performance - Non-Example
10-11-12
2012-2013 NJ APA Administrator Training
2-1-13
113
Performance
The initial piece of evidence should not be more
difficult than the final piece of evidence.
◦ If a support (e.g., a calculator or formula) is provided in the
final piece of evidence, it must also be provided in the initial
piece of evidence. Otherwise, the initial piece of evidence
would be more difficult.
◦ Supports provided in the initial piece of evidence do not
have to be present in the final piece; they can be faded if no
longer needed by the student.
◦ The final piece of evidence can be of the same difficulty level
as the initial piece of evidence, or it can be more difficult.
◦ The student should be given enough task directions,
supports, and prompting (if needed) on the initial piece of
evidence to elicit on-topic responses, even if incorrect.
2012-2013 NJ APA Administrator Training
114
Busy, Overly Complicated
4.3.12B2: Near Link:
“Given a graph of a
linear function, identify
the x- and y- intercept”
Problems: the coordinate
grid isn’t labeled, the
answer choices indicate
the grid was intended to
be marked by 2’s, the
colored lines make the
graph jumbled, and use of
colored pencils makes the
task overly complicated.
Sandy
10-25-12
2012-2013 NJ APA Administrator Training
115
Make It Clear
Sandy
Date: 1-17-13
100% Independent
4.3.12B2: Near
Link: “Given a
graph of a linear
function, identify
the x- and yintercept”
This activity
clearly and
simply
assesses
the link.
2012-2013 NJ APA Administrator Training
116
Performance
• When assessing a 3.2 Writing Link that requires a writing
rubric, there should be feedback on the student’s work
explaining how the work was scored in relation to the
writing rubric.
◦ If the student did not earn all of the points for a particular
dimension on the writing rubric, there should be feedback on the
student’s work explaining why/how the work was insufficient or
incorrect.
◦ Even when a student earns all of the points allotted on a writing
rubric, there should still be feedback affirming the student’s
work or explaining why it was particularly acceptable in
accordance with the writing rubric.
2012-2013 NJ APA Administrator Training
117
Performance
Writing Feedback
Non-example: Evidence does not include any feedback that
corresponds to writing rubric.
3.2.8B2 Near: Write a short story that includes a setting, a plot, and characters rubric
Amanda
•Larissa T.
10/28/12
10/28/12
2012-2013 NJ APA Administrator Training
118
Independence Dimension
Independence evaluates the extent to
which the student completed items/tasks
independently.
2012-2013 NJ APA Administrator Training
119
Supports vs. Prompts
Not all direction is considered a prompt. Some direction is simply a
support, which does not need to be indicated, tracked, or part of the
score. Examples of supports:
•
•
•
•
•
•
Redirecting the student to stay on task is not a prompt; it is a support.
Reading directions with the student is a support.
Restating the directions for the student is a support.
Scribing for the student is a support.
Providing accessible materials (large print, textured paper,
manipulatives, etc.) is a support.
Supports are not included when calculating the independence
score for the student’s work.
2012-2013 NJ APA Administrator Training
120
Examples of Supports for Mathematics








Use of manipulatives or pictures
Use of a calculator
Task-analyzed skills
Number line
Stamps to write numbers
Word problems read to student
Enlarged text
Chunked problems
2012-2013 NJ APA Administrator Training
121
Examples of Supports for Reading

Student reads by
 Following along with the text
 Following along with picture symbols
 Following along with objects
 Following along with tactile cues

Adapted text
 Shortened text
 Chunked information
 Large font
 Highlight to emphasize important information
2012-2013 NJ APA Administrator Training
122
Examples of Supports for Writing








Stamps
Graphic organizers
Write using pictures or objects
Write by completing cloze sentences
Scribing (work must use the student’s
communication mode)
Speech-to-text writing programs
Large writing utensils or pencil grips
Raised lined paper
2012-2013 NJ APA Administrator Training
123
Examples of General Supports


Reading initial task directions to the student
Redirecting the student to stay on task
◦ “you need to finish math now”
If a student requires supports, he/she must be
given supports for both the first and second
piece of evidence.
2012-2013 NJ APA Administrator Training
124
Independence Dimension
0
1
2
3
4
Evidence
is not clear or all
items are not
marked for
Independence/
prompt level
Student
completed
items/tasks
independently
0-39% of the
time, based on
the final
activity
Student
completed
items/tasks
independently
40-59% of the
time, based on
the final activity
Student
completed
items/tasks
independently
60-80% of the
time, based on
the final
activity
Student
completed
items/tasks
independently
81-100% of
the time, based
on the final
activity
2012-2013 NJ APA Administrator Training
125
Teacher Scoring for
Independence
• A prompt level must be documented next to each item
performed by the student.
•
•
•
•
•
•
I = independent
V= verbal
G= gestural
M= model
P= physical
If some other system is used there must be a key.
• Remember, if your prompt hierarchy is not the same as the one above,
you must explain your prompt hierarchy clearly, giving the names of
the prompts, hierarchical order, and letter key for the prompts.
• Even if the student performs with 100% independence, each item
must be marked with an I.
• Independence scores must be summarized as a percent.
• Reviewers must be able to recreate the score. If the score is
not replicable, the dimension can not be scored and will
receive zero for Independence.
2012-2013 NJ APA Administrator Training
126
Prompting During AssessmentAllowable Prompts
• An indirect verbal prompt (V) can
• provide the student with a clue to try and spark the
student’s recollection of the activity or lesson so that
he/she can respond to the question (e.g., “Who was the
story about?” or “Remember, the main character has
red hair and pigtails. Point to the main character.”)
• An indirect gestural prompt (G) can
• Provide the student with a clue as to the general
location of an answer (e.g., when looking up a word in
the dictionary, the teacher may tap the page on which
the word can be found but not exactly where the word
is on the page)
2012-2013 NJ APA Administrator Training
127
Prompting During Assessment Allowable Prompts
• An indirect model prompt (M) can
• provide the student with a clue through demonstration of the
skill (e.g., demonstrate how to carry in an addition problem
then give the student a different problem)
• Provide the student with a clue through acting out a scenario
(e.g., when presenting a choice of three pictures and asking
the student which picture represents an unbalanced force, the
teacher may make a sweeping or moving motion to represent
an “unbalanced force”)
2012-2013 NJ APA Administrator Training
128
Direct Prompting During
Assessment - NOT ALLOWED
•
•
•
A direct verbal prompt
• provides the student with the specific answer to a question or item
(e.g., “Remember, the main character was Pippi. Point to the picture
of the main character.”)
A direct gestural prompt
• points out the specific answer to the student (e.g., when presenting
a choice of three pictures and asking the student which picture
represents an unbalanced force, the teacher points to or taps the
correct picture)
A direct model prompt
• models the exact problem and answer that the student must
perform (e.g., when sorting producers and consumers, the teacher
says, “Remember, corn is a producer,” picks up the picture of corn,
and places it in the producer column of a chart then asks, “Which
one is a producer?”)
2012-2013 NJ APA Administrator Training
129
Prompting During Assessment Allowable Prompts
• A physical prompt (P)
• requires physical contact between the student and teacher
• may be a touch at the elbow prompting the student to move
his/her hand to a particular answer, or it may be more
invasive, involving the teacher physically moving the student’s
hand to the correct answer
• If the student must be given a physical prompt to
correctly answer the test question, the answer must be
marked wrong.
• This applies to full and partial physical prompts.
2012-2013 NJ APA Administrator Training
130
Scoring Evidence for Independence
•
The total Independence score must be in the form of a
percent.
• Take the number of items completed independently divided by the
total number of items multiplied by 100 to calculate the total
Independence score.
•
When scoring for Independence, the information does not
include accuracy.
Independence
# items performed independently divided by # of total items times 100
equals % Correct
____/_____ x 100 = ______%
2012-2013 NJ APA Administrator Training
131
LANGUAGE ARTS
LITERACY
Sample Entry
From 2011-2012
Albert
2012-2013 NJ APA Administrator Training
132
Language Arts Literacy Entry - Example
APA 2011-2012
ENTRY COVER SHEET
Student Name__Albert__________
Evidence of Initial Activity – Dated Between September 1, 2011 - November 11, 2011
Evidence of Final Activity – Dated Between December 12, 2011 - February 17, 2012
This entry is for: (Circle one)
LAL
This is entry number: (circle one)
___3.1___
Standard
_4______
Grade
MATH
1
__F______
Strand
2
3
SCIENCE
4
_2_______
CPI
In addition to the code above, you may want to include the written information:
Standard: 3.1 Reading
Strand: F Vocabulary and Concept Development
CPI: 3.1.4F2 Infer specific word meanings in the context of reading passages
CPI Link: (record the specific bulleted CPI Link to be assessed) After reading a
passage, identify the meaning of unknown words using context clues____________
CPI Link type: (Circle one)
Matched
Near
Far
Description of Initial Activity:
Albert will read the passages by following along with the picture cues and text. He
will then identify the meaning of the underlined words using context clues.
_
Description of Final Activity:
Albert will read the passages by following along with the picture cues and text. He
will then use the word bank to identify the appropriate definition of the underlined
word, using context cues to find the definition.
_
1
2012-2013 NJ APA Administrator Training
133
9-15-11
2012-2013 NJ APA Administrator Training
134
2a
2012-2013 NJ APA Administrator Training
135
2b
2012-2013 NJ APA Administrator Training
136
12-15-11
2012-2013 NJ APA Administrator Training
137
3a
2012-2013 NJ APA Administrator Training
138
3b
2012-2013 NJ APA Administrator Training
139
Language Arts Literacy Evidence Index – Sample Entry for Albert 2011-2012
4th Grade LAL Entry
Entry 1
Standard 3.1
Strand F
CPI 2
Matched Link : “After reading a passage, identify the meaning of unknown words using context cues”
Rubric Dimension
Score
Evidence
Complexity
4
 Pgs. 2-2b and 3-3b show the student working on the complete CPI
Link from the “Matched” column
 Pgs. 3-3b (evidence 2) are a different activity than pgs. 2-2b
(evidence 1)
 Different context: different passages and words assessed
 Different application: multiple choice vs. fill in the blank
Performance
3
 Pgs. 2-2b student scored 20% (which is below
39%) on initial activity
 Pgs. 3-3b student scored 80%
 Each item is clearly marked
 Pgs. 3-3b do not include a more intrusive prompt
than pgs. 2-2b
Independence
3
 Pgs. 3-3b student scored 80% Independence
 Each item is clearly marked
2012-2013 NJ APA Administrator Training
140
Professional and Ethical Responsibility
The Alternate Proficiency Assessment is intended to assess
the academic progress of students with significant cognitive
disabilities toward the standards established by the state of
New Jersey.
 The format of the APA requires teachers administering this
assessment to develop assessment activities appropriate for
each participating student and to provide appropriate
documentation of how the activities were used to assess
students on what they gained from instruction.
 Instruction is the key. Assessments should never be used
as the instruction. Instruction should take place first, and
then assessments should be designed to reflect what was
taught.

 The first piece of APA evidence is based on the initial
introduction of the skill, which usually occurs prior to
implementing extensive instruction.
2012-2013 NJ APA Administrator Training
141
Professional and Ethical Responsibility


Teachers are to follow the steps for the development of
an entry described in the teacher training materials.
Teachers are to save not only copies of the assessments
that are being included in the student’s portfolio but
also copies of student work that, while not included in
the portfolio, provide further proof that the student
received instruction and was given opportunities to
practice and show what he/she was taught prior to the
final assessment activity.
◦ These additional student work activities may be reviewed during
a site visit or required to be presented for a security breach
visit.
2012-2013 NJ APA Administrator Training
142
Professional and Ethical Responsibility


Each assessment presented to the student should be unique, meaning
that the student should not have previously seen the assessment.
The following practices are not allowed:
 Teaching students to memorize answers for an assessment.
 Giving students the same assessment (activity) over and over until they
complete the assessment satisfactorily.
 Allowing students to see the assessment and its test questions ahead of
time.
 Allowing someone else to complete work for a student. Remember, there
are guidelines for scribing that must be followed if a student needs
assistance documenting their answer.
 Giving an assessment when no instruction has occurred.
 Giving the students a writing task response and having them copy it and
submitting it as their own work.
 Leaving the date blank on a completed activity and adding a date later.
 Changing the date on a completed activity.
 Any other practice that results in misleading information
regarding what the student was taught and has learned.
2012-2013 NJ APA Administrator Training
143
Professional and Ethical Responsibility

Some students may not be able to complete an entire
assessment activity all at once. It is acceptable to administer
assessment items one or two at a time, as appropriate for
the student, but when doing so, the student should only be
presented with the item or items that he/she will be
expected to complete at that time.
 It is acceptable to mask the remaining items on the page until
they are completed by the student.
If a student becomes sick during an assessment or cannot
complete an assessment at the time it is given, that
assessment should be used as practice and a new assessment
should be given at a better time.
 Remember that the APA has two rather long testing
windows that should help address most special testing
conditions that may arise for students participating in the
APA.

2012-2013 NJ APA Administrator Training
144
Professional and Ethical Responsibility

It is the professional and ethical responsibility of all
contributors to a student’s portfolio to ensure that any
and all documentation reflect authentic, accurate, and
truthful information.
◦ Reminder, it is not appropriate to manipulate the date the
evidence was created, the prompt/independence levels, the
student’s work, or the scoring percentages.
◦ Include only evidence that reflects the student’s actual
performance, even if the student did not make as much progress
as you expected.
◦ Follow all APA guidelines when administering the assessment.
2012-2013 NJ APA Administrator Training
145
Professional and Ethical Responsibility

Any student portfolio that is found to contain
inauthentic documentation may result in professional
consequences for school and district staff.
◦ All contributors to a portfolio must sign and submit the
assessment security form verifying that the APA was conducted
properly and in accordance with all directions and training
materials. District administrators will provide this form. This is a
new policy, and is in line with the general assessment
procedures.

It is important that administrators review the teacher
training modules to ensure you know what is
acceptable, and not acceptable, regarding APA
assessment. Remember that this is an individual
assessment, not a group assessment.
2012-2013 NJ APA Administrator Training
146
Test Security Agreement

Give the teachers the agreement this fall
so that they can see what they will sign
and submit when they turn in the
portfolio.
◦ Every teacher that produces APA evidence
must submit a signed Test Security Agreement
to be placed in the binder.
◦ If teachers worked on only one subject, they
may write that on the agreement.
2011-2012 NJ APA Administrators Training
147
Status of 2011 - 2012 Portfolios and
Score Reports




Pearson Scoring Center staff scored the portfolios
during April and May.
Score reports were shipped in June.
Most portfolios have been returned.
Portfolios designated as security breaches (Void 5)
are in process. Information regarding the breach
consequences will be forthcoming.
◦ Please contact me to arrange a meeting to discuss the
district’s security breach portfolios.
2012-2013 NJ APA Administrator Training
148
Contact Information



For information on interpreting the 2011-2012 APA
score reports, please review the Score Interpretation
Manual on Pearson’s website
http://pem.ncspearson.com/nj/apa
For missing score reports please contact:
NJ Customer Service Team – 888-705-9416
For score report interpretation please contact:
Jo-Lin Liang – 609-633-6884
2012-2013 NJ APA Administrator Training
149
What is Proficient?

New cut scores were set in June 2009 which
determine how the total score for each subject area
is classified. Total scores may be classified as
advanced proficient, proficient, or partially proficient.
◦ The cut scores were determined based on the grade-level
linked skills and content being assessed; therefore, the same
number of total points may be classified as advanced
proficient in one subject area for a grade level, and classified
as proficient for a different grade level, due to differences in
the CPI Links and developmental demands of students in
different grades.
◦ This mirrors the general assessments where each grade
level and subject area cut scores are determined
independently.
2012-2013 NJ APA Administrator Training
150
How is a Total Score Calculated?

The scoring formula was determined by reflecting on
the purpose of the APA, which is to:
◦ assess the district and school grade-level instructional
program provided to the student,
◦ assess a student’s attainment of knowledge and skills of the
CCCS, and
◦ encourage high but appropriate instructional standards for all
students, while providing necessary supports to access the
curriculum and foster the highest level of independence
possible for the student.

The scoring formula uses the three dimension scores
that each entry receives:
◦ Complexity
◦ Independence
◦ Performance
2012-2013 NJ APA Administrator Training
151
How is a Total Score Calculated?



The total score for each content area is
calculated by adding the scores for all three
dimensions for each of the four entries.
Each entry has a minimum possible score of
zero and a maximum possible score of 16
points.
The total score for each subject area has a
minimum score of 0 and a maximum score of 64
points.
2012-2013 NJ APA Administrator Training
152
How is a Total Score Calculated?

Each scorable entry will receive a score of 0 – 4 for each
dimension.
◦ Unscorable entries receive a zero for each of the three dimensions.

Each entry is scored by a minimum of two people (readers).
◦ First person is reader one (R1) and second person is reader two (R2).

The performance dimension score for any entry is the sum
of the two reader scores (R1+R2).
◦ The addition of the two reader scores gives more weight to this
dimension in the overall total score than the other two dimensions.


The complexity dimension score for any entry is the
average of R1 & R2.
The independence dimension score for any entry is the
average of R1 & R2.
2012-2013 NJ APA Administrator Training
153
Accountability



APA test results will be combined with the results
from the general assessment for accountability
purposes for state and federal reports.
For accountability purposes, the APA is both a
student assessment, and a school/district program
assessment.
APA test results are included in the Final
Accountability report.
2012-2013 NJ APA Administrator Training
154
Federal Results – 1% Cap

The NCLB 1% cap limits the number of APA proficient
and advanced proficient results that can be used in the
accountability calculation. It does not limit the number
of students who may participate in the APA.

Districts legitimately testing more than 1% of the total
grade population with the APA should apply for a waiver
from the Office of Special Education. The waiver, if
approved, allows proficient and advanced proficient
results greater than 1% to be used in the accountability
calculations.
◦ This form is usually mailed by the department to districts in
the winter.
2012-2013 NJ APA Administrator Training
155
Accountability Reports

Student data is collected on the APA scan sheet. This
data is submitted to districts for verification.

All requested revisions to student data must be made
during the record change process. No changes may be
submitted after this time frame.

Accountability reports are produced based on the
student data on file as of the close of the record change
process.

Reports for sending districts include students assessed
out of district, out of state, and in district.
2012-2013 NJ APA Administrator Training
156
Information – Title I Office

Up-to-date information on the calculation of
Accountability Reports can be obtained on the
Department of Education’s website
www.state.nj.us/education
◦ Review the information provided from the Office
of Title I. The Accountability Workbook provides
specific details.

If you have questions after reviewing the online
information, please call the Office of Title I at
609-943-4283.
2012-2013 NJ APA Administrator Training
157
Main Responsibilities

Complete scan sheet materials in February 2013.
◦ Work directly with administrators from the child’s
sending or receiving school to ensure accurate
information. Errors on the scan sheet may affect
your Accountability Report calculations.
◦ Several fields on the scan sheet must be completed in
February since they are time sensitive and can not be
completed during the Pre-ID process.
◦ A few fields on the scan sheet may need to be
completed by the student’s teacher.

Collect portfolios from teachers by end of day on
February 15, 2013.
◦ Administrators may choose to collect them earlier in
order to complete portfolio review process when high
number of portfolios are produced.

Review final student data and submit record change
rosters. This is the last opportunity to make data
changes.
2012-2013 NJ APA Administrator Training
158
Final Administrator Review

Test Design Review
◦ Did the evidence address the required components?
 Prescribed standards and grade-level CPIs
 Correct amount of evidence
 Meets the Universal Scoring Rules
◦ Conducting this review periodically will allow you to find and correct
errors during the collection period.

Security Breach Review
◦ Is white out being used to change information?
◦ Were prompt levels changed to independent performance?
◦ Were dates on the evidence changed?
◦ Were wrong answers changed to correct answers?
◦ Were wrong answers marked correct?
◦ Are the percentage scores accurate?
◦ Are there other indications of inauthentic information?
2012-2013 NJ APA Administrator Training
159
Final Administrator Review

Review portfolios for accuracy and authenticity
by February 22, 2013.
◦ Review the portfolios in light of the student’s assigned
grade level.
◦ Review the portfolios in light of the current CPI Links.
◦ Review the portfolios in light of the test design
requirements.
◦ Review the portfolio to ensure all evidence is authentic.

Reminder that portfolios should be reviewed
multiple times within each collection period.
◦ Doing so helps you identify errors early, giving enough
time to produce new evidence without errors prior to
the end of the collection period.
2012-2013 NJ APA Administrator Training
160
Shipment of Portfolios

The shipment process will be communicated to
districts at a later time. Portfolios must be
shipped between February 26 and March 11,
2013.
◦ Any portfolio shipment dated after March 11 will not be
accepted for scoring.
◦ Portfolios must be submitted on time in order
to be processed.
◦ Shipping materials and instructions will arrive in the
Chief School Administrator’s office in February
2013.
2012-2013 NJ APA Administrator Training
161
Administrator/Coordinator Activities
General Responsibilities

When information is sent to you, pass it on to district and
school administrators and teachers. Review the time lines
and request APA information from your chief school
administrator when it is scheduled to be delivered.

Review participation guidelines carefully.

Ensure that all students who need to be assessed are
identified (both in-district and out-of-district).
2012-2013 NJ APA Administrator Training
162
Administrator/Coordinator
Activities
General Responsibilities (continued)

Review school educational programs and ensure alignment
with dimensions of the scoring rubric.

Ensure that the APA teachers participate in the online
teacher training sessions.

Review the APA Procedures Manual.

Pass along questions and concerns to NJ DOE or
contractor.
2012-2013 NJ APA Administrator Training
163
Administrator/Coordinator Checklist
Before Assessment
Identify a School Coordinator for each school.
 Register number of students participating in APA and
update mailing addresses using Material Survey no later
than Date to Be Determined.
 Create a teacher training schedule for your school,
identify teachers required to participate, and implement
training.

◦ Ensure all materials are downloaded prior to the selected training
day(s).
◦ Have a sign in sheet for all teachers to verify attendance and
completion of the training.
◦ Utilize all APA training materials, including the Content Resource
Guide and Five Item documents, in addition to the Teacher Training
Modules and Procedures Manual.

Distribute materials to schools/teachers.
2012-2013 NJ APA Administrator Training
164
Administrator/Coordinator Checklist
During Assessment

Distribute materials to schools/teachers. Binders and
manuals will be shipped to district/receiving school.

Copies of the APA training materials may be downloaded as
of September 7, 2012, from the Pearson’s website
http://pem.ncspearson.com/nj/apa. Click on Documentation,
then Procedures Manual.
◦ Content Resource Guide and Five Item Sample documents may not be
available until September 19, 2012.
2012-2013 NJ APA Administrator Training
165
Administrator/Coordinator Checklist
During Assessment (continued)

Gather information on students’ sending/receiving status
and collect student demographic info.
◦ Work with the receiving/sending school to confirm student
data.

Check periodically that teachers are working on the APA.
2012-2013 NJ APA Administrator Training
166
Administrator/Coordinator Checklist
After Assessment
Notify teachers of timeframe for returning the
completed APA portfolios.
 Collect and review all APA portfolios for accuracy and
authenticity of information and inclusion of all required
components.

◦ Be sure to review the portfolios for adherence to the
universal scoring rules.

Review the scan sheet information to ensure accuracy of
all data.
2012-2013 NJ APA Administrator Training
167
Administrator/Coordinator Checklist
After Assessment (continued)

Package the APA materials securely and label them
according to instructions.

Contact the designated carrier for return of the
portfolios to arrange pickup of the materials.

Ensure all portfolios are submitted by March 11, 2013
deadline.
◦ Portfolios not received by shipping carrier by March 11, 2013 will
not be accepted for scoring.
2012-2013 NJ APA Administrator Training
168
APA Teacher Involvement


Teachers are needed in March 2013 for a week to score
portfolios. Send an email to Elizabeth Celentano if
someone is interested in assisting in the scoring process.
Email address is Elizabeth.Celentano@doe.state.nj.us
Summer committees will be formed to assist in the test
specification revision for alignment to the general
assessments and Common Core Standards.
◦ Teachers and curriculum specialists will be needed to complete this
work.
◦ More information will be forthcoming in Spring 2013.
2012-2013 NJ APA Administrator Training
169
Dynamic Learning Maps



New Jersey is one of 13 states collaborating with the
University of Kansas on the future version of an alternate
assessment for students with the most significant cognitive
disabilities.
Opportunities arise for districts to participate in the
process via document reviews, meetings, and implementing
pilot components. If you would like to become involved
please email Elizabeth Celentano.
This revised assessment is expected to be implemented in
some form (pilot?, operational?) in 2014-2015. As
information becomes available it will be disseminated.
2012-2013 NJ APA Administrator Training
170
Where To Get More Information?
Educational Links, Information, and Resources on
Assistive Technology and Universal Design for
Learning, an excellent reference document for
teachers and administrators can be downloaded from
the DOE website
http://www.state.nj.us/education/specialed/tech/
2012-2013 NJ APA Administrator Training
171
Where To Get More Information?
Other educational materials and resources can be
accessed through the New Jersey Learning Resource
Center Network website:
http://www.nj.gov/education/lrc
2012-2013 NJ APA Administrator Training
172
Where To Get More Information?
Elizabeth Celentano (test design, training, test administration, etc.)
Office of Assessments
New Jersey Department of Education
609-984-0107
elizabeth.celentano@doe.state.nj.us
Jo-Lin Liang (student data, Pre-ID, record changes, score interpretation, etc.)
Office of Assessments
New Jersey Department of Education
609-633-6884
Jo-Lin.Liang@doe.state.nj.us
Office of Special Education (general special education questions)
New Jersey Department of Education
609-292-0147
http://www.nj.gov/education/specialed/
Department of Education website:
www.state.nj.us/education
2012-2013 NJ APA Administrator Training
173
Where to Get More Information?
Topics: Training materials, score reports 2011-2012,
portfolio binder return
Pearson - Bruce Murphy bruce.murphy@pearson.com
Customer Service njapacustsvc@support.pearson.com
Phone:
888-705-9416
Pearson website:
http://pem.ncspearson.com/nj/apa
Topics: Explanation and Error Sheet issues, Evidence
questions (through November 2012)
ILSSA – Lou Ann Land
lland@uky.edu
Alson Cole alson.cole@uky.edu
2012-2013 NJ APA Administrator Training
174
Where to Get More Information?
Topics: Student registration, Pre-ID, scan sheets, portfolio
shipments, record changes, online entry cover sheets,
etc.
Questar Assessment, Inc.
NJ APA Customer Support:
NJAPACustomerSupport@questarai.com
Teri Broderick Tbroderick@questarai.com
Phone: 866-690-4733
NJ APA Website address:
https://nj-servicepoint.questarai.com/Login.aspx
2012-2013 NJ APA Administrator Training
175
Winter 2013 Training Materials



In January 2013 a new Power Point presentation will be
uploaded for your review. You will be notified when it is
posted and on which website.
It will include reminders about key concepts provided
today as well as information about necessary tasks
related to returning the portfolios.
Please make sure you read these materials and contact
the Office of Assessments if you have questions.
2012-2013 NJ APA Administrator Training
176
Thank You
Thank you for taking time to attend this session. If you
have not had all of your questions answered, please
see one of the presenters for more information.
2012-2013 NJ APA Administrator Training
177