North Carolina Department of Public Instruction
Division of Accountability Services
North Carolina Testing Program
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Public Schools of North Carolina
State Board of Education
Department of Public Instruction
Accountability Services/Test Development Section
Raleigh, North Carolina 27601-2825
www.ncpublicschools.org/accountability/testing
STATE BOARD OF EDUCATION
The guiding mission of the North Carolina State Board of Education is that every public school student
will graduate from high school,
globally competitive for work and postsecondary education and prepared for life in the 21st Century.
WILLIAM C. HARRISON
Chairman :: Fayetteville
REGINALD KENAN
Rose Hill
JOHN A. TATE III
Charlotte
WAYNE MCDEVITT
Vice Chair :: Asheville
KEVIN D. HOWELL
Raleigh
ROBERT “TOM” SPEED
Boone
WALTER DALTON
Lieutenant Governor ::
Rutherfordton
SHIRLEY E. HARRIS
Troy
MELISSA E. BARTLETT
Roxboro
CHRISTINE J. GREENE
High Point
PATRICIA N. WILLOUGHBY
Raleigh
JANET COWELL
State Treasurer :: Raleigh
NC DEPARTMENT OF PUBLIC INSTRUCTION
June St. Clair Atkinson, Ed.D., State Superintendent
301 N. Wilmington Street :: Raleigh, North Carolina 27601-2825
In compliance with federal law, NC Public Schools administers all state-operated educational programs,
employment activities and admissions without discrimination because of race, religion, national or ethnic origin,
color, age, military service, disability, or gender, except where exemption is appropriate and allowed by law.
Inquiries or complaints regarding discrimination issues should be directed to:
Dr. Rebecca Garland, Chief Academic Officer :: Academic Services and Instructional Support
6368 Mail Service Center, Raleigh, NC 27699-6368 :: Telephone: (919) 807-3200 :: Fax: (919) 807-4065
Visit us on the Web:: www.ncpublicschools.org
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
This manual was prepared by:
Dr. Tammy Howard, Chief
Test Development Section
North Carolina Department of Public Instruction
Division of Accountability Services
Jim Kroening, Performance Assessments Director
Test Development Section
North Carolina Department of Public Instruction
Division of Accountability Services
Akia Beverly-Worsley, Writing Content Specialist
Center for Urban Affairs & Community Services
North Carolina State University
Technical Outreach for Public Schools
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
i
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Table of Contents
Section I






Purpose/Background
Introduction ........................................................................................... 1
Background ............................................................................................ 1
NCEXTEND2 OCS Regulations ........................................................ 2
NCEXTEND2 OCS Modifications ..................................................... 3
Writing Advisory Consultants ............................................................... 3
Timeline for NC Writing Assessments .................................................. 5
Section II Scoring Information
 Scorers ................................................................................................... 6
 Regional Scoring..................................................................................... 7
 Reliability Standards for Regional Scoring ............................................ 7
 Scorer Bias ............................................................................................. 9
 Total Writing Score Calculation Examples .......................................... 10
 Achievement Levels (NCEXTEND2 OCS) ........................................ 10
 Achievement Level Descriptors (NCEXTEND2 OCS) .......................11
Section III Score Reporting
 State Data ............................................................................................. 12
 School and Student Data ...................................................................... 12
Section IV Composing Features
 Content component ..................................................................................
o Focus ......................................................................................... 13
o Organization.............................................................................. 14
o Support and Elaboration .......................................................... 14
 Conventions component...........................................................................
o Sentence Formation...................................................................15
o Usage ..........................................................................................15
o Mechanics ..................................................................................15
 Content Rubric for NCEXTEND2 OCS ............................................ 16
 Conventions Rubric for NCEXTEND2 OCS ......................................17
 Examples of Common Conventions Errors ......................................... 18
Section V Scoring Rubric Applications
 Guide Set for Grade 10 NCEXTEND2 OCS Assessment.............. 19-31
o 2010 Writing Prompt for Grade 10 NCEXTEND2 OCS ............
o Student Exemplars (anchor papers) ............................................
o Guide Set Annotations (score explanations) ...............................

Training Set for Grade 10 NCEXTEND2 OCS Assessment ........ 32-40
o Score Tally Sheet: Training Set A .................................................
o Student Exemplars (sample papers).............................................
o Training Set Annotations (score explanations) ............................
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
ii
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010

Qualification Set for Gr. 10 NCEXTEND2 OCS Assessment……..41-46
o Score Tally Sheet: Qualification Set I ...........................................
o Student Exemplars (sample papers) ............................................
o Qualification Set Annotations (score explanations) ....................

Contact Information ................................................................................
o Accountability Division............................................................. 47
o Instructional Services Division ................................................. 47
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
iii
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
INTRODUCTION
This training has been designed to deepen your understanding of the North Carolina
Writing Assessment and the test development process. This is accomplished through
historical perspective and the explanation of field testing, prompt review, prompt
selection criteria, scorer qualifications, scoring procedures, and rubric application to
student responses.
It is not the intent of this manual to neither represent nor reflect classroom practice in
the teaching of writing to students. Quality writing instruction is based on sound
pedagogy from which student performance is reflected. The North Carolina Department
of Public Instruction Division of Accountability Services/Test Development Section
provides this manual and training material for use in understanding the writing
assessment and the application of the rubric to selected student responses. The student
responses represent a range of potential score points that demonstrate the performance
of features indicative of those being assessed. The responses do not represent the only
way a student’s response might attain a particular score point. It is important to read
the annotations for each sample to understand how the rubric was applied to that
particular response.
BACKGROUND
North Carolina, believing that an emphasis on writing instruction was needed and that
the measurement of writing would enhance instruction, began a statewide writing
assessment program during the 1983–84 school year.
In September 2001, the North Carolina Department of Public Instruction (NCDPI)
Testing Program, NCDPI English/Language Arts Section, and North Carolina State
University/Technical Outreach for Public Schools (NCSU/TOPS) staff met and began
the process that would result in new writing assessments and scoring procedures at
grades 4, 7, and 10. Writing committees for each grade level were established that
included general education teachers, English as Second Language (ESL) teachers,
Exceptional Children (EC) teachers, curriculum supervisors, principals, and university
faculty. The North Carolina Writing Assessment Scoring Model was developed by the
NCDPI Testing Program and NCSU/TOPS staff, refined by the Writing Assessment
Advisory Committee, and approved by the State Board of Education on January 9, 2003.
Beginning with the 2005-06 school year, the writing program expanded to include the
NCEXTEND2 Writing Assessment as an alternative to the North Carolina General
Writing Assessment. The NCEXTEND2 OCS Writing Alternate Assessment uses a
modified test format and measures student performance against grade level modified
achievement standards. Based on guidance from the United States Department of
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
1
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Education, Federal Register (December 15, 2005) Title I for Improving the Academic
Achievement of the Disadvantaged, Individuals with Disabilities Education Act (IDEA)Assistance to States for the Education of Children with Disabilities, and the NPRM
(Notice of Proposed Rule Making) dated December 15, 2005, the NCDPI Division of
Accountability Services, in conjunction with the NCDPI Exceptional Children’s Division
developed guidelines for participation in the NCEXTEND2 OCS Writing Alternate
Assessment. NCEXTEND2 OCS for writing is only available to eligible students in
grade 10 who participate in the Occupational Course of Study.
Grade Level Type of Writing by Year
Year
10 OCS
2006-07
Expressive
2007-08
Expressive
2008-09
Expressive
2009-10
Expressive
NCEXTEND2 OCS REGULATIONS
NCEXTEND2 OCS is an alternate for the required high school assessments. The
NCEXTEND2 OCS is only available to eligible students with disabilities for the
required assessments in reading, mathematics, writing, and science.
It is the expectation that ALL students who participate in NCEXTEND2 OCS are
receiving instruction in the Occupational Course of Study (OCS) for the subject(s) in
which the students are being assessed.
To determine student participation in the NCEXTEND2 OCS program (Occupational
English, Occupational Mathematics, Life Skills Science), the following eligibility
requirements must be considered:




Student must have a current IEP;
Student does NOT have a current 504 plan;
If student is identified as limited English proficient (LEP), he/she must also have
a current IEP;
The student is enrolled for credit in courses in the Occupational Course of Study
which require an NCEXTEND2 OCS assessment:
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
2
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
o OCS students enrolled in Occupational Mathematics I and/or
Occupational English I must participate in the NCEXTEND2 for
Occupational Mathematics I and/or Occupational English I;
o OCS students enrolled for credit in the second course of Life Skills Science
I and Life Skills Science II, regardless of the order in which they are taken,
must participate in the NCEXTEND2 for Life Skills Science I and II;
o For the NCEXTEND2 OCS Writing Assessment, the student must be
assigned to grade 10 according to the student information management
system (e.g., SIMS/NCWISE) and is following the Occupational Course of
Study (OCS) Occupational English.
A student may be assessed based on modified achievement standards in one or more
subjects for which assessments are administered. Students eligible to take assessments
based on modified achievement standards may be in any of the 13 disability categories
listed in the IDEA. The decision to assess a student based on modified achievement
standards must be reviewed annually as part of the IEP process.
NCEXTEND2 OCS MODIFICATIONS
A modified assessment can have a variety of elements. For example, on a multiplechoice assessment, there may be shorter passages, fewer answer choices, or fewer
overall test items. For an extended response assessment such as writing, modifications
occur in the level of depth and breadth of the composing features for each grade level. It
is important to recognize that the prompts, rubrics, and the composing features of the
NCEXTEND2 OCS Writing Alternate Assessment are different from the Grade 10
General Writing Assessment. Style has been eliminated from the Content rubric and
case has been removed from standard usage in the Conventions rubric.
While the general and NCEXTEND2 OCS scoring rubrics have some similarities, it is
in the application of those rubrics and criteria that a modified assessment is designed.
To that end, it is critical that the annotated anchor papers and training papers are
reviewed in order to understand the way in which the rubrics and the features are being
defined for this population of students.
WRITING ADVISORY CONSULTANTS
During the summer, prior to the start of the school year, North Carolina educators are
recruited to participate as Writing Advisory Consultants. The writing consultants
evaluate materials for the NCEXTEND2 OCS Writing Assessment at Grade 10.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
3
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Participation is based on staggered terms and consists of (at least):
 six grade-level specific practicing classroom teachers (3 year term)
 five grade-span specific practicing classroom teachers (2 year term)
 one professional in the area of special education (2 year term)
 one professional in the area of limited English proficiency (2 year term)
 one post-secondary professional with experience relevant to the specific
grade-level (3 year term)
 one professional from the English/Language Arts department of NCDPI
 one professional from the Test Development Section of NCDPI
Writing consultants are qualified individuals who are divergent thinkers, possess
leadership skills which support the NCEXTEND2 OCS Writing Assessment at Grade
10, have a positive record of service, and will devote the necessary time required to
ensure that the objectives of the writing assessment are achieved. Efforts are made to
ensure that the consultants reflect the racial and ethnic diversity of North Carolina’s
tenth grade student population and that the major geographic regions in the state are
represented.
The primary purpose of the Writing Advisory Consultants is to make recommendations
based not on what students can do, but what students should be expected to do as
outlined in the NC Occupational Course of Study (OCS) for English/Language Arts.
Another duty of the Writing Advisory Consultants is to provide advice and input for the
selection of prompts, sample papers, and annotations to be used in the scoring of
student responses.
In October of each school year, the Writing Advisory Consultants meet to select the
operational and alternate prompts for the current school year, as well as prompts
necessary for field testing. The consultants examine each potential prompt using
statistical data from the field tests. All prompts are reviewed for socioeconomic,
racial/ethnic, gender, regional, and religious biases before being selected for statewide
administration. Each prompt is also reviewed for accessibility in order to ensure that all
prompts are appropriate for the NCEXTEND2 OCS Writing Assessment student
population.
In February, the Writing Advisory Consultants convene to conduct the first round of
range-finding. Range-finding is a major step in the scoring process. The range-finding
process involves the scoring contractor, Writing Advisory Consultants, NCDPI
Accountability Services/Test Development Section staff, NCSU-TOPS staff, NCDPI
Instructional Services, NCDPI English/Language Arts staff, and NCDPI Exceptional
Children staff. The contractor provides samples of student field test responses. The
Writing Advisory Consultants view and score the student field test responses to establish
“anchor papers.” All consultants must come to consensus and unanimously agree on the
content and conventions scores given to each response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
4
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
The anchor papers represent examples of particular score points and are referenced by
the scorers during the scoring process. They are used in conjunction with scoring
rubrics to help in deciding what score points student responses are assigned. This helps
to ensure that consistency in standards is applied to all responses.
In March, soon after the operational administration, range-finding occurs again. Round
two of the range-finding process is the same as the first EXCEPT that the writing
samples gathered from students are “Live” responses. “Live” student responses refer to
responses that students wrote during the operational test administration. In order to
obtain “Live” student responses, a representative sample of schools is contacted and
asked to return test materials to the contractor immediately after the test. The
contractor uses these student responses to conduct the second range-finding session
prior to the start of the scoring project.
WRITING ASSESSMENTS TIMELINE
September
NC Writing Advisory Consultants meet to
select Field Test, Operational, and Alternate
prompts
February
NC Writing Advisory Consultants meet for RangeFinding meetings (Round–One)
March
Operational Administration (Second Tuesday)
[Exception: When Easter is in March, then first Tuesday]
NC Writing Advisory Consultants meet for RangeFinding meetings (Round-Two)
Late March –
End of April
Operational Assessment Scoring Window
(conducted by the vendor)
May – June
Scores uploaded, reports generated, project
closes, and training materials are developed.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
5
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Scoring Information
Scorers
The scoring project occurs after training materials are assembled using the papers
scored during the range-finding process. There are over 100,000 responses that are
scored for the grade 10 writing assessments. Each response is 100% second scored,
meaning that two scorers, independent of each other, assign content and conventions
scores for each response.
Individuals who score the North Carolina Writing Assessments (general and
NCEXTEND2 OCS) must successfully complete a variety of activities prior to scoring
student responses.
First, a potential scorer must have the necessary qualifications. A post-secondary
degree is required of all applicants. Scorers must have documented experience in the
field of English, Writing, or Education. Scorers must also pass a preliminary
grammatical and written exam. Many who score the NCEXTEND2 OCS Writing
Assessment have prior scoring experience and exceed the minimum requirements.
When initial employment is granted, scorers receive training from the contractor
specific to bias and large scale assessment. Examples of scorer bias are reviewed with
the scorers, and the information has been included in the Scorer Bias chart provided in
this manual.
Next, potential scorers must successfully complete the training and qualifying phase
specific to the North Carolina project. The training materials consist of a scoring rubric,
a set of guide (anchor) papers, three training sets, and three qualifying sets. Scorers who
have had extensive experience scoring the North Carolina project are trained as scoring
supervisors. These scoring supervisors are given additional training to act as an initial
resource when scorers have questions. They assist in the monitoring of scorers.
Those individuals trained as scorers are given the scoring rubrics, composing features,
and guide set. They are not supplied with additional materials such as the NC
Occupational Course of Study (OCS) as this leads to individual interpretation of the
curriculum and would impact the validity and reliability of scoring. Scorers are trained
using the student responses that were scored by the North Carolina Writing Advisory
Consultants during range-finding in order to understand how the rubric is to be applied
to student responses.
After scorers are trained, they must qualify to score. Qualification involves applying the
scoring rubrics and guide set to selected student responses. Potential scorers have three
opportunities to accurately assign both content and conventions scores to at least
seventy percent of the student responses. Scorers must go through all three
opportunities regardless of when in the process they qualify.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
6
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Scorers are monitored by both the scoring contractor and NCDPI on a daily basis
throughout the project. All scorers who begin to drift from the 70% reliability
requirement are retrained or dismissed. Any responses scored by scorers who are
dismissed are sent back into the pool of responses to be rescored, and the previously
assigned scores are removed.
Data is collected throughout the course of the scoring process and is reviewed each day
by NCDPI Accountability/Test Development Section and NCSU-TOPS staff. This data is
compiled and analyzed for several purposes, including but not limited to, validity,
reliability, and frequency (score point and Total Writing Score) distributions.
Regional Scoring
Traditionally, the NCDPI has contracted with a vendor to score the large-scale writing
assessments in regional scoring centers. The vendor operated these regional scoring
centers and supervised the scorers under strict quality control measures. All training
sessions for scorers, however, were conducted by NCDPI Test Development and NCSUTOPS staff who were present at these scoring centers. The NCEXTEND2 OCS Writing
Assessment continues to be scored in this manner due to the small population size and
modified nature of the assessment.
Pearson, the vendor for the writing projects, maintained a central headquarters to
supervise the distributed scoring (General Assessment) and regional scoring
(NCEXTEND2 OCS Assessment) operations.
In addition, NCDPI personnel
monitored scorers and the scoring process through secure online web access. The
NCDPI generated real-time scoring reports and daily data statistics.
Reliability Standards for Regional Scoring
All scorers had to meet the rigorous requirements set forth by the NCDPI as in previous
years. Scorers first had to meet the eligibility criteria, sign Test Security and
Confidentiality Agreements, pass the necessary training requirements, and qualify for a
scoring position.
After qualifying to score the assessments, scorers were required by NCDPI to maintain
the industry standard inter-rater perfect agreement (reliability) of 70 percent. Scorers
also had to maintain a 70 percent validity standard (agreement with the range-finding
scores assigned to responses by the Writing Advisory Consultants and NCDPI Test
Development Staff). All scorers who did not meet or exceed the 70 percent standards
NCDPI Division of Accountability Services/Test Development Section
7
Summer 2010
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
(inter-rater and validity) were removed from the project and all scores assigned to
student responses were invalidated. These student responses were subsequently
rescored by two qualified scorers.
Grade Level
10 NCEXTEND2 OCS
10 NCEXTEND2 OCS
Rubric Trait
Content
Conventions
IRR*
93.6
93.2
Validity
84.9
83.7
*IRR refers to inter-rater perfect agreement (reliability).
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
8
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Scorer Bias
Scorer Bias refers to personal factors that may affect a scorer’s perception of a student’s
response, but have no basis in the scoring guide. Below are factors that scorers may
encounter but must not allow to impact scoring.
Appearance of
Response
Length of
Response
Repetition of
Response
Offensive or
Disturbing
Content
Unusual
Approaches to
the Prompt
Response to
Prompt
Reactions to
Style
Writer
Personality
Reactions to
Performance
Assessments
The quality of the handwriting, the use of cursive or printing, margins,
editing marks, cross-outs, and overall neatness are not part of the
scoring criteria.
The length of a student’s paper is not part of the scoring criteria. Scorers
should take into consideration only whether the finished piece feels
complete and has the components required. The size of a student’s
handwriting can make a paper look longer or shorter on the page than it
actually is.
Although scorers may tire of reading several papers on the same topic, it
is important to remember that for each student the response represents
a unique attempt.
If a student uses vulgar language, adopts a sexist or racist point of view,
or perhaps takes a naïve or narrow approach to a topic, should not let
the student’s point of view affect the score. Likewise, scorers should not
let a student’s lifestyle or maturity level influence them either positively
or negatively regarding their writing.
It is tempting to want to reward an especially creative approach to a
prompt, a poem for example, or a slant on the topic no one else has used.
Scorers should remember that an unusual or creative attempt alone does
not necessarily constitute an upper level paper. The overall attempt
must be successful.
Likewise, an unusual approach handled
successfully should not be scored punitively.
In the classroom, there may be a “correct” response to a writing
assignment. For this assessment, students are free to respond any way
they choose. There is no right or wrong “answer” as long as it is clear the
student is attempting to reply to the prompt.
A scorer’s own grammatical biases should not play a part in assigning a
score if the student has not violated standard writing conventions. In
other words, beginning a sentence with “and”, the absence of a formal
thesis sentence, the use of first or second person, or an informal tone are
not necessarily wrong in this type of assessment.
Writers may come across as brash, sassy, cute, self-aware, shy, surly,
flat, honest, or naïve. Scorers are scoring the written passage, not the
writer’s personality.
Some scorers may approach writing assessments with their own biases
in favor of one type of assessment over another. Or, they may feel as
though the standards used in an assessment violate their own sense of
good writing. It is important for each scorer to set aside his/her own
biases in order to keep the scoring as standardized and as fair to each
student as possible.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
9
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Total Writing Score Calculation Examples
TWS = 2 X (R1 Content + R2 Content) + (R1 Conventions + R2 Conventions)
Example 1
Scorer 1 Content
4 Scorer 2 Content
4 Total Content Score
Conventions 1
Conventions 2 Total Conventions Score
(Total Content Score x 2) + Total Conventions Score = Total Writing Score
8 x 2 = 16
+ 3
=
19 Achievement Level IV
8
3
Example 2
Scorer 1 Content
3 Scorer 2 Content
3 Total Content Score
Conventions 0
Conventions 0 Total Conventions Score
(Total Content Score x 2) + Total Conventions Score = Total Writing Score
6 x 2 = 12
+ 0
=
12 Achievement Level III
6
0
Example 3
Scorer 1 Content
2 Scorer 2 Content
2 Total Content Score
Conventions 2
Conventions 2 Total Conventions Score
(Total Content Score x 2) + Total Conventions Score = Total Writing Score
4x2=8
+ 4
=
12 Achievement Level III
4
4
Achievement Levels:
NCEXTEND2 OCS Writing Assessment at Grade 10
On December 6, 2007, the State Board of Education approved the NCEXTEND2 OCS Writing
Assessment Achievement Levels for Grade 10. The final Achievement Level ranges and descriptors
are as follows and can be found in the State Board of Education Policy Manual listed as HSP-C-027:
Subject
Writing OCS
(Effective
2006-07)
Grade
Level I
Level II
Level III
Level IV
10
4-6
7-11
12-16
17-20
According to State Board of Education policy, the standard for proficiency is a test score of
Achievement Level III or above on the North Carolina NCEXTEND2 OCS Writing Assessment.
NOTE: The approved final Achievement Level ranges and descriptors for the
NCEXTEND2 OCS Writing Assessment at Grade 10 are posted on the NC State Board
of Education website: http://sbepolicy.dpi.state.nc.us/.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
10
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
NCEXTEND2 OCS Writing Assessment Achievement Level Descriptors
Grade 10
Achievement Level I
Students performing at this level do not have sufficient mastery of knowledge and skills in this
subject area to be successful at the next grade level.
Students performing at Achievement Level I have made an attempt to address the task. The
topic/subject is not developed and ideas may be presented in a random or confusing manner.
Details, if provided, lack development and/or are unrelated to the topic/subject. The students
demonstrate an insufficient knowledge of sentence formation and lack skills in conventions
necessary to be successful at the next grade level.
Achievement Level II
Students performing at this level demonstrate inconsistent mastery of knowledge and skills in
this subject area and are minimally prepared to be successful at the next grade level.
Students performing at Achievement Level II may or may not maintain focus on the
topic/subject. Ideas are presented in an attempt to establish a logical progression and the details
provided are sparse. The students demonstrate a limited knowledge of sentence formation, do
not maintain a consistent use of conventions, and are minimally prepared to be successful at the
next grade level.
Achievement Level III
Students performing at this level consistently demonstrate mastery of grade level subject matter
and skills and are well-prepared for the next grade level.
Students performing at Achievement Level III establish a focus on the topic/subject and the ideas
presented progress logically. Most of the details provided are sufficiently developed. The
students demonstrate a general knowledge of sentence formation and conventions and are wellprepared for the next grade level.
Achievement Level IV
Students performing at this level consistently perform in a superior manner clearly beyond that
required to be proficient at grade level work.
Students performing at Achievement Level IV consistently focus on the topic/subject and use
clear transitions that define a logical progression of ideas. Most of the details provided are
developed and supported with appropriate examples. The students demonstrate a consistent
knowledge of sentence formation and conventions clearly beyond that required to be proficient at
grade level work.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
11
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Score Reporting
State Data
Data is reported at the state level. Results are disaggregated by subgroups, i.e. gender,
disability or ability (AIG), Title I, and free-reduced lunch. Scores are reported by the
total population tested, percent of students at proficiency, and results by achievement
level. A writing report is generated each year compiling a summary of all writing
assessment results, including general scoring observations.
School and Student Data
Each school is provided with rosters listing student results by class. The information is
provided by the scoring contractor to LEAs on CD. Each CD contains class rosters with
individual student results, as well as, the imaged student responses by class.
School CD’s are shipped to each LEA test coordinator along with two copies of the
Individual Student Report (ISR). The ISR is a paper report of the individual student
results. The report lists the total content score, total conventions score, total writing
score, and corresponding achievement level for each student. The ISR also provides an
explanation of the composing features, information about scoring procedures, and
definitions of the four achievement levels specific to the grade level and particular
assessment.
More information about the state testing results can be found on the NCDPI website at:
http://www.ncpublicschools.org/accountability/testing/shared/statetestsresults
More information about the reports of academic performance can be found on the NCDPI
website at:
http://www.ncpublicschools.org/accountability/testing/reports/writingandopenended
More information about the writing assessments can be found on the NCDPI website at:
http://www.ncpublicschools.org/accountability/testing/writing/.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
12
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
NCEXTEND2 OCS
COMPOSING FEATURES
Students participating in the Grade 10 NCEXTEND2 OCS Writing Assessment receive
a prompt (writing topic) based on the Occupational English I and II curricula. The
expressive response for the NCEXTEND2 OCS Writing Assessment requires
students to explore/and or communicate his or her personal knowledge, experiences,
and insights. The expressive response focuses on events related to learning occupational
skills and is written in the form of letters or journal entries. There are three types of
expressive responses: Work Skills, Life Skills, and Personal Skills.

In the Work Skills response, the student examines modes of communication in
employment settings.

In the Life Skills response, the student examines modes of communication in
daily living.

In the Personal Skills response, the student expresses factual, functional
information.
NOTE: The composing features that are to be observed assume specific meanings when
applied to student responses. In order to demonstrate a reasonable level of control in
any of the features below, the students must have written a sufficient amount. The
North Carolina Writing Assessments have two components for which scores are given:
Content and Conventions. An explanation of each feature and its application to the
responses are provided below.
CONTENT COMPONENT:
Focus, Organization, Support and Elaboration, and Style
FOCUS
Focus is the topic/subject established by the writer in response to the writing task. The
writer must clearly establish a focus as he/she fulfills the assignment of the prompt. If
the writer retreats from the subject matter presented in the prompt or addresses it too
broadly, the focus is weakened. The writer may effectively use an inductive
organizational plan, which does not actually identify the subject matter at the beginning
and may not literally identify the subject matter at all. The presence, therefore, of a
focus must be determined in light of the method of development chosen by the writer.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
13
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010

If the scorer is confused about the subject matter, the writer has not effectively
established a focus.

If the scorer is engaged and not confused, the writer probably has been effective
in establishing a focus.
ORGANIZATION
Organization is the progression, relatedness, and completeness of ideas. The writer
establishes for the scorer a well-organized composition, which exhibits a constancy of
purpose through the development of elements forming an effective beginning, middle,
and end.

The writer establishes relationships between and among ideas and/or events
throughout the response.

The response demonstrates a clear progression of related ideas and/or events and
is unified and complete.
SUPPORT AND ELABORATION
Support and Elaboration is the extension and development of the topic/subject. The
writer provides sufficient elaboration to present the ideas and/or events clearly. Two
important concepts in determining whether details are supportive are relatedness and
sufficiency.

Relatedness: To be supportive of the subject matter, details must be related to the
focus of the response. Relatedness has to do with the directness of the
relationship that the writer establishes between the support and elaboration and
the topic/subject. Supporting details should be relevant and clear. Effective use of
concrete, specific details strengthens the response.

Sufficiency: Sufficiency has less to do with the amount and more to do with the
specificity and effectiveness of the support and elaboration provided. The writer
must present his or her ideas with enough power and clarity to cause the support
to be sufficient. Undeveloped details, redundancy, and the repetitious
paraphrasing of the same point often characterize insufficiency. Effective use of
concrete, specific details strengthens the response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
14
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
CONVENTIONS COMPONENT:
Sentence Formation, Usage, and Mechanics
CONVENTIONS
Conventions involve correctness in sentence formation, usage, and mechanics. The
writer has control of grammatical conventions that are appropriate to the writing task.
Errors, if present, do not impede the scorer's understanding of the ideas conveyed.

Sentence Formation is the complete expression of an assertion, explanation,
proposal, question, or command.

Standard usage includes agreement, tense, and case.

Mechanics involve the use of capitalization, punctuation, and spelling.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
15
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Content Rubric
Points
4
3
Descriptions













2
1
NS




















Topic/subject is clear, though it may or may not be explicitly stated
Maintains focus on topic/subject throughout the response
Organizational structure establishes relationships between and among ideas and/or events
Consists of a logical progression of ideas and/or events and is unified and complete
Support and elaboration are related to and supportive of the topic/subject
Consists of specific, developed details
Exhibits skillful use of vocabulary that is precise and purposeful
Demonstrates skillful use of sentence fluency
Topic/subject is generally clear, though it may or may not be explicitly stated
May exhibit minor lapses in focus on topic/subject
Organizational structure establishes relationships between and among ideas and/or events,
although minor lapses may be present
Consists of a logical progression of ideas and/or events and is reasonably complete, although
minor lapses may be present
Support and elaboration may have minor weaknesses in relatedness to and support of the
topic/subject
Consists of some specific details
Exhibits reasonable use of vocabulary that is precise and purposeful
Demonstrates reasonable use of sentence fluency
Topic/subject may be vague
May lose or may exhibit lapses in focus on topic/subject
Organizational structure may establish little relationship between and among ideas and/or events
May have major lapses in the logical progression of ideas and/or events and is minimally complete
Support and elaboration may have major weaknesses in relatedness to and support of the
topic/subject
Consists of general and/or undeveloped details, which may be presented in a list-like fashion
Exhibits minimal use of vocabulary that is precise and purposeful
Demonstrates minimal use of sentence fluency
Topic/subject is unclear or confusing
May fail to establish focus on topic/subject
Organizational structure may not establish connection between and among ideas and/or events
May consist of ideas and/or events that are presented in a random fashion and is incomplete or
confusing
Support and elaboration attempts to support the topic/subject but may be unrelated or confusing
Consists of sparse details
Lacks use of vocabulary that is precise and purposeful
May not demonstrate sentence fluency
This code may be used for compositions that are entirely illegible or otherwise
unscorable such as: totally blank responses, responses written in a foreign language,
exact restatements of the prompts, and responses that are completely off- topic or
incoherent.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
16
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Conventions Rubric
Points
2
1
0
Descriptions
Exhibits reasonable control of grammatical conventions
appropriate to the writing task
 Exhibits reasonable control of sentence formation
 Exhibits reasonable control of standard usage
including agreement, tense, and case
 Exhibits reasonable control of mechanics including
use of capitalization, punctuation, and spelling
Exhibits minimal control of grammatical conventions
appropriate to the writing task
 Exhibits minimal control of sentence formation
 Exhibits minimal control of standard usage including
agreement, tense, and case
 Exhibits minimal control of mechanics including use
of capitalization, punctuation, and spelling
Lacks control of grammatical conventions appropriate
to the writing task
 Lacks control of sentence formation
 Lacks control of standard usage including agreement,
tense, and case
 Lacks control of mechanics including use of
capitalization, punctuation, and spelling
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
17
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Examples of Common Convention Errors
Sentence Formation:
A sentence is an expression of an assertion, explanation, proposal, question, or
command.
Fragment
Run-ons




Phrases or clauses used incorrectly which interfere
with the meaning of the sentence.


When I go to school.
Then I started to write.
I think they need to get up earlier so they
can get ready for school and have time to
eat breakfast they need to get up at an
earlier time.
I knew that I would never get away with it
and plus I had two tests that day and I
didn’t really want to go.
While sleeping, they need to go to bed
earlier.
Drinking my milk, the cookies seemed
irresistible.
Usage:
Standard usage includes agreement, tense, and case.
Incorrect use of verbs
Pronoun misuse
Incorrect formations
Failure to use a word according to its standard
meaning (homophone)







Students is very disruptive.
People was laughing at the guy’s answers.
The girls went to play with there own teams.
Between you and I, the test was hard.
hisself, theirselves, bestest
How did you no (know)?
Tell them to right (write) a letter home.
Mechanics:
Mechanics involves the use of capitalization, punctuation, and spelling.
Incorrect Capitalization
Incorrect Punctuation
Pattern of misspellings of common words or incorrect
pluralization.










Did he give It away?
The teacher’s name is tom evans.
Jose and i went to the store.
George eats Bananas and Oranges.
Why did she go home early.
John plays golf tennis and baseball.
“Tom said Go to the store.”
freind for friend
boxs for boxes
droped for dropped
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
18
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Guide Set
GRADE 10
Content & Conventions
Scoring Rubric
Applications
Note: Papers are ordered by Content Score, not Total Writing Score
This publication and the information contained within must not be used for personal or
financial gain. North Carolina LEA school officials and teachers, parents, and students may
download and duplicate this publication for instructional and educational purposes only.
Others may not duplicate this publication without prior written permission from the North
Carolina Department of Public Instruction (NCDPI) Division of Accountability Services/North
Carolina Testing Program.
© 2010 All Rights reserved. This document may not be reproduced by any means, in whole or
in part, without prior written permission from the North Carolina Testing Program, Raleigh,
North Carolina 27601-2825.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
19
2010 North Carolina Testing Program
NCEXTEND2 OCS Writing, Grade 10
Do Not Reproduce—NCDPI
You are in the Occupational Course of Study (OCS). Write a letter to a new student
explaining the OCS program.
As you write a letter explaining the OCS program to a new student, remember to
❑ Explain the OCS program.
❑ Organize your ideas.
❑ Write your letter so it makes sense.
❑ Use important details.
❑ Review and correct your letter for capitalization, punctuation, and spelling.
Use the blank sheet of paper given to you by your teacher to plan your letter.
Anything you write on the blank sheet will not be scored. You must write the final
copy of your letter on pages 3 and 4 of your test booklet.
Write the final copy of your letter on pages 3 and 4 of your test booklet.
© 2010 All rights reserved. This document may not be reproduced by any means, in whole or in part, without the express written permission of the North Carolina
Department of Public Instruction, Raleigh, N.C.
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
GUIDE SET ANNOTATIONS
Paper
G-1
Score
Notes
Content/Conventions
(TWS=Total Writing Score)
1/0
Guide Paper 1 (TWS =4)
Content Score 1— This response establishes an unclear
topic (OCS program is fun to work you ling new thing).
Details are sparse and the organizational structure is
minimal. Ideas are presented in a random fashion (live and
hard work you make all A-B in you class . . . . u make good
garn and your class become your dono’t take all of your
class). Although there is an attempt to support the topic,
the sparse details are unrelated and confusing.
Conventions Score 0– This brief response lacks control of
grammatical conventions appropriate to the writing task.
The errors in sentence formation, standard usage, and
mechanics are dense, varied, and severe.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
20
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-2
1/0
Guide Paper 2 (TWS =4)
Content Score 1– This response initially addresses the
prompt (When you first come to the Occupational Course
of Study. The program is about they helping people out),
but fails to maintain focus on the topic of life “[w]hen you
finish school.” Ideas are presented in a random fashion
that results in an incomplete and confusing response.
Although some detail is provided (When you move out you
will need some money so that you can by...food, shoes,
clous, for your family that’s what kinds of things you need
to stay alive. if it wasn’t for that you mike be bead or
alive), support and elaboration remain unrelated and
confusing as a whole.
Conventions Score 1– This response lacks control of
sentence formation, standard usage, and mechanics.
Several run-on sentences are present (If you don’t know
what to do you can start by looking in the newspaper or
telling an friend or family memder they can help you find
an jod that you like doing because) and little control of
capitalization or punctuation is demonstrated.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
21
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-3
1/1
Guide Paper 3 (TWS =6)
Content Score 1– While this brief response establishes a
focus (Hey T---- welcome to the OCS program), the
organizational structure consists of ideas presented in a
random fashion (It can show easy ways to learn and show
you ways to be yourself. Also the teachers is the best, they
teach the best and they work with you). Although the
response contains some detail, elaboration is sparse and
the progression of ideas is confusing.
Conventions Score 1– This response exhibits minimal
control of sentence formation, standard usage, and
mechanics. Run-on sentences, major spelling and
punctuation errors, as well as subject/verb agreement
errors are present (See the classes is’nt what people think it
is, some people thank the class is for something else but
it’s for you to get your High School diaploma if you what
it).
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
22
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-4
2/1
Guide Paper 4 (TWS =10)
Content Score 2 – Although the topic is not explicitly
stated, this response is somewhat focused on showing a
new student what “to do in this [OCS] class.”
Organization is limited and major lapses in the progression
of ideas occur. Support and elaboration consist of
undeveloped, list-like details (get some good sleep . . . .
always have papper and penci or pen . . . . always pay
atnin . . . . the most thing you need is a time talb sheet and
some thing for you can count on for you can get the right
answer).
Conventions Score 1– This response exhibits minimal
control of sentence formation and contains several run-ons
and fragments. Some control is demonstrated in
punctuation and the spelling of most basic words.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
23
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-5
2/1
Guide Paper 5 (TWS =10)
Content Score 2– This response begins with a strong
focus on the topic (. . . and I’m writing you to tell you how
OCS works. OCS is Occupational Course of Study. It help
you get ready for the real world.), but due to
organizational issues the focus breaks down, leading to a
major lapse in focus. While some elaboration is provided,
the details are general and list-like.
Conventions Score 1– This response exhibits minimal
control of punctuation, spelling, and capitalization.
Sentence formation and mechanical errors are present in
the response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
24
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-6
2/2
Guide Paper 6 (TWS =12)
Content Score 2 – Although a topic is not explicitly
stated, this response remains somewhat focused on
introducing the OCS program. Demonstrating major lapses
in the logical progression of ideas, the response is
minimally complete. The organizational structure
establishes little relationship between ideas (We look at the
news on cnn to see what happening in the world. We do
school base hours for helping teachers out), and support
and elaboration consist of undeveloped, list-like details.
Conventions Score 2– This response exhibits reasonable
control of sentence formation, usage, and mechanics.
There are spelling and usage errors, but these errors do not
impede the meaning of the response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
25
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-7
2/2
Guide Paper 7 (TWS =12)
Content Score 2 – This response is focused on the
positive aspects of the OCS program, although the topic is
not explicitly stated. Some organizational structure is
present, but little relationship is established between ideas,
leading to major lapses in the logical progression of ideas
(Do you have any classes? I will so you what classroom
you will be in. Around the OCS Program, they have
Technical sessons). Support and elaboration consist of
some details presented in a list-like fashion (You can also
join a after-school club. . . . like Spanish, french, S.A.D.D.
. . . You can be a cheerleader, girls basketball, girls
baseball, girls sccoer, girls tinnis, & girls wrestsler) and
exhibit major weaknesses.
Conventions Score 2– This response displays reasonable
control of sentence formation, standard usage, and
mechanics. Subject/verb agreement and verb tense
agreement are demonstrated. Although some errors in
capitalization and punctuation are present, the errors do
not impede the meaning of the response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
26
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-8
3/2
Guide Paper 8 (TWS =16)
Content Score 3 – The focus of this response is generally
clear (I welcome you to the (OCS) program. . . . In this
program you will learn about lots of new things). The
response demonstrates a logical progression of ideas that is
reasonably complete by introducing OCS classes to a “new
student”(Social Studies in this class yo will get to learn
about Americas past and hostorical time too. . . . The more
thing you get to learn more about laws and about all the
people in the governments house). Support and elaboration
consist of some specific details.
Conventions Score 2– This response exhibits reasonable
control of sentence formation, standard usage, and
mechanics. Errors are present in spelling and punctuation,
but these errors do not impede the meaning of the
response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
27
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-9
3/2
Guide Paper 9 (TWS =16)
Content Score 3 – The focus of this response is generally
clear (you are about to . . . probably like school more now
that your going to be in the (OCS) program . . . . because
the work is easy but hard enough where you learn
something and the teachers are really nice). The
organizational structure demonstrates relationships
between and among ideas. The response is reasonably
complete (You will also have to get a non paid job out in
the community to graduate. Then you will be asked to
create a portfolio . . . you will have to present that in front
of a few people your senior year in order to graduate).
Support and elaboration consist of some specific details.
Conventions Score 2– This response exhibits reasonable
control of sentence formation, standard usage, and
mechanics. Subject/verb agreement and verb tense
agreement are demonstrated. The errors in this response do
not impede the meaning of the response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
28
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-10
3/2
Guide Paper 10 (TWS =16)
Content Score 3 – Although the topic is not explicitly
stated, this response remains focused on introducing the
OCS program to “the new student.” The chronological
organizational structure is maintained throughout the
response; however minor lapses occur (They take you on
feild trips to learn about stuff. The OCS program does pay
tenth graders that work around the school but only tenth
graders. If a student is in the ninth grade inthe ocs
program they do not get paid). Despite the minor lapses,
the response displays a logical progression of ideas and is
reasonably complete. Several specific examples are
provided about how the OCS program helps “teach you.”
Conventions Score 2– This response exhibits reasonable
control of sentence formation, standard usage, and
mechanics.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
29
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-11
4/2
Guide Paper 11 (TWS =20)
Content Score 4– The topic of this response is clear (The
OCS Program consists of three major components. These
components are work, portfolio, and classwork). Focus is
maintained throughout and a logical progression of ideas is
established, resulting in a complete response. Elaboration
consists of supporting details that are specific and related
(The portfolio is a notebook that you put together during
all four years of High School. . . . It has personal
information about you . . . such as the classes your took,
the jobs you volunteered at and got paid at).
Conventions Score 2– With few errors, this response
displays reasonable control of grammatical conventions.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
30
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
G-12
4/2
Guide Paper 12 (TWS =20)
Content Score 4– The topic of this response is clear (I am
a student . . . in the Occupational Course of Study
program. I am going to be explaining to you how the
program works), and a strong focus on the benefits of
attending the OCS program is maintained throughout. The
organizational structure establishes relationships between
and among ideas. A sustained logical progression of ideas
makes the response complete (Than I would work in the
libarary and help out the libarians. Most of those duties
are required to gain hours to graduate . . . . All of the OCS
requirements sound hard but once you go threw and see
for yourself it’ll be one of your first fun easy years of high
school). The student provides ample specific details and
elaborates on each experienced benefit of the program.
Conventions Score 2– This response exhibits reasonable
control of sentence formation, standard usage, and
mechanics. Subject/verb agreement and verb tense
agreement are demonstrated.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
31
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Training Set
GRADE 10
Content & Conventions
Scoring Rubric
Applications
This publication and the information contained within must not be used for personal or
financial gain. North Carolina LEA school officials and teachers, parents, and students may
download and duplicate this publication for instructional and educational purposes only.
Others may not duplicate this publication without prior written permission from the North
Carolina Department of Public Instruction (NCDPI) Division of Accountability Services/North
Carolina Testing Program.
© 2010 All Rights reserved. This document may not be reproduced by any means, in whole or
in part, without prior written permission from the North Carolina Testing Program, Raleigh,
North Carolina 27601-2825.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
32
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Score Tally Sheet: Training Set A
You may use this sheet to practice scoring the following student responses. Record your scores
for Content and Conventions and then compare them against the actual scores located at the end
of this section.
PAPER #
My
Content
Score
My
Conventions
Score
State
Content
Score
State
Conventions
Score
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
% Agree 1
1
In order to maintain industry standard and NC scoring requirements of at least 70%, you must
have an exact agreement in Content and Conventions for 11 of the 15 responses.
NCDPI Division of Accountability Services/Test Development Section
33
Summer 2010
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TRAINING SET ANNOTATIONS
Paper
Score
Notes
Content/Conventions
TA-1
1/0
Training Set A, Paper 1
Content Score 1-The focus of this minimal response
attempts to address the topic but lacks support and
elaboration. The organizational structure consists of a
brief, randomly ordered list of ways in which OCS helps
(The program have a good teacher that can help you. Get
you to your goles in life. OK if you are in the 9th graed you
have to work at school.).
Compare the content score to Guide Paper G-1.
TA-2
2/2
Conventions Score 0- This response is not sufficient to
demonstrate control of grammatical conventions and
receives a score of 0.
Training set A, Paper 2
Content Score 2- This response focuses on the topic, but
the list-like organizational structure establishes little
relationship between and among ideas. The loosely related,
vague events create major lapses in the logical progression
of ideas.
Compare the content score to Guide Paper G-5.
Conventions Score 2- This response demonstrates
reasonable control of grammatical conventions with errors
in sentence formation and usage.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
34
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TA-3
3/2
Training set A, Paper 3
Content Score 3- The focus of this response is generally
clear on the ways in which OCS class is “just like other
ones”. Organized around a discussion of the types of
activities in the OCS program (First you will be building
stuff like bird house’s, making brick snowman and a lot
more other stuff. Then when you get out of that then you
will be on the campson working making hours.), the
response demonstrates a logical progression of ideas.
Compare the content score to Guide Paper G-10.
TA-4
4/2
Conventions Score 2- This response demonstrates
reasonable control of grammatical conventions with errors
in usage and spelling.
Training set A, Paper 4
Content Score 4- While maintaining clear focus
throughout and presenting a logical, complete progression
of ideas, this response offers supporting details that are
specific and related.
Compare the content score to Guide Paper G-11.
TA-5
2/2
Conventions Score 2- With few errors, this response
displays reasonable control of grammatical conventions.
Training set A, Paper 5
Content Score 2- The organizational structure in this
response establishes little relationship between ideas. Only
minimally complete, support and elaboration have major
weaknesses in relatedness to and in support of the topic
(You will have to bring in you chaet stubs and your hours
to get counted. You will get fuss at If you don’t do your
work and get the hours in.). General, undeveloped details
are presented in a list-like fashion.
Compare the content score to Guide Paper G-5.
Conventions Score 2- This response shows reasonable
control of grammatical conventions with scattered
sentence formation, usage and spelling errors.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
35
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TA-6
1/0
Training set A, Paper 6
Content Score 1- This response displays a focus on the
topic, but the organizational structure does not establish
any relationship between and among ideas (In OCS you
will be test or occuparichal Ehghn, mqfn, Rinqd and unre
caren. You will be gening test on thes caren the tesd will
be. . . .). The loosely related, vague events create lapses in
the logical progression of ideas and support is sparse.
Compare the content score to Guide Paper G-2.
Conventions Score 0- This response lacks control of
grammatical conventions. Errors in sentence formation,
usage, and spelling impede the meaning of the response.
TA-7
2/1
Training Set A, Paper 7
Content Score 2- This response focuses on the ability to
“go to work win your in school”, but the list-like
organizational structure establishes little relationship
between ideas and events. The unrelated, undeveloped
details lead to major lapses in the logical progression of
ideas.
Compare the content score to Guide Paper G-5.
Conventions Score 1-This response displays minimal
control of grammatical conventions with sentence
formation, usage, punctuation, and spelling errors.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
36
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TA-8
1/1
Training set A, Paper 8
Content Score 1- The focus of this response is somewhat
unclear (The ocs will help you understand a lot better), and
the organizational structure fails to establish connections
between and among ideas. Although there is an attempt to
support the topic, the response consists of a repetitive,
unrelated list of activities (It will teach you from hard
work to easy work. But don’t get me wrong beside it did
help me understand in my other classes.).
Compare the content score to Guide Paper G-3.
TA-9
2/1
Conventions Score 1- This response demonstrates
minimal control of grammatical conventions with sentence
formation, spelling, and punctuation errors.
Training set A, Paper 9
Content Score 2- This response focuses on examples of
the various classes in the OCS program. While the
organizational structure establishes little relationship
between and among ideas and events, some general,
undeveloped support is displayed (You get for classes a
semester. You have to have 3 maths, 4 reading, 2Sciense,
2SStudys, 4 voc classe and 4 occupatianl class.).
Compare the content score to Guide Papers G-6 and G-7.
TA-10
1/0
Conventions Score 1- This response demonstrates
minimal control of grammatical conventions with sentence
formation, spelling, and punctuation errors.
Training set A, Paper 10
Content Score 1- The focus of this response is unclear,
and the organizational structure fails to establish
connections between and among ideas. Although there is
an attempt to support the topic, the response consists of a
sparse list of activities.
Compare the content score to Guide Paper G-1.
Conventions Score 0- This brief response is not sufficient
to demonstrate control of grammatical conventions and
receives a score of 0.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
37
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TA-11
2/1
Training set A, Paper 11
Content Score 2- This response focuses on what OCS is
and the many classes and activities connected to the
program. The chronological organization resulting from
the use of personal examples provides some progression of
ideas (First thing you need to know is that your based
hours on campus are exactly three hundred hours and off
campus is 360 hours but when you go to work remember
this is just like a real job you have got to work your
hardest to get those hours if you don’t you will fail),
although major lapses do occur.
Compare the content score to Guide Paper G-7.
TA-12
3/2
Conventions Score 1- This response displays minimal
control of grammatical conventions with some basic errors
in sentence formation, usage, and dropped words.
Training set A, Paper 12
Content Score 3- Generally focused on the benefits in
which OCS can help a new student, this response presents
an extended personal example to support the topic. The
response demonstrates a logical progression of ideas,
although supporting details are sometimes general
(Learning new skills is a good way to show that you no
what you are doing. Here are some skills you will learn.
Things you will learn will be how to dress when going to
get a job. Learn how to save your money. Next, learn how
to open your own bank account.).
Compare the content score to Guide Paper G-9.
Conventions Score 2- This response shows reasonable
control of grammatical conventions, although errors are
present in sentence formation and the spelling of difficult
words.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
38
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TA-13
1/0
Training set A, Paper 13
Content Score 1-While this response focuses on the topic
(OCS is A good program to be in. It dezined to. Help you
grauwate ), the organizational structure does not establish
a relationship between ideas and events. The unrelated,
undeveloped details are presented in a random fashion and
lead to confusion (You have to get so many hour unpayed
work At the school. Befor you can go off campes. Then
after you get your hours you can go off capes).
Compare the content score to Guide Paper G-3.
TA-14
1/0
Conventions Score 0- Dense, varied, and severe sentence
formation, usage, and mechanical errors are present in this
response.
Training set A, Paper 14
Content Score 1- While the focus of this response is clear
(in this coure of biode you will git a little more help then
the other Class’s), the organizational structure fails to
establish connections between and among ideas. Although
there is an attempt to support the topic, the response
consists of a sparse list of activities.
Compare the content score to Guide paper G-3.
Conventions Score 0- Dense, varied, and severe sentence
formation, usage, and mechanical errors impede the
meaning of this response.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
39
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
TA-15
2/1
Training set A, Paper 15
Content Score 2- This response focuses on the many ways
in which OCS helps students but the list-like
organizational structure establishes little relationship
between ideas and events. The unrelated, undeveloped
details lead to major lapses in the logical progression of
ideas (In this program you will have to obtain a certain
amount of work hours you can get these hours by picking
up trash, painting, cleaning the cafeteria, and school or by
doing work outside of school.).
Compare the content score to Guide paper G-5.
Conventions Score 1-This essay displays minimal control
of grammatical conventions with sentence formation,
usage, punctuation, and spelling errors.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
40
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Qualification Set
GRADE 10
Content & Conventions
Scoring Rubric
Applications
This publication and the information contained within must not be used for personal or
financial gain. North Carolina LEA school officials and teachers, parents, and students may
download and duplicate this publication for instructional and educational purposes only.
Others may not duplicate this publication without prior written permission from the North
Carolina Department of Public Instruction (NCDPI) Division of Accountability Services/North
Carolina Testing Program.
© 2010 All Rights reserved. This document may not be reproduced by any means, in whole or
in part, without prior written permission from the North Carolina Testing Program, Raleigh,
North Carolina 27601-2825.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
41
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Score Tally Sheet: Qualification Set I
You may use this sheet to practice scoring the following student responses. Record your scores
for Content and Conventions and then compare them against the actual scores located at the end
of this section.
PAPER #
My
Content
Score
My
Conventions
Score
State
Content
Score
State
Conventions
Score
1
2
3
4
5
6
7
8
9
10
% Agree 2
2
In order to maintain industry standard and NC scoring requirements of at least 70%, you must
have an exact agreement in Content and Conventions for 7 of the 10 responses.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
42
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
Qualification Set Annotations
Paper
Score
Notes
Content/Conventions
QI-1
2/0
Qualification Set 1, Paper 1
Content Score 2- In this response the focus is clear (The
thing that I do in the OCS is git to go to work). The
organizational structure does not always establish
connections among events, and the support, though
specific at times, is repetitive overall (My favet thing that
we do for fun is going to the movies and we git to veto on
what we want to see. Sometime your job can be fun on
what you pick).
Compare the content score to Guide Paper G-5.
Conventions Score 0- This response demonstrates a lack
of control of grammatical conventions such as sentence
formation, standard usage, and mechanics.
QI-2
3/1
Qualification Set 1, Paper 2
Content Score 3- This response maintains a generally
clear focus (The Occupational Course of Study it’s a Fun
Program to be in because you get to Learn About what to
say in a interview and what not to say), though minor
lapses are present. The organizational structure establishes
relationships between ideas, and ideas progress logically (I
see Now why the teachers Push us so hard they want us to
sucess in Life and Never give up. Sometimes it is not
Always fun and games but at the end of the day it is worth
it).
Compare the content score to Guide Paper G-8.
Conventions Score 1- With various sentence formation,
usage, and spelling errors, this response shows minimal
control of mechanical conventions.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
43
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
QI-3
1/0
Qualification Set 1, Paper 3
Content Score 1- While the topic of this response is clear
(the ocs program help you get a good A Job it help you
Find A good house), the brief organizational structure does
not sustain the focus or the relationships between ideas.
Support and elaboration are attempted, but details are
sparse.
Compare the content score to Guide Paper G-1.
Conventions Score 0- This brief response lacks control of
grammatical conventions appropriate to the writing task.
QI-4
2/1
Qualification Set 1, Paper 4
Content Score 2- Though brief, this response maintains
focus on the topic of what makes OCS fun. There are a
few specific details (three Engishes and three Prep’s and
three Science’s. . . . pound cake and cookies, Special
Olymipcas), but ideas are undeveloped and presented in a
list-like fashion, with little relationship established
between the ideas.
Compare the content score to Guide Paper G-5.
Conventions Score 1- Although some errors occur in
spelling, usage, and punctuation, this response exhibits
minimal control of grammatical conventions.
QI-5
1/0
Qualification Set 1, Paper 5
Content Score 1- The focus of this response is unclear (…
I am in the OCS program and it help me out some). The
organizational structure fails to establish connections
among events, and the sparse support is vague (The OCS
program is something that can and will help you get a Job.
When you sin up for the $40 dollers you can’t have a Job).
Compare the content score to Guide Paper G-1.
Conventions Score 0- This brief response is insufficient
to demonstrate control of grammatical conventions.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
44
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
QI-6
3/2
Qualification Set 1, Paper 6
Content Score 3- This response maintains a generally
clear focus on the ways in which OCS can benefit a new
student. The organizational structure establishes
relationships between and among ideas that result in a
reasonably complete, logical progression of ideas (If your
in the 9th grade you will get to work in the lunchroom the
things you get to do is sweep, mop, take out trash, Clean
windows and a little bit more fun things).
Compare the content score to Guide Paper G-8.
Conventions Score 2- With few errors, this response
displays reasonable control of grammatical conventions.
QI-7
4/2
Qualification Set 1, Paper 7
Content Score 4-Maintaining a clear focus throughout,
this response has an organizational structure that
establishes relationships between and among ideas that
progress logically throughout (Sometimes we have went
out in the community and done odd jobs. We went out and
decorated Main Street for our school parade. Then
another time we help some elderly gentalmen unload a
truck load of very heavy christmas trees). Details are
specific and developed and are related to and supportive of
the topic.
Compare the content score to Guide Paper G-11.
Conventions Score 2- With only a few errors, this
response displays reasonable control of grammatical
conventions.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
45
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
QI-8
1/0
Qualification Set 1, Paper 8
Content Score 1- The focus of this minimal response is
unclear, and the organizational structure consists of ideas
presented in a random fashion. Although there is an
attempt to support the topic, the sparse details are
unrelated and confusing.
Compare the content score to Guide Paper G-1.
Conventions Score 0- Dense, varied, and severe sentence
formation, usage, and mechanical errors impede the
meaning of this response.
QI-9
2/2
Qualification Set 1, Paper 9
Content Score 2- The response focuses on the benefits of
the OCS program, but the list-like organizational structure
establishes little relationship among events. The unrelated,
underdeveloped details create major lapses in the logical
progression of ideas.
Compare the content score to Guide Paper G-6.
Conventions Score 2- With few errors, this response
displays reasonable control of grammatical conventions.
QI-10
1/0
Qualification Set 1, Paper 10
Content Score 1- This very minimal response presents a
random list of ways in which the OCS program helps a
new student.
Compare the content score to Guide Paper G-1.
Conventions Score 0- This brief response is insufficient
to demonstrate control of grammatical conventions.
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
46
NCEXTEND2 OCS Writing Alternate Assessment
at Grade 10
Trainer Manual 2010
CONTACT INFORMATION
ACCOUNTABILITY DIVISION
Tammy Howard, NCDPI, Test Development Section Chief
THoward@dpi.state.nc.us
Jim Kroening, NCDPI, Senior Education and Evaluation Consultant
for Performance Assessments
JKroening@dpi.state.nc.us
Akia Beverly-Worsley, NC State University – Technical Outreach for Public Schools (TOPS),
Education Consultant for NC Writing Assessments
Akia_Worsley@ncsu.edu
Additional Writing Assessment information and resources may be found at:
www.ncpublicschools.org/accountability/testing/writing
INSTRUCTIONAL SERVICES DIVISION
Cindy Williamson, NCDPI, K12 Curriculum, Instruction and Technology Director
CWilliamson@dpi.state.nc.us
Vinetta Bell, NCDPI, English/Language Arts Consultant
Grades 9-12
VBell@dpi.state.nc.us
Freda Lee, NCDPI, Exceptional Children’s Division Consultant
Grade 10 OCS
FLee@dpi.state.nc.us
Additional Instructional information and resources may be found at:
www.learnnc.org/dpi/instserv.nsf
NCDPI Division of Accountability Services/Test Development Section
Summer 2010
47