Interpretive Guide to the
WinScan Score Reports for
NCEXTEND1 Assessments
4/21/2010
North Carolina Department of Public Instruction
STATE BOARD OF EDUCATION
The guiding mission of the North Carolina State Board of Education is that every public school
student will graduate from high school, globally competitive for work and postsecondary
education and prepared for life in the 21st Century.
WILLIAM C. HARRISON
Chairman :: Fayetteville
REGINALD KENAN
Rose Hill
JOHN A. TATE III
Charlotte
WAYNE MCDEVITT
Vice Chair :: Asheville
KEVIN D. HOWELL
Raleigh
ROBERT “TOM” SPEED
Boone
WALTER DALTON
Lieutenant Governor ::
Rutherfordton
SHIRLEY E. HARRIS
Troy
MELISSA E. BARTLETT
Roxboro
JANET COWELL
State Treasurer :: Raleigh
CHRISTINE J. GREENE
High Point
PATRICIA N. WILLOUGHBY
Raleigh
NC DEPARTMENT OF PUBLIC INSTRUCTION
June St. Clair Atkinson, Ed.D., State Superintendent
301 N. Wilmington Street :: Raleigh, North Carolina 27601-2825
In compliance with federal law, NC Public Schools administers all state-operated educational programs,
employment activities and admissions without discrimination because of race, religion, national or ethnic origin,
color, age, military service, disability, or gender, except where exemption is appropriate and allowed by law.
Inquiries or complaints regarding discrimination issues should be directed to:
Dr. Rebecca Garland, Chief Academic Officer :: Academic Services and Instructional Support
6368 Mail Service Center, Raleigh, NC 27699-6368 :: Telephone: (919) 807-3200 :: Fax: (919) 807-4065
Visit us on the Web:: www.ncpublicschools.org
NC Dept. of Public Instruction | Accountability Services | Test Development Section
I
Contents
Introduction ......................................................................................................................... 1
How to Use This Guide....................................................................................................... 1
The WinScan Reports ......................................................................................................... 1
NCEXTEND1 Assessments ............................................................................................... 3
Individual Student Reports ................................................................................................. 4
Class Roster Reports ........................................................................................................... 5
Score Frequency Reports .................................................................................................... 7
Index of Terms by Label Number ..................................................................................... 14
NC Dept. of Public Instruction | Accountability Services | Test Development Section
II
List of Figures
Figure 1. Sample Individual Student Report for NCEXTEND1 Assessments................... 4
Figure 2. Sample Roster Report. ......................................................................................... 5
Figure 3. Sample Retest Roster Report. .............................................................................. 6
Figure 4. Sample Score Frequency Report for NCEXTEND1 Assessments. .................... 7
Figure 5. The Template Design of the Score Frequency Report for NCEXTEND1
Assessments ........................................................................................................................ 8
Figure 6. The Header of the Score Frequency Report for NCEXTEND1 Assessments. ... 9
Figure 7. The Summary Table of the Score Frequency Report for NCEXTEND1
Assessments. ..................................................................................................................... 10
Figure 8. The Frequency Table of the Score Frequency Report for NCEXTEND1
Assessments ...................................................................................................................... 11
Figure 9. The Achievement Level Frequency Table of the Score Frequency Report for
NCEXTEND1 Assessments ............................................................................................. 12
Figure 10. The Summary Table of the Achievement Level Frequency Table of the Score
Frequency Report for NCEXTEND1 Assessments .......................................................... 13
NC Dept. of Public Instruction | Accountability Services | Test Development Section
III
Introduction
WinScan is a software application provided to schools by the North Carolina Department
of Public Instruction (NCDPI). This software permits administrators at the district level
to produce a variety of score reports on demand, including Individual Student Reports,
Achievement Level Frequency Reports, Class Roster Reports, Score and Frequency
Reports. The Interpretive Guide to the WinScan Score Reports for NCEXTEND1
Assessments is intended to help educators understand these reports and to inform decision
making at the student, classroom, school, and district levels. This guide will also help
administrators and educators explain test results to parents and the general public.
For each assessment, the North Carolina Department of Public Instruction produces an
interpretive guide that is designed specifically to help parents understand the Individual
Student Report (ISR). This document entitled Understanding the Individual Student
Report (UISR) is available online for every End-Of-Course, End-Of-Grade,
NCEXTEND1, and NCEXTEND2 assessment
(http://www.ncpublicschools.org/accountability/policies/uisrs). The UISR is designed for
parents and teachers; whereas, the interpretive guides are designed for teachers and
administrators at the school, district, and state levels. Together, these documents provide
guidance in interpreting the many reports that are generated by the WinScan software
application.
How to Use This Guide
This guide has been designed to give users quick access to the information needed when
interpreting a specific WinScan report. The table of contents can be used to identify a
sample report for any type of WinScan report. Users can learn about all of the key
features of the sample report by simply matching label numbers in the sample reports to
the label numbers in the Index of Terms by Label Number.
The WinScan Reports
Each WinScan report has a standard template. Except for the Individual Student Reports,
the standard templates can be modified through user-defined options. When the standard
report templates are combined with different options, assessments, and data filters, over
300 unique reports can be produced. This guide focuses on the most commonly used
reports for NCEXTEND1 assessments. Table 1 shows a list of the reports described in
subsequent pages and the audiences for which these reports are intended. The ISRs are
designed for students, teachers, parents, and school administrators. Class Rosters are
designed for teachers and school administrators. Score Frequency Reports and
Achievement Level Frequency Reports are designed for teachers, principals, district
administrators, and state administrators.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
1
Table 1. WinScan Reports and Intended Audience
Audience
Administrators
Report
Individual Student Report
Class Roster Reports
Parent
Teacher
School
Score and Achievement Level Frequency Reports
District
State
The WinScan reporting system can aggregate data at various levels, including class,
school, district, and state levels. Table 2 presents the reporting levels of each group-level
WinScan report.
Table 2. Reporting Levels for Group-Level WinScan Reports
Report
Class Roster Reports
Score and Achievement Level Frequency Reports
Class
Reporting Level
School
District
State
The WinScan reporting system can also summarize scores across various subgroups
including gender (male and female), ethnicity (American Indian, Asian, Black, Hispanic,
Multi-racial, and White), and grade level. Table 3 presents the standard reporting groups
available for each group-level WinScan report. School districts also have the option to
merge in nonstandard subgroups such as Comprehensive Exceptional Children
Accountability System Exceptionality (CECAS), Students With Disabilities, Targeted
Assistance Schools (reading and math), Limited English Proficient, and Migrant. When
multiple subgroups are selected, reports are produced for every combination of the
chosen subgroups.
Table 3. Standard Reporting Groups for Group-Level WinScan Reports
Groups
Report
Class Roster Reports
Score and Achievement Level Frequency Reports
All
Gender
Ethnicity
Gender &
Ethnicity
Note: Ethnicity includes the following subgroups: American Indian, Asian, Black, Hispanic, Multi-racial,
and White.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
2
As can be seen from Tables 2 and 3, users have many options when producing WinScan
reports, including many subject areas, four possible reporting levels, and four grouping
variables to choose from, resulting in over 300 individual reports.
NCEXTEND1 Assessments
During the school year, students are assessed using the NCEXTEND1 alternate
assessment to the following state-required North Carolina tests: the North Carolina Endof-Grade (EOG) Tests in Reading and Mathematics at grades 3–8; the North Carolina
EOG Test in Science at grades 5 and 8; the North Carolina Writing Assessment at Grade
10; and the North Carolina End-of-Course (EOC) Tests of English I, Algebra I, and
Biology at grade 10.
The NCEXTEND1 is an alternate assessment for students who have a significant
cognitive disability. The NCEXTEND1 is administered to eligible students at grades 3–8
and 10 as part of the statewide assessment program. These curriculum-based assessments
are specifically aligned to the North Carolina Standard Course of Study (NCSCS)
Extended Content Standards. The NCEXTEND1 measures grade-level extended content
standards using a variety of performance tasks. The achievement of students participating
in the NCEXTEND1 is measured based on alternate academic achievement standards for
the content area(s) being tested. Student scores in reading and mathematics and writing at
grade 10 from the NCEXTEND1 are used in the computation of school performance
composites as required by the state-mandated ABCs Accountability Program and for
determining adequate yearly progress (AYP) under Title I mandates of the No Child Left
Behind Act of 2001. The science test is required to comply with the federal No Child Left
Behind Act of 2001. Scores from the NCEXTEND1 science test are included in the
computation of the school performance composite as part of the ABCs Accountability
Program.
Key features of the NCEXTEND1 assessments:
•
The NCEXTEND1 assesses student achievement on the North Carolina Standard
Course of Study (http://www.ncpublicschools.org/curriculum/ncecs).
•
The assessment takes place during a designated testing window in the spring.
Each student has two assessors for each content area assessed with the
NCEXTEND1. There are 15 test items for reading, 15 items for mathematics, 12
items for science, and 7 items for writing.
•
Students are presented each item one at a time. Each assessor independently
scores the student’s response as correct or incorrect. Both assessors submit their
results for the student via an online system. The student’s score is calculated from
the assessors’ results.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
3
Individual Student Reports
For students at grades 3, 4, 6, and 7, the Individual Student Report (ISR) for the
NCEXTEND1 provides information concerning performance on the NCEXTEND1 for
reading and mathematics. For students at grades 5 and 8, the report includes information
concerning performance on the NCEXTEND1 for reading, mathematics, and science. For
students at grade 10, the report provides information concerning performance on the
NCEXTEND1 reading, mathematics, science, and writing. An ISR report is provided in
Figure 1. Key features are labeled and explained in the Index of Terms by Label Number.
1
2
3
4
5
6
1
2
3
4
5
6
NC1.
Dept.
of Public
Instruction
| Accountability
Services | Test Development
Section
Figure
Sample
Individual
Student
Report for NCEXTEND1
Assessments.
4
Class Roster Reports
The Class Roster Reports take on many different combinations. A class roster report can
contain grade-specific student scores for each content area independently or a class roster
report can contain grade-specific student scores for combinations of content areas. The
most typical combination for NCEXTEND1 is a class roster report that displays Reading,
Mathematics, Writing, and Science scores together on one report for a specific grade.
Figure 2 displays a sample NCEXTEND1 Roster Report. This report is often produced at
the class level and the school level. The report’s features and layout do not differ across
levels.
1
2
1
2
1
2
1
2
7
Figure 2. Sample Roster Report.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
5
Figure 3 displays a sample Retest Roster. This roster is often produced at the class and
school levels, reports the achievement level, and provides a column that indicates if a
student needs to take a retest (Label 8). For instance, the first student earned a Reading
Achievement Level of 2 and a Math Achievement Level of 1. This student needs to be
retested on the Reading assessment, but is not required to retest on the Math assessment.
With parental consent, students who earn an Achievement Level 1 may opt to retest. The
NCDPI does not retest on the Writing assessment.
A
2
8
2
8
2
2
8
Figure 3. Sample Retest Roster Report.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
6
Score Frequency Reports
Frequency tables are used to summarize scores. The Frequency Reports available in
WinScan are often used to summarize score information at the class, school, system, and
state levels. The WinScan Score Frequency Report presents the frequency, percent,
cumulative frequency, and cumulative percent of each score at a specific grade. These
reports can be created for each NCEXTEND1 assessment including Reading,
Mathematics, Science, and Writing. Figure 4 presents a sample Score Frequency Report
for an NCEXTEND1 Mathematics Assessments. The Index of Terms by Label Number
can be used to learn more about each labeled feature of this report. Figures 5–10 describe
each section and each label of the report.
10
12
9
11
13
1
14
15
16
17
18
15
16
17
18
16
19
1
2
13
19
2
A1
Figure 4. Sample Score Frequency Report for NCEXTEND1 Assessments.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
7
Figure 5 presents the template design of the Score Frequency Report. (The text of the
report has been masked to reveal the locations of each section of the report.) The template
consists of five sections: the header (F1), a summary table for scores (F2), a score
frequency table (F3), an achievement level frequency table (F4), and a summary table for
achievement levels (F5). Figures 6 through 10 describe the contents of each section.
A
F1
7
F2
1
F3
2
F4
F5
A1
Figure 5. The Template Design of the Score Frequency Report for NCEXTEND1
Assessments
NC Dept. of Public Instruction | Accountability Services | Test Development Section
8
Figure 6 displays the header which specifies the school year, the type of assessment
(general or alternate), the exam type (Mathematics, Reading, Science, or Writing), and
test administration type (Regular, Retest 1, or Retest 2). The LEASchCode (Label 9) is
listed as well as the SchoolName (Label 10). If this report were reporting district-level
data, the SystemCode would appear at Label 9 and the SystemName would appear at
Label 10. If this report were subdivided by subgroups, then the subgroup would appear
at Label 11. A separate report will appear for each subgroup included within the grouping
variable. If two grouping variables are selected, as in this example, then a separate report
will be produced for every combination of the categories within each grouping variable.
The grouping variable Sex (Label 11) includes male and female categories. The grouping
variable Ethnic (Label 12) includes American Indian, Asian, Black, White, Hispanic, and
Multi-racial categories. No report will be generated for subgroups that have no students.
A
F1
9
11
10
12
F2
1
F3
2
F4
F5
Figure 6. The Header of the Score Frequency Report for NCEXTEND1 Assessments.
Figure 7 displays a summary of the score frequency table. The first column displays the
range of possible scores. The Number of Observations column (Label 13) presents the
NC Dept. of Public Instruction | Accountability Services | Test Development Section
9
number of students from this particular school that sat for the assessment and received a
score. The Mean column (Label 14) indicates the average score. The average is the sum
of all scores included in the report divided by the number of scores in the report. The
sample shows 16 students received scores, and the average math score of this group was
5.94.
A
F1
1
13
14
F2
1
F3
2
F4
F5
A1
Figure 7. The Summary Table of the Score Frequency Report for NCEXTEND1
Assessments.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
10
Figure 8 presents the frequency table for 16 students. The MathScore column (Label 1)
presents every score earned by the 16 students. The Frequency column (Label 15)
presents the number of students that earned each score. For example, 2 students earned a
score of 6. The Percent of Total column (Label 16) presents the percent of students that
earned a given score (number of students that earned the score divided by total number of
observations). This column shows that 12.50% of the students earned a score of 6. The
Cumulative Frequency column (Label 17) presents the total number of students that
earned up to and including a score in a given row. This column shows 9 students earned
up to and including a score of 6. The Cumulative Percent column (Label 18) displays the
percent of students that earned up to and including a score in a given row. In the sample
shown, 56.25% of the students earned up to and including a score of 6.
A
F1
15
1
16
17
18
F3
2
A1
Figure 8. The Frequency Table of the Score Frequency Report for NCEXTEND1
Assessments
NC Dept. of Public Instruction | Accountability Services | Test Development Section
11
Figure 9 presents the Achievement Level Frequency table for 16 students. The Math Ach
Level column (Label 2) presents every achievement level earned by the 16 students. The
Frequency column (Label 15) presents the number of students that earned each
achievement level. For example, 9 students earned an Achievement Level of 1. The
Percent of Total column (Label 16) presents the percent of students that earned a given
achievement level (number of students that earned the score divided by total number of
observations). This column shows that 56.25% of the students earned an Achievement
Level of 1. The Cumulative Frequency column (Label 17) presents the total number of
students that earned up to and including a given achievement level. This column shows
16 students earned up to and including an achievement level of 2. The Cumulative
Percent column (Label 18) displays the percent of students that earned up to and
including a given achievement level. This columns shows 56.25% of the students earned
up to and including an Achievement Level of 1.
A
F1
F2
1
F3
2
15
16
17
18
F4
F5
A1
Figure 9. The Achievement Level Frequency Table of the Score Frequency Report for
NCEXTEND1 Assessments
NC Dept. of Public Instruction | Accountability Services | Test Development Section
12
Figure 10 presents a summary of the Achievement Level Frequency table for 16 students.
The Achievement Level column (Label 2) displays the achievement levels grouped into
proficient (Levels III and IV) and not-proficient (Levels I and II) categories. The
Number of Observations column displays (Label 13) the number of students that were
classified into proficient/not-proficient categories. The Report Card column (Label 19)
provides a line for LEAs to record the score that is reported on the report card to see if
the numbers match. The Percent column (Label 16) displays the percent of students that
were classified into each proficient/not-proficient categories.
A
F1
F2
1
F3
2
2
13
19
16
19
F5
A1
Figure 10. The Summary Table of the Achievement Level Frequency Table of the Score
Frequency Report for NCEXTEND1 Assessments
NC Dept. of Public Instruction | Accountability Services | Test Development Section
13
Index of Terms by Label Number
1 Score - The number of questions the student answers correctly is called a raw score. The raw
score is converted to a score. This score is determined by a formula that involves the results
independently assigned by each individual assessor. The scores for NCEXTEND1
assessments differ from general assessments in that the scores for NCEXTEND1
assessments do not permit comparisons over time. For the general assessments, the scale
score depicts growth in achievement from year to year. For the general assessments, you can
compare the scale scores on the End-Of-Grade test given during the last three weeks of the
previous school year and the End-Of-Grade test given during the last three weeks of the current
school year to determine a student’s growth. In contrast, NCEXTEND1 assessments are not on a
vertical scale and do not permit year-to-year comparisons.
2 Achievement Level – Achievement Levels group scores according to different levels
of performance as defined by content standards. Four achievement levels (i.e., Levels I,
II, III, and IV) are reported. Achievement levels are predetermined standards that allow a
student’s performance to be measured against alternate academic achievement standards.
These alternate academic achievement standards are based on extended content standards
and are listed at: http://sbepolicy.dpi.state.nc.us/policies/GCS-C029.asp?pri=01&cat=C&pol=029&acr=HSP.
3 Top Row of the Scale Diagram - The diagram to the right of this label indicates the
range of student scores from the lowest possible score to the highest possible score.
4 Middle Row of the Scale Diagram - This diagram shows the four achievement
levels and their relation to the possible student scores. The standard for grade-level
proficiency is a test score at Achievement Level III or above on the NCEXTEND1 tests.
The shaded areas in the diagram indicate proficiency Achievement Levels III and IV.
5 Bottom Row of the Scale Diagram - This diagram shows a student’s score in
relation to the range of possible scores and in relation to the four Achievement Levels. A
student’s score is represented by a closed diamond (♦).
6 Achievement Level Description - The description of the achievement level is
reported for a student’s performance in each subject area. A complete listing of the
achievement level descriptors for Achievement Levels I–IV by grade level may be found
at http://sbepolicy.dpi.state.nc.us.
7 Group Means – The average of a group of scores. The mean is the sum of all scores
in the roster divided by the number of scores in the roster.
8 Needs Retest – Students earning Achievement Level II are required to be retested.
Some LEAs may have more rigorous retesting policies. However, local policies cannot be
NC Dept. of Public Instruction | Accountability Services | Test Development Section
14
developed that specifically exclude any particular group of students, such as students
with disabilities or students identified as limited English proficient. Policy ID Number
GCS-C-031 describes the retesting policy and can be found at
http://sbepolicy.dpi.state.nc.us.
9 Reporting Level Code – The reporting level code indicates the class, school, district,
or state code for which the report was produced.
10
Reporting Level Name - The reporting level name indicates the class, school,
district, or state name for which the report was produced.
11
Gender Label – The gender label displays the gender of the group included in the
report.
Ethnicity Label – The grouping variable Ethnic displays the ethnicity of the group
included in the report. Ethnicity includes American Indian, Asian, Black, White,
Hispanic, and Multi-racial categories.
12
13
Number of Observations – This number is the count of all students who received a
score for the group specified for the report.
14
Mean - The average of a group of scores. The mean is the sum of all scores included
in the report divided by the number of scores in the report.
15
Frequency - The Frequency column presents the number of students that earned each
score.
16
Percent - The Percent of Total column presents the percent of students that earned a
given score (number of students that earned the score divided by total number of
observations).
17
Cumulative Frequency – The value in the Cumulative Frequency column in a
frequency table is the total number of students that earned all scores up to and including
the score in the same row.
18
Cumulative Percent - The value in the Cumulative Percent column in frequency
tables is the percent of students that earned all scores up to and including the score in the
same row.
19
Report Card Score Line - LEAs are often interested in the percent of students
scoring at Levels III and IV. LEAs can use this line to write on this report what is
reported on the North Carolina Report Card (http://www.ncreportcards.org/src) to see if
their numbers match.
NC Dept. of Public Instruction | Accountability Services | Test Development Section
15