The National External Diploma Program
Writing Assessment in the New NEDP
Linda Taylor, CASAS Director of Assessment Development
National External Diploma Program Council Conference
October 2012, Providence, RI
www.nedp.org
1
The Future of NEDP Writing
• CASAS’ review of NEDP Competencies and Tasks
revealed a need for increased rigor.
• In terms of Writing, there is a need for:
– More writing
– More challenging writing
• In current NEDP development work, CASAS is
incorporating more writing.
• The CASAS Written Prompt has been piloted for use in
Diagnostics.
www.nedp.org
2
Writing Assessment in the New NEDP
Four general types of writing in NEDP Tasks:
– Extended Response
• Includes the Written Prompt in Diagnostics
• Each Competency Area will have at least one Extended
Response item
– Short Constructed Response
– Specific Applications – e.g., letters, resume, power
point presentation
– Fill Matrix
www.nedp.org
3
CASAS Written Prompt
January 2013 - June 2014
CASAS will phase in the CASAS Written Prompt to replace
the current NEDP writing process for Diagnostics
• January 2013 - June 2013
– Agencies may voluntarily transition to the CASAS Written
Prompt
• July 2013 – June 2014
– Agencies to be trained will be assigned timeframes for
training
• CASAS has developed a self-paced electronic training that
will be available 24/7
• Additional implementation support will be scheduled
periodically
www.nedp.org
4
NEDP Written Prompt Background
• The CASAS Functional Writing Assessment (FWA)
Picture Task does not assess the highest levels of ability
• The CASAS National Consortium Writing Committee
identified a need to develop a Written Prompt Task to
assess higher level writing ability
• The Singapore Workforce Development Agency (WDA)
funded and worked with CASAS to develop this new
writing assessment from 2006 – 2008
• Tryouts were also conducted in CT in 2007 and from
2010 – 2012 to ensure appropriateness in the US
www.nedp.org
5
Background
• The Written Prompt Task is a new writing
assessment in the CASAS Functional Writing
Assessment series.
• The task is to write an extended response
generated by a written essay topic.
• Takes 45 minutes to one hour
• Holistically scored on a scale of 1 – 5
• Diagnostic feedback is provided
• Designed to be computer-delivered
www.nedp.org
6
Development
• Target population
– Advanced adult ESL and ABE learners and workforce
participants
– GED preparation, high school programs, NEDP
– Transition to post-secondary
• Content of written prompts must be appropriate for
both employed and unemployed test takers
• Developed 10 prompts; selected 5 for US field
testing
www.nedp.org
7
Written Prompt Testing Process
– Test Directions embedded in the NEDP software
– New spell check feature tutorial is included
– Client enters writing into NEDP software
– Two trained and certified NEDP assessors score
independently.
• Can be two assessors or one assessor and one advisor
– If they don’t agree, they discuss and come to
consensus
– Client must score 3 (out of 5) to enter General
Assessment
www.nedp.org
8
www.nedp.org
9
www.nedp.org
10
www.nedp.org
11
www.nedp.org
12
www.nedp.org
13
Written Prompt Rubric Development
• CASAS researched academic essay rubrics
• Wanted continuity with CASAS Functional Writing
Assessment Picture Task rubric
• Created/revised numerous versions
• Issues: Voice/style, key elements, formatting of rubric for
holistic scoring
• Scored both Singaporean and US responses
• Trained Singapore staff and part-time scorers
– good success rate
www.nedp.org
14
Providing Diagnostic Feedback
• Holistic scoring with targeted feedback on
relevant aspects of the writing sample
• Drop-down menu items AND
• Open-ended comments
• Show example
www.nedp.org
15
Results from Phase 1 Pilot
(2009 – May 2010)
 Collected information about the writing ability
levels of the NEDP population
 Adjusted the Written Prompt rubric to distinguish
more clearly between rubric score points
 Revised the wording in the directions to the
examinee (“Write about the essay topic” instead
of “Write three paragraphs…”)
www.nedp.org
16
Results from Phase 1 Pilot
• Collected writing samples for scorer training
materials
• Realized that computer delivery lends itself to
including a “Spell-check” feature which would
make the writing assessment more realistic
• Converted to CASAS eTests software delivery
platform to conduct more tryouts (2010)
www.nedp.org
17
Written Prompt -- Current Status
• Created and tried out self-paced scorer training
materials (2011 – 2012)
– Currently converting into Moodle platform
• Now embedding the test directions and delivery
into the NEDP online software
• CASAS will work with State Departments of Ed
to determine training schedule and who will
score
www.nedp.org
18
Written Prompt Research Studies
• Inter-rater Reliability studies
– Two local program scorers score the same
responses
• Parallel forms
– At least 100 test takers take a set of two
prompts on the same day or within one or two
days
www.nedp.org
19
Writing in Generalized Assessment
• Extended writing is included in all NEDP
Competency Areas (except Health)
• These items require writing based on analyzing or
summarizing information from higher level reading
• Scoring Guidelines are provided for each item that
requires writing.
• Items requiring writing are judged on content and
ability to clearly communicate in writing. Errors in
mechanics are only considered if they interfere with
clarity of communication.
www.nedp.org
20
NEDP Writing Foundation Competency 3
Write to:
• present, explain and examine information and ideas
• state a position, with clear and coherent supporting
arguments
• describe experiences and events, real or created, and
• express thoughts, feelings and emotions,
• using writing techniques, organizational structures, word
choices, English language conventions, and styles of
expression that are effective and appropriate to the
purpose.
www.nedp.org
21
Writing Assessment in the New NEDP
Four general types of writing in NEDP Tasks:
– Extended Response
• Includes the Written Prompt in Diagnostics
• Each Competency Area will have at least one Extended
Response item
– Short Constructed Response
– Specific Applications – e.g., letters, resume, power
point presentation
– Fill Matrix
www.nedp.org
22
Extended Written Response
Type of Writing
Assessment
Phase 1
4-5 paragraphs
Phase 2
3-4 paragraphs
PI
Item Title
Description of Writing
Required
41.3 The Legal
-4
System
Write an argument for
and against Miranda
Rights
31. Critique Live
1-2 Performance
Apply vocabulary;
evaluate quality of
cast, costumes, music,
special effects
www.nedp.org
23
Short Paragraph
Phase 2
3-4 sentences
43.2 National
Geography
Explain how
geographic feature has
been affected by
environmental factor
Phase 1
2 or more
sentences
2.0
Apply vocabulary;
analyze central ideas;
evaluate literary
elements; make
inferences
Read literary
texts ̶
literature
www.nedp.org
24
Extended Narrative: The Legal System
4. Required: Client must write four or five
paragraphs that cover components a, b and c,
below. Assess this response for content only;
mistakes in mechanics are to be considered only if
they interfere with clarity
www.nedp.org
25
Extended Narrative: The Legal System
• Guideline: Check each box that is demonstrated.
If PTA is required, the response must be
rewritten; however, boxes that have already
been checked will not require re-demonstration.
 a.
Identifies and summarizes the main
idea behind the argument for the reading of the
Miranda Rights to an individual under arrest
www.nedp.org
26
Extended Narrative: The Legal System
 b. Identifies and summarizes the main idea behind
the justification for not reading the Miranda Rights to a
suspected terrorist
 c. Cites two resources researched at end of the
response from which main ideas were drawn for arguments
for and against reading the Miranda Rights, using citation
guidelines in R4
D
 ND You provided an:
 incorrect  incomplete
Written response describing circumstances under which
the Miranda Rights should or should not apply to an
individual, citing two sources of information.
www.nedp.org
27
Short Paragraph: Science Issues in the News
3. Required: Two sentences interpreting data in line graph of temperature
change:
 one sentence summarizing temperature movement between 1900
and 2000, and
 one sentence that makes prediction for next 100 years; prediction
may mention general trend or more specific temperature increase
Guideline: Assess this response for content only; mistakes in mechanics
are to be considered only if they interfere with clarity.
For sample responses, see following box.
 D  ND You provided an:
 incorrect  incomplete
statement on temperature changes displayed in graph and
prediction of future temperature change.
www.nedp.org
28
Sample Responses
3. Sample response, acceptable:
Between the years of 1900 and 2000, average
temperatures have increased two degrees due to
human effects. Based on this data, we can predict
that average temperatures will increase a similar
amount in the next 100 years.
Sample response, not acceptable:
This graph shows effect of natural forces and human
effects. Temperatures are changing.
www.nedp.org
29
One or Two Sentences
One or two
sentences
53.1 Elements of
the Scientific
Method
www.nedp.org
Provide evidence for
hypothesis, dependent
and independent
variables; identify
correct steps in
experiment
30
Specific Applications: Resume
Resume
62.3 Creating a
Resume
Design and write a
resume for a job notice
identified in earlier
assessment
(6 content criteria;
5 formatting criteria)
www.nedp.org
31
Specific Applications: Formal Letter
49.1-2 Consumer
Complaints
Write a formal complaint letter
(4 types of information; 6
elements of a formal letter)
62.4
Write an email cover letter for
job notice identified in earlier
assessment (9 criteria including
no errors in spelling, grammar
and punctuation)
Employment Cover
Letter
www.nedp.org
32
Formal Letter: Consumer Complaints
• 4a. Required: A complaint letter to Copper City
Electronics with each of the four types of
information below:
• 1) stating the problem
• 2) asking for specific action
• 3) including copies of all related documents, if
applicable
• 4) providing contact information
www.nedp.org
33
Formal Letter: Consumer Complaints
4b. Required: All six elements of a formal letter, following
guidelines for written statements and placement of each
convention
• 1) Date
• 2) Salutation
• 3) Sender Address
• 4) Receiver Address
• 5) Closing
• 6) Signature
Guideline: Check each box that is demonstrated. If PTA is
required, the letter must be rewritten; however, boxes that
have already been checked will not require
re-demonstration
www.nedp.org
34
Formal Letter: Consumer Complaints
Click Sample Letter Response and Letter
Formats to guide evaluation. Any of the six
displayed formats is acceptable.
 D  ND
You provided a
 incorrect  incomplete
formal complaint letter containing four types of
information and six formal elements.
www.nedp.org
35
Topics for Discussion
• Will writing mechanics (e.g., spelling,
punctuation) be assessed whenever writing is
required in a task?
– No, in short written responses, the focus will
be on content.
• Other Topics?
www.nedp.org
36
NEDP Contact Information
• Jim Harrison
Director, National External Diploma Program
jharrison@casas.org
800-255-1036, ext. 191
• Melissa Dayton
NEDP Coordinator
mdayton@casas.org
800-255-1036, ext. 181
Linda Taylor
CASAS
Director of Assessment Development
ltaylor@casas.org
800-255-1036, ext. 186
• Christine O’Hara
NEDP Operations
cohara@casas.org
800-255-1036, ext. 135
37
www.nedp.org
37