Lessons Learned:
Writing Collection of Evidence
2007 Scoring Results…
Implications for Instruction
Steve Pearse, Ed.D.
Writing COE Specialist
CAA Options / OSPI
Washington State Assessment Conference
Guiding Principle for the COE…
and for Scoring:
“…only objective assessments that are comparable
in rigor to the state assessment [WASL] are
authorized as an alternative assessment.”
—ESSB 6475, Sec. 1
Writing COE Scoring Results
2
Results of COE Writing Scoring
• 31 Writing Collections met sufficiency requirements and were scored as per ondemand WASL Writing responses (expository + persuasive).
• Each Collection was scored twice, with any non-adjacent scores triggering a
third score; the two scores were then added to arrive at a total score.
• Collections were scored holistically—across all Work Samples—for content,
organization & style (COS) and for conventions (CONV).
• As for Reading, we were unable to set standard this year, due to the small
number of COE submissions.
• The Writing Standard Setting Panel determined that seven (7) Collections were
proficient*.
* see document, Writing Working Descriptions of Performance—Grade 10
Writing COE Scoring Results
3
Which Collections Met Proficiency?
• Proficient Collections align with Proficient Writing Descriptors, as listed in
Writing Working Descriptions of Performance—Grade 10, March 2004.
• Proficient Writing Collections’ holistic scores—across all Work Samples—
meet or exceed WASL standard as per aligned WA Writing Grade 10
Baseline Anchor Set responses.
• Proficient Collections incorporate a variety of tasks/prompts, often
including authentic, classroom-based assignments.
• Each of the seven (7) proficient writing collections originates from a
different Washington state school district.
Writing COE Scoring Results
4
Which Collections Did NOT Meet Proficiency?
(continued)
• These Writing Collections fall below 9, the cut line the Standard Setting
Panel established for proficiency this year.
• Non-Proficient Writing Collections reflect Basic Writing Descriptors, as
per Writing Working Descriptions of Performance—Grade 10, March
2004.
• Non-Proficient Writing Collections align holistically with below-standard
Anchors used to score the WASL.
• Non-Proficient (Basic) Collection Work Samples often reflect a lack of
emphasis on process.
Writing COE Scoring Results
5
Which Collections Did NOT Meet Proficiency?
• Persuasive responses among these Basic Collections often lack a
sense of energy and/or purposefulness.
• Formulaic organization and/or ineffective introductions and conclusions
characterize many Basic Collection Work Samples.
• These Basic Collections frequently reflect careless editing—or an
apparent lack of attention to writing conventions (CONV).
• Many Basic Collections include Work Samples whose topics or prompts
seem inappropriate for or unhelpful for student-writers.
Writing COE Scoring Results
6
Lessons Learned
• Purposeful instruction is essential to student success on the writing
COE.
—Many Collections’ Work Samples and documentations of process suggest minimal
instructional support.
• Carefully-chosen topics or tasks (prompts) often correlate with
effective writing.
—Uninspiring, misleading, overly general, and/or narrowly-focused prompts align with
an apparent lack of commitment to topic.
• Engaging in a purposeful process contributes to writing performance.
—Many Work Samples reflect “missed opportunities,” suggesting the potential to
meet proficiency but failing to do so from initial work (pre-writing) to final drafts.
—Little evidence of process characterizes many extended-time as well as ondemand Work Samples.
Writing COE Scoring Results
7
Lessons Learned (continued)
• Taking advantage of word processing and attendant software (e.g.,
Spell Check) contributes to a higher level of writing performance.
—Many Writing COE student-writers failed to do so, as evidenced by
minimal or overly general Work Sample process documentation and by final
drafts that were largely unchanged—and unimproved—from initial draft(s).
• Research-based prompts and assignments should invite studentwriter interpretation and analysis. Assignments or prompts that invite
plagiarism place student-writers at risk of not presenting their own work.
—Some Work Samples incorporating research reflect a lack of personal
commitment (voice), focus and approach (ideas & organization), and/or
expression (sentence variety & word choice).
Writing COE Scoring Results
8
Augmented Collection Requirements
• The cut score for proficient Writing Collections was 9 points.
• To be eligible for augmentation, Writing Collections must have earned a
score of 8.
• Eligible students MUST submit 3 additional Work Samples:
one persuasive sample, one expository sample, and one
additional—either persuasive or expository—sample.
Writing COE Scoring Results
9
POINTS
4
3
2
1
Writing Content, Organization & Style (COS) Scoring Criteria: Evidence of Skill
• Has clear main idea/position and stays focused on that main idea/position
• Elaborates by using reasons/arguments, well-chosen and specific details, examples, anecdotes, facts and/or statistics as evidence to support ideas
or a position
• Includes information that is thoughtful and useful for the audience to know
• Organizes writing to make the best cases to explain ideas or support a position
• Introductions engage reader’s attention
• Writes conclusions that leave the reader with something to think about
• Organizes writing into effective, cohesive paragraphs • Provides transitions which clearly serve to connect ideas
• Uses language effectively by exhibiting word choices that are effective and appropriate for intended audience and purpose
• Writes (where appropriate) sentences or phrases that are varied in length and structure
• Provides the reader with a sense of the person behind the words
• Maintains adequate focus on main idea/position
• Elaborates by using reasons/arguments, adequate details, examples, anecdotes, facts and/or statistics as evidence to support ideas or a position
• Includes some information that is thoughtful and useful for the audience to know
• Has logical organizational pattern, although some lapses may occur
• Introductions sometimes draw the reader into the main idea or position
• Writes conclusions that leave the reader with something to think about
• Organizes writing into meaningful paragraphs • Provides adequate transitions which serve to connect ideas
• Uses adequate language and appropriate word choices for intended audience and purpose
• Writes sentences or phrases that are somewhat varied in length and structure
• Provides the reader with some sense of the person behind the words
• Has a broad or inconsistent focus on main idea or position
• Includes some supporting details and may include listed, extraneous, and/or loosely related material
• Sometimes includes information that is thoughtful and useful for the audience to know
• Shows attempt at an logical organizational pattern
• Introductions are often formulaic, predictable
• Conclusions are often repetitious
• Organizes writing into loosely structured and/or unfocused paragraphs • Provides adequate transitions which are formulaic, weak or inconsistent
• Uses common, limited and/or predictable vocabulary which may be inappropriate for intended audience
• Shows limited variety in sentence length and structures
• Provides the reader with a limited sense of the person behind the words
• Demonstrates little or no focus
• Provides few supporting details which may be inconsistent or interfere with the meaning of the text
• Rarely includes information that is thoughtful and useful for the audience to know
• Has little evidence of organizational pattern
• Introductions are lacking or undeveloped • Conclusions are lacking or undeveloped
• Often only one paragraph that demonstrates a lack of organizing ideas into paragraphs • Provides transitions that are poorly used or fails to provide
transitions
• Has limited or inappropriate vocabulary for intended audience, purpose, and form
• Haas little or no variety in sentence length or structures • Provides the reader with little sense of the person behind the words
Writing COE Scoring Results
10
Writing Conventions (CONV) Scoring Criteria
Points
2
Description
Consistently follows the rules of Standard English for:
• grammar and usage
• spelling of commonly used words
• capitalization
• punctuation
• exhibits the use of complete sentences except where purposeful phrases or clauses are used for effect
• indicates paragraphs consistently
1
Generally follows the rules of Standard English for:
• grammar and usage
• spelling of commonly used words
• capitalization
• punctuation
• exhibits the use of complete sentences except where purposeful phrases or clauses are used for effect
• indicates paragraphs for the most part
0
Mostly does not follow the rules of Standard English for:
• grammar and usage
• spelling of commonly used words
• capitalization
• punctuation
• exhibits errors in sentence structure that impede communication
• mostly does not indicate paragraphs
Writing COE Scoring Results
11
Sample COE Writing Collections…
Representative Basic (Non-Proficient) COE Collection:
“G5” (Green binder, Collection #5)
Score: 8 (below cut line of 9)
(Scorers independently scored this Collection as a 3,2 for COS
and as a 2,1 for CONV = 8 total scoring points.)
This Collection consists of six (6) Writing Samples, all but one of which
responding to Grade 10 WASL expository or persuasive prompts
administered in previous years. Although the student-writer chose to
provide drafts as evidence of process (as opposed to explanations),
final texts reflect only minor revisions of initial work. The following
Work Samples typify the Collection as a whole…
Writing COE Scoring Results
12
Work Sample #3: “Memorable Class”
On-Demand / Expository
Prompt: By the time you are a sophomore in high school,
you have been in many different classes. Think about a
memorable class. Write a multiple-paragraph letter to a
teacher explaining why that class was memorable.
[Source: WASL expository prompt, 2002]
Writing COE Scoring Results
13
“Memorable Class”
Dear Teacher,
My name is [
] and I’m writing to you to tell you my most memorable class
during my sophmore year. I had many fun classes that were memorable
because I have memories from one class. My most memorable class was Deca.
Deca was my most memorable class during my sophmore year. It is my
most memorable class for many reasons. One of thease reasons was because
me and my friends had it together, so we had a lot of fun. Me and my friends
had a lot of fun because we got to work in the ramshack together and listen to
music.
Another reeson Deca is my most memorable class is because you got to do
many cool thinks in there. Like we had to do real job interviews with the teacher.
And dress for success. We would get to do presentations in front of the class
advertising something that we had come up with and try to sale it.
Writing COE Scoring Results
14
“Memorable Class”
My best memory from this class was (flc) fall leader ship confrences.
When you get to go out of town and stay in a hotel for two nights, you get to
meet business people and get samples of what they sele. you get shirts for
memories.
Therefore Deca is my most memorable class for all these reasons. it was
the best class. it was a good experience and memory. There were many good
memorable things in that class during my sophmore year.
Writing COE Scoring Results
15
Work Sample #3: “Memorable Class” Analysis
The following Basic Level Writing Descriptors apply to this
work sample:
• Concrete thinking, limited ability to distinguish between an idea and supportive
detail…
• Little expansion of ideas and elaboration of details…repetition of ideas…
• …basic formulaic organization…basic, often ordinal, transitions between
paragraphs
• ‘Kidspeak’ (unable to—or doesn’t think to—adjust language for audience and
purpose
• Limited revision; haphazard editing…
• Fragments and run-on sentences
Writing COE Scoring Results
16
Work Sample #3: “Memorable Class” Analysis
(continued)
The writer of this Work Sample engaged in only the most basic of writing processes:
• a simple topic-based web/cluster, characterized by single-word references
(“partys,” “food,” “movies”);
• an initial draft that, aside from arrows indicating paragraph divisions, is nearly
identical to the final text;
• a final text that retains conventions errors that could easily have been
eliminated via minor editing, including the misspelling of a word taken from the prompt
(“sophmore”).
Writing COE Scoring Results
17
Work Sample #6: “Starting Time”
Extended-time / Persuasive
Prompt: Many people have expressed concern about the
starting time for high school. The school board has suggested
that school begin two hours later and end two hours later. Take
a position on this proposal, and write a multiple-paragraph letter
to the school board persuading them to agree with your position.
[Source: WASL persuasive prompt, 2002]
Writing COE Scoring Results
18
“Starting Time”
Dear school board,
My name is [ ] and Im writing you to explain why school
shouldnt start two hours late. There are many reasons why
school shouldnt start two hours late and thoose reasons are all
about responsibility and the future. you will be amazed with why
school sholdnt start late. you will agree and not think it over
twice.
My first reason for school not to start two hours late is because
students are lazzy. The schools already make everything so
easy on students and giving students more freedom than they
deserve wouldnt be right. It will probably increase the number
of car accidents and schools dont need there teenagers to be
giving them a bad name. Keeping the time how it is now will
help the students at your school suceed. because in the future
they will have real jobs where they cant wake up two hours late.
Writing COE Scoring Results
19
“Starting Time”
My last reason why I think school shouldn’t start two
hours late is because students need to be more responsible
with how they use there time. If you were to start school two
hours late then the students will party till later and be home till
later. That wouldnt be right, so it will be better if we kept our
normal schedule so that students will be tired and realise they
need to be more responsible and go to bed earlier.
Therefore these are my reasons for school not to start
two hours late. So that there could be dicipline for there future.
and they not that when they get jobs there boss isnt going to
say lets start work two hours later.
Sincerely
Writing COE Scoring Results
20
Work Sample #6: “Starting Time” Analysis
The following Basic-Level Writing Descriptors apply to this Work Sample:
• Concrete thinking, limited ability to distinguish between an idea and supporting
detail;
• Blurred understanding of the difference between writing to explain and writing to
persuade…”persuasitory”
• Some evidence of a plan…repetition of ideas
• Little expansion of ideas and elaboration of details…
• …basic formulaic organization…basic, often ordinal, transitions between
paragraphs
• ‘Kidspeak’ (unable to—or doesn’t think to—adjust language for audience &
purpose)
• Limited revision; haphazard editing
Writing COE Scoring Results
21
Work Sample #6: “Starting Time” Analysis
(continued)
The writer of this Work Sample engaged in the most basic of writing
processes:
• a simple topic-based web/cluster, limited to single clauses or
phrases (e.g., “need to stop being laazy”);
• an initial draft that, aside from removing a sentence, is nearly
identical to the final text; no evidence of reordering, rewording, or
providing additional details or elaboration;
• Although Writing Sample #6 comes closer to meeting CONV
proficiency than does Sample #3, its final text retains conventions
errors that could have easily been eliminated via minor editing,
including misspellings (“suceed,” “laazy”) usage and sentence
formation errors.
Writing COE Scoring Results
22
Sample COE Writing Collections…
 Representative Proficient COE Collection:
“O1” (Orange binder, Collection #1)
Score: 10 (above cut line of 9)
(Scorers independently scored this Collection as a 3,3 for COS and
as a 2,2 for CONV = 10 total scoring points.)
This Collection consists of six (6) Writing Samples, including responses
to local school-related issues, classroom assignments, and personalinterest topics. Rather than include actual process documents, the
student-writer provides often-detailed explanations of process
undertaken in support of purposeful revising and editing. The following
Work Sample profiles typify the Collection as a whole…
Writing COE Scoring Results
23
Work Sample #1: “My Little Brother’s Birth”
On-demand / Expository
Prompt:
Write a personal essay entitled “A Life Changing Experience.”
You must show the reader what you have learned and how it
has changed your perspective. Your only requirement is to
[practice] ‘SHOW’ writing, and to be as honest as possible
(without revealing anything that might make you uncomfortable).
Writing COE Scoring Results
24
“My Little Brother’s Birth”
Five years ago, my entire family had a huge life changing experience; my
little brother was born. It doesn’t sound like a big change but what happened
was a true miracle. This even was life changing because it affected everyone
differently; each person was changed for the better.
My mom is diabetic, so for her to even have children is really challenging.
The reason it is so hard for my mother is due to the regulating of her sugar
blood level. It becomes very difficult and at times uncontrollable while carrying a
fetus. She was willing to monitor her daily diet so carefully that her required
insulin was limited. This made it possible for the fetus to survive. I admire her
courage to restrict her diet to save a life.
The day my brother was born was a sunny spring Sunday. I woke up very
happy because I realized I had missed church, not my favorite activity. Actually it
was the last place I wanted to be. I went downstairs to enjoy some TV and my
sister came down stairs and told me that our mom was having the baby. The
rest of the family was already at the hospital. I know I should have been worried,
but I am not a real emotional guy, and I felt more excitement than fear.
Writing COE Scoring Results
25
“My Little Brother’s Birth”
My sister drove too fast, she was anxious. When we arrived at the
hospital, my little brother was already born. Unfortunately, there was a problem,
he was sick. He did not have enough sugar in his blood. This is a very
dangerous condition for a newborn and we were advised that he had a slim
chance of surviving. I felt sad but useless. Fortunately, with the miracle of
medicine he survived. I can remember the tension and how nervous we were
that first week. The days seemed like an endless desert.
Why he has been such a miracle to all of us is that before his birth we just
didn’t understand how precious life was. Like most teenagers we took things for
granted. Additionally, how important this life was to our parents as well. We
learned the true value of unconditional love. We learned how to support one
another. We learned how become a family. One life, one simple life changed us
into better people and most importantly unified us as a family.
Writing COE Scoring Results
26
Work Sample #1: “My Little Brother’s Birth” Analysis
The following Proficient-Level Writing Descriptors apply to this Work Sample:
• Connection to topic; communication of basic ideas although demonstrates some original
thinking
• Adequate control of topic; some courage or risk-taking
• Understanding of the difference between main idea and supporting details
• Adequate elaboration—often layered
• Adequate sentence-to-sentence progression within paragraphs
• A logical organization pattern
• Introductions sometimes pull the reader into the main idea or position; writes conclusions that
leave the reader with something to think about
• Adequate—sometimes sophisticated—transitions which serve to connect ideas between
paragraphs
• A sense of the person behind the words (voice); evidence of the writer’s own life in the
communication (emotional connection)
• Adequate word choice…sentence variety
• Consistent use of a dictionary and/or Spell Check, and appropriate use of thesaurus
• Attention to editing; consistent application of the rules for capitalization, usage, punctuation,
spelling; non-distracting use of conventions
Writing COE Scoring Results
27
Work Sample #1: “My Little Brother’s Birth” Analysis
(continued)
The explanation of the writing process used to produce this response
indicates the writer took advantage of COE requirements and opportunities,
as a proficient final draft indicates. Student explanation:
This was a timed write in class where I did a limited pre-write in the
form of a web, typed out a rough draft, revised on the spot, completed [the]
final.
Writing COE Scoring Results
28
Work Sample #4: “Stay, Fight or Cut & Run?”
Extended time / Persuasive
Prompt:
There are many opposing views and opinions regarding the war
in Iraq. Choose a specific side to persuade another student
answering the following question: Do we “Stay and Fight or Cut
and Run?”
Write a multi-paragraph essay which takes a specific side on the
war in Iraq. Your writing should be in the persuasive mode with
clear defense and references of your view point.
Writing COE Scoring Results
29
“Stay and Fight, or Cut and Run?”
[excerpted]
The conflict in Iraq has been one of the key reasons the United States our
government and its constituents are in debate and turmoil. One cannot turn on the news
without hearing people complain about the war in Iraq. Their complaints focus on the
increasing casualities, deaths, money being spent to finance the war. Some people are
literally sick over the war. Our congress has said if we cannot accomplish our original goal
and finish our task of a government that can survive on its own. If that cannot be done in
Iraq we will “cut and run.” This means that we will cut our loses and resources and leave
Iraq for good. Since over half the population is against the war not many are fighting it. That
being said I believe that The United States of America has never retreated for any war, why
would we start now? Should we leave and act like it is not our problems. Or stay and fight
for democracy. Clearly, we should stay and fight for the implementation of a true democracy
and the sustaining of human rights for the Iraq people. By leaving before this task is put in
place we are jeopardizing hundreds of innocent’s lives and quality of life.
There are two sides that are in the middle of this conflict in Iraq. There are pro-war,
and the anti-war. National Security adviser S. [states] “Congressional Democrats’ efforts to
put time limits on the Iraq war could doom U .S. military efforts and leave the strife-torn
country a hub of international terrorism.” S indicates: “That if we leave Iraq regardless of
what you believe in this could affect world-wide terrorism…
Writing COE Scoring Results
30
“Stay and Fight, or Cut and Run?”
[excerpted, continued]
The other side of the arguments is the anti-war side. There are people
that my have not been against the war at the beginning, but because of the
length, money, and the number of deaths people are wanting for the troops to
come home. Seventy-three percent of America want our troops to come home.
This is a hard position for the President…to convince the people of America to
want to stay involved because of the high number of lives lost. He has to choose
between the countries dissatisfaction of his stance in the war, and finishing what
he thinks is right, the mission in Iraq. This pushes the pro-war people to try and
fight to stay over there. The cause of leaving may put a profound effect on the
people of Iraq and worldwide terror. Do the ends really justify the means?…
The one main reason we went over into Iraq is that we knew that they had
[ties] with al quida, and the fact that they were known and believed to have
weapons of mass destruction. Even the ehad of the CIA M.…said that his Intel
said that there were weapons of mass destruction. We went over there to stop
Writing COE Scoring Results
31
“Stay and Fight, or Cut and Run?”
[excerpted, continued]
the spread of terror. We did not find any WMD there is a lot of speculation of
what happened to them but the Intel said otherwise. This is the reason that we
should stay over there and finish what we have started. As a woman said in an
interview with CNN, “We need to fight for people that could not fight for
themselves.
My position is clear—we absolutely need to stay and fight for the rights of
Iraqis’. It is our duty as Americans to uphold civil rights. We are a country that
believes in the democratic process and we cannot leave until this is in place and
fully functioning. If we leave now we would be leaving this country in the same
circumstances and situation we initially found it in. This would be a great
injustice and a waste of time, money and lives. Keeping our commitment
guarantees the protection of Iraqi’s future generations. We might not be able to
stop terrorism; but we can give the Iraqis’ a foundation for peace and equal
rights for all its citizens.
Writing COE Scoring Results
32
Work Sample #4: “Stay and Fight, or Cut and Run?”
Analysis
The following Proficient-Level Writing Descriptors apply to this Work Sample:
• Connection to topic; communication of basic ideas although demonstrates some
original thinking
• Adequate control of topic; some courage or risk-taking
• Understanding of the difference between main idea and supporting details
• Adequate elaboration—often layered
• Adequate sentence-to-sentence progression within paragraphs
• A logical organization pattern
• Introductions sometimes pull the reader into the main idea or position; writes
conclusions that leave the reader with something to think about
• Adequate—sometimes sophisticated—transitions which serve to connect ideas
between paragraphs
• A sense of the person behind the words (voice); evidence of the writer’s own life in
the communication (emotional connection)
• Adequate word choice…sentence variety
• Consistent use of a dictionary and/or Spell Check, and appropriate use of thesaurus
Writing COE Scoring Results
33
Work Sample #4: “Stay and Fight, or Cut and Run?”
Analysis (continued)
This writer’s explanation of the writing process used to produce this
response takes advantage of COE requirements and opportunities, as
evidenced by a clearly proficient (especially for COS) final draft. The
student’s explanation emphasizes the power of authentic classroom
learning and work in the service of COE Writing extended time
response performance:
“This was an extended paper done in several phases with some teacher
assistance. We have been working on the Persuasive Essay this year and this
was our culminating essay. We debated sides in class, took notes, went on the
internet for research, watched the debates on CNN and then had to choose a
side. This paper went through three drafts all typed because my handwriting is
so poor. I did my best to show both sides but I am very clear that we should stay
the course in Iraq. This was not a popular stance in my class. I felt this is one of
my better essays.”
Writing COE Scoring Results
34
Writing Assessment Instructional Support Modules
www.k12.wa.us/assessment/WASL/writing/writing
• High School Persuasive Writing Module (Grades 8-10) Version 2
(June 2007)
• High School Four-Week Course
• Middle and High School Introductions and Conclusions
• Middle and High School Elaboration
[ plus ELL Appendices and Resources]
Writing COE Scoring Results
35
Certificate of Academic Achievement Options
2007-2008 Key Dates:
The Collection of Evidence (COE)
• 12/15: COE registration for February submission
closes
• 2/15: COE submission date to OSPI
• 4/15: COE registration for June closes
• 6/16: COE submission date to OSPI
Writing COE Scoring Results
36