Notes for Monday, Jan. 24, 2011 - Testing Course
O’Malley, J.M., & Pierce, L.V. (2002) Authentic assessment for English language learners: Practical
approaches for teachers (2nd ed.). London: Longman. (Call number PE1128 .O47 1996)
Chapter 1: Authentic Assessment

alternative assessment
o any method enabling teachers to find out what a student knows (or can do); hence, a valuable
alternative to traditional testing
o necessarily criterion-referenced (based on standardized tests; shows a high degree of
correlation of established tests)
o typically authentic, based on real-life settings
The term “authentic assessment” is meant to capture the multitude of forms of assessment consistent with
classroom goals, curricula, and instruction.
Interest in authentic assessment is mostly due to overt drawbacks of traditional assessment procedures – e.g.,
 Failure to assess student outcomes
o In classrooms, students
 collaborate with peers (acceptable in classroom discussions but totally taboo in
testing)
 integrate resource info with personal opinions
 use knowledge derived in content area X to solve problems in content areas Y or Z
(traditional testing rarely asks students to use cross-curricular thinking)
o Traditional testing captures NONE of these things! The multiple-choice test (M-C), e.g.
 doesn't’ show where or why the student chose incorrectly
 students have a 25% chance to guess the right answer without knowing anything
 but M-C testing does have a high degrees of validity and reliability and is easy to
grade (not subjective on the part of the scorer)
 Inadequacy to assess the knowledge and skills needed for being successful in the society of
tomorrow. (NB: Ability to pick one of several ready-made options is NOT what real life is about!)
 Assessment of English language learners (ELLS) is difficult, mainly due to
o diversity of levels of conversancy (proficiency) with English to be concurrently determined
o varied and complex nature of purposes underlying assessment
 Primary purposes of ELL assessment
1. identification of students eligible for special education
2. placement of students into appropriate programs
3. reclassification or exit -- determining whether or not an ELL has reached grade-level
classroom level
4. monitoring student progress
5. program evaluation
6. accountability -- i.e., guaranteeing attainment of educational goals or standards


Standardized, norm-referenced tests (NRTs) -- often not appropriate for ELL
o main shortcoming is their tendency to go for M-C format
 assumption of a high level of proficiency in English
 assumption of knowledge about English-speaking culture
 measurement not of what one knows but what one does NOT know (no middle ground
between the correct answer and the incorrect ones; answers are either right or wrong)
However, NRTs (hence, M-C tests) do have some beneficial aspects in overall testing -- specifically,
when it comes to
o comparing performance of an individual or a group to that of an external, normative group
o identifying strengths/weaknesses in skill areas
o evaulating programs of instruction
Testing notes - Jan. 31, 2011

Authentic assessment
o all forms of assessment reflective of student learning, achievement, motivation, and attitudes.
All of these are displayed in/through instructionally-relevant classroom activities.
o examples of authentic assessment:
A. performance assessment
B. portfolio assessment
C. student self-assessment
A. Performance assessment:


any form of assessment requiring a student to construct a response, whether oral or written
student responses: either elicited or observed in naturally occurring production of language; possibly, yet
not necessarily, involving
o prior knowledge (often the source of our responses; what we already know; info from context
clues)
o recent learning (something learned earlier in that class period)
o skill to solve realistic/authentic problems
 examples of performance assessment
o oral reports
o writings
o individual/group projects
 specific (major) characteristics of performance assessment
o constructed nature of response (responses must be student-generated, student-created – not
just a reading of a definition in a textbook)
o higher-order thinking skills
o authenticity of task
o integrative nature of response (has to be a part of something)
o access, not only to student’s response, but also to process(es) whereby response is arrive at
o increased depth of production of response, but mostly depth of assessment (better way for the
teacher to judge a student’s response)

other characteristics of performance assessment
o requiring judgment on part of teacher – hence need for a scoring scale (rubric) assigning
numerical values to performance levels [Note: Scoring criteria must be made known to
students in advance.]
o performance level is precisely defined in terms of what the student needs to actually do to
demonstrate skill or proficiency (what the student is able to do to earn a grade level)
B. Portfolio assessment:


systematic collection of samples of student work potentially indicative of progress (or lack thereof!)
over time – e.g.,
o writings (preferably produced in class to show that the work really was created by the student
– not a parent, sibling, etc.!)
o reading logs
o teacher’s comments on student work (shows student-teacher interaction)
important feature – student’s involvement in selecting samples of his/her own work to show growth
C. Student self-assessment:


gives student’s control over his/her learning (makes them feel more important, more invested in their
education, more responsible for their own progress academically)
main features – students…
o exchange ideas (mainly during class interactions)
o offer support to peers (mainly during class interactions; e.g., pairs trade papers to share
writing/editing, question/answer to practice speaking/listening skills)
o request teacher’s assistance when in need
o have a say, not only in terms of what they learn, but also in terms of how they will be assessed
– hence the possibility that a student’s content knowledge (not knowledge of English, per se,
but academic course knowledge) be assessed in
 English (for English courses and content-area knowledge)
 his/her L1 (not for English/ESL knowledge)
Chapter 2: Designing Authentic Assessment



constructivism – recent view on teaching and learning – individuals learn (or at least are assumed to
learn) by constructing information through use of active mental processes (based on Piaget)
implications for assessment – students have the opportunity to
o construct responses (not just recitation of memorized responses; build responses in their own
words)
o apply their learning to problems that
 mirror their classroom activities
 imitate real-life situations
types of ESL-related authentic assessment
o oral interviews –
 especially beneficial when
 used with young children (and/or those with very beginning- or low-level
skills)
 conducted regularly (so students will be accustomed to the format)
 students respond orally to questions (are often preceded by warm-up questions)
 teacher’s probe questions to determine student’s exact “ceiling,” i.e. –
 comprehension of language
 command over (specific aspects of) language
o story telling – students produce oral language
 can be rated on how students present/organize/connect events, how they follow the
structure of a story, etc.
 can involve imagination but is not necessary
o writing samples – students generate writing samples to meet a number of purposes
(narratives, argumentative essays, reports, etc.) [Note: Pre-established scoring rubrics are
greatly helpful in assessment of writing and oral communication.]
o projects and exhibitions – students carry out, and display, projects; can involve oral
presentations as well
o experiments or demonstrations – students conduct an experiment in science using actual
materials
o constructed-response items – students read or review texts, then respond to open-ended
questions eliciting comprehension and higher-order thinking
 highly authentic because they draw on the kinds of thinking and reasoning skills
actually used in classrooms
 highly practically-oriented because they involve application of classroom learning to
real-life settings (hypothetical situations, scenarios)
o teacher observations of
 attention paid in class
 concentration
 participation (and quality thereof)
 cooperation
 quality of assignments
o portfolios – collections of student work meant to show progress over time
Designing authentic assessment


group work recommended (colleagues, administration) – with involvement of parents, where possible
steps involved in creating an authentic assessment team
1. build a team (choose members)
2. determine the purpose of assessment
3. specify objectives
4. conduct professional development on authentic assessment (study on your own), and share
information with team
5. find examples of authentic assessments (journals, books, conferences, etc.)
6. adopt/adapt existing assessment, or develop a new one
7. try out the assessment
8. review the assessment
Technical quality of authentic assessments

reliability – consistency in producing same score
o on different testing occasions – intra-rater reliability (judged by the same rater)
o with different raters – inter-rater reliability (judged by several raters scoring independently of
each other)
o means of boosting reliability – use a scoring rubric! (helps eliminate subjectivity)
 holistic – each of the sequential levels of the rubric requires a more demanding
performance (level two is more challenging than level one, e.g.)
 analytical – separate rating given to each component in the scale (content,
organization, vocabulary, etc.)
tonight’s stopping point
Docimology – study of how to write tests? How to assess? Not sure but Kovalik mentions this a lot.
Doc`i*mol"o*gy\, n. [Gr. ? a test + -logy.] A treatise on the art of testing, as in assaying metals, etc.
Webster's Revised Unabridged Dictionary, © 1996, 1998 MICRA, Inc.
Docimology. (n.d.). Webster's Revised Unabridged Dictionary. Retrieved February 07, 2011, from
Dictionary.com website:http://dictionary.reference.com/browse/Docimology
Read chapters 1 and 2 in Hughes for next Monday.