AssessmentCritique

advertisement
Overview
Description of the Course
The larger curricular aims of Latin I are to develop early Latin skills as well as an awareness of
the Latin remnants that pervade the everyday life of Americans. Latin I also prepares the
foundation for students to understand the cultural perspectives of the Romans by examining
the influences, practices, and products of the Romans. Latin I is the base upon which students
begin to realize that Latin is not dead, and that it is relevant to their lives. Latin I also serves as
the foundation of the Latin grammar. The grammar base will grow throughout the course and
subsequent courses, allowing the students to reach the overarching goal of Latin-- to be able to
translate authentic Latin pieces while understanding not only the grammar but the context and
culture embedded within the pieces.
Description of the specific unit
This test is part of a unit on noun-adjective agreement. The students will learn the rules of using
adjectives to describe nouns. This process is much different from how we use nouns and
adjectives in English. Students often have a hard time with this concept because nouns and
adjectives that agree don’t always look the same. Also, in Latin, the noun and adjective do not
have to be beside each other for the adjective to modify the noun. This also confuses students
who are accustomed to reading and speaking English, where nouns and adjectives are next to
each other when they agree. There are three components when creating or reading a nounadjective pair in Latin-- if one component is missing, then it is not correct. Because of these
complex issues and differences from the native language, students find it difficult to create,
recognize and translate noun-adjective pairs.
The culture aspect of this unit is on legendary heroes of Rome. The students will read about
four different heroes and their actions that made them infamous. Also, the students learn
about the Roman value of peitas, an important value to the Roman people that often appears
in authentic Roman texts.
Description of Intended Learning Outcomes
All intended learning outcomes (ILOs) for this unit will be assessed on the test in the same
cognitive levels that they are being taught. There will also be a few scaffolding questions in
lower cognitive levels to get the students started. The different portions of the test will cover
different material and cognitive levels in ways that the students are familiar with and that
match instruction. The main knowledge item, vocabulary, is assessed through matching, while
the application objectives will be assessed through multiple choice. The higher level thinking
skill of evaluation will be assessed in a short essay form. Matching will also be used to assess
the application of reading noun-adjective pairs.
Since all of the ILOs are assessed at their cognitive level, the only other item in the unit
assessment plan is a vocabulary quiz. This will ensure that the students have mastered the
vocabulary necessary to complete the translation and multiple-choice sections of the test.
Without this vocabulary knowledge, the students will not do well on the test.
The ILOs in this unit are mainly at the application level. The multiple choice questions that we
have created allow the students to show that they not only can recall and understand nounadjective pairs, but that they know how to translate and create the pairs as well. This is the
most important ILO for this unit and will show a clear picture of student learning.
Cognitive
Level
Content
Agreement
clues
5%
Knowledge Comprehension
Analysis Synthesis Evaluation
X
Short
Answer
Noun
adjective
agreement
45%
Vocabulary
10%
Application
X
Multiple Choice
X
Matching
Translation
25%
X
Alternate
Choice,
Multiple
Choice
X
Translation
X
Multiple Choice
Heroes of
Early Rome
15%
X
Translation
X
Multiple
Choice
X
Essay
Table of Specification for Test
Cognitive
Level
Content
Knowledge
Agreement
clues
5%
X
MC Test
Comprehension
Application
Analysis Synthesis Evaluation
Noun
adjective
agreement
45%
Vocabulary
10%
Translation
25%
Heroes of
Early Rome
15%
X
MC Test
X
Supply
Response
Test
Vocab Quiz
X
AC/MC Test
X
Translation
(Test)
X
MC Test
X
Translation
(Test)
X
MC Test
X
Essay
(Test)
Unit Assessment Plan
Overview Description of Characteristics of the Classroom
Constructing a test for a unit that is generally the hardest or least understood unit of the year
was difficult. We took in to consideration that this concept was foreign to the students, and
while they had been using the Latin language all year, this was the first time they had seen
something truly different from the English language. Therefore, we created a test that would
utilize the application of the students’ learning. We aim for our students to be able to translate
authentic Latin, and authentic Latin poets will not always use noun-adjective pairs that match
or that are side by side. Thus, we wanted our students to master the application of both
creating and translating noun-adjective pairs. Between the two of us, we only have three
students with IEPs. While this is a very small number, it still affected how we created the test
and in turn how we will teach our students. With such a difficult concept for students, it is good
to do a lot of practice in a variety of different learning styles and multiple intelligences while
still letting them know the format that will be on the test. Since our cooperating teachers do
not necessarily use the same type of test, we will definitely model and practice in the same way
we will be testing the students. In this, and other ways, we took all of our students into
consideration.
Explicit Purpose of the Test and Results
The explicit purpose of this test is to assess whether the students have mastered nounadjective agreement, the legendary heroes of Rome, and the translation from this chapter. The
grammar aspect of this chapter will be very important as they move forward, thus, we will not
be able to move on until most students display mastery. By using a test that allows the students
to show their learning in many ways, we will be able to judge if the class can move on, if certain
students need help, or if we need to reinforce the concept.
Design Elements of the Test
Intended Learning Outcome
 Students will identify the agreement clues (gender, case, and number) of adjectives and
nouns.
 Students will match noun-adjective pairs in Latin.
 Students will recognize and translate correctly noun-adjective pairs while reading Latin
passages.
 Students will recognize and accurately translate Chapter 18 vocabulary words.
 Students will read and understand the chapter 18 story, “Arrival at the Inn.”
 Students will compare and contrast the legendary heroes of early Rome.
 Students will defend a position on which early Roman hero was the bravest.
Table of Specifications
Cognitive
Level
Content
Agreement
clues
5%
Knowledge Comprehension
Translation
25%
Heroes of Early
Rome
15%
Analysis Synthesis Evaluation
X
Noun adjective
agreement
45%
Vocabulary
10%
Application
X
X
X
X
X
X
X
X
Construct Validity
The construct validity of this assessment is strong. In order to match the noun-adjective pairs,
students must identify the agreement clues and apply them to each noun and adjective that
appears. The vocabulary that appears on this test is largely from words that appear in the
chapter 18 story and fewer words that appear in previous chapters. Words in future chapters
are not included.
Content Validity
The bulk of the test focuses on students’ ability to identify clues and match nouns and
adjectives (Multiple choice questions 1-8, 3rd declension adjective chart, Alternate choice
section for a total of 45 points on the 95-point assessment). There is a substantially lighter
vocabulary section (11%), and a moderately heavy translation portion (29%). Students should
be able to, but are not required to (or assessed on) translating the alternate choice paragraph.
The legendary heroes of early Rome are assessed in 2 multiple-choice questions and the essay
(13%). These percentages do not exactly match up with our table of specifications, but they are
fairly close which reflects a fairly strong level of content validity.
Rationale for Test Item Types
The first section of the test consists of multiple choice questions about nouns and adjectives.
We decided to use multiple choice questions to assess this content because we were able to
assess knowledge, comprehension, and application of this content relatively quickly. It was
important for us to be able to assess this content quickly because we wanted to include the
essay about Roman heroes on this assessment rather than as a separate assignment, so a quick
and accurate measure for assessing acquisition of other intended learning outcomes was
desirable.
We also included a supply-response paradigm chart because we believe it is important for
students to carry these constructions in their head for reference. So that when they encounter
a word that they do not recognize, they will at least know the form and function of the word.
This chart is also an efficient measure of student knowledge of the case, number, and gender of
adjectives.
We chose a supply-response list of vocabulary for similar reasons. Because so many English
words are derived from Latin, it is often very easy for students to choose the correct definition
of a particular word from a list without having learned the vocabulary. When, as is our ultimate
goal, students are faced with authentic Latin texts, they will not have a list of vocabulary to
choose from. Additionally, the words they see will not be devoid of context, so we chose to
provide the vocabulary words in context.
The Noun-adjective pair alternate choice section was selected for reasons similar to the
multiple choice section, but the focus is heavily on application of the skill. The attention in this
section is to students’ ability to pick out matching grammatical features (noun-adjective
agreement clues) in an unfamiliar text.
The chapter 18 reading comprehension section is multiple choice because we needed an
efficient way to measure student comprehension and recall of the story that they read in class.
The translation portion assesses students’ understanding of the chapter 18 story and their
ability to refine translations of texts that they have seen before.
The essay assesses students’ ability to evaluate the legendary Roman heroes as well as their
knowledge of the heroes and ability to apply the concept of pietas to those heroes.
The only non-original portions of this test are the texts for the alternate choice section and the
translation section. The alternate choice section text is from p.25 (I-A Final Test) of the Ecce
Romani I Test Masters book, 2nd edition. The translation section is from chapter 18 of Ecce
Romani.
Potential Threats to Reliability
The alternate choice section of the test contains two pairs in which the second word is neither
an adjective nor a noun, but has a superficially matching ending. This may cause students who
know the grammatical content, but not the vocabulary, or who are inattentive to miss the
questions although they know the content.
Additionally, the essay section is a subjective assessment and our rubric controls for a lot of
subjectivity but it is difficult to avoid. The reliability is additionally threatened because there are
two graders for separate sets of students, so we may vary in our judgments of what constitutes
an adequate response.
The multiple choice section has some minor threats to reliability as well, such as in question 2
where students must choose the correct noun-adjective pair for “angry Cornelia.” In this
question, the answers are similar and could potentially confuse students (especially those with
visual processing disabilities). Additionally, if the student did not know the content, they could
select instead from one of the two matching ending pairs and increase their chance of providing
a false positive.
The threats to reliability for the essay would include a teacher’s indication of bias toward any
particular legendary hero and thus cause students to feel that they must choose that hero in
order to get a good grade. In addition to measuring the ability of students to evaluate the
legendary Roman heroes, this question also measures students’ ability to apply the concept of
pietas. This may cause students to shift focus to the concept of pietas rather than adequately
defend their selected hero.
Potential for and Cautions about Predictive Validity
The alternate choice noun-adjective pair recognition section serves as a good indicator of
students’ potential for successful experiences with sight-reading passages (fairly rare in Latin).
Overall the test is a fairly accurate measure of student acquisition of the intended learning
outcomes, based on its alignment with the table of specifications and instruction. However, if
students introduce random error (recent break-up, recent arguments, stress, etc.) the test may
become a less accurate portrayal of their acquisition of the intended learning outcomes.
Scoring and Grading Procedures
Multiple choice questions 1-7 are each worth 2 points, questions 8 and 9 are worth 1 point
each. Each response in the 3rd declension chart is worth 1 point. In the vocabulary section,
students will receive ½ point per principle part and 1 point for the definition (e.g. the response
for legatum requires 3 parts aside from the definition and is worth 2 ½ points). Each alternate
response choice is worth 2 points. The Reading comprehension questions are each worth 1
point. The translation section is worth ½ point per word. This is broken down into ¼ point for
the correct vocabulary word translation and ¼ point for the correct grammatical translation
(e.g. translating “canes” as dog instead of dogs will earn the student ¼ point rather than the full
½ point). This significantly reduces the subjectivity in grading translations, since it is broken
down word-by-word rather than by sentence or even overall sense of the passage. The essay
uses the rubric in appendix A. The items on the test were weighted to reflect the table of
specifications and the focus that we intend to use for instruction of this chapter. Since this is a
chapter quiz, it will be weighted along with all other quizzes at 30% of total assessments in the
class. The material will appear again in a portion of the unit test which will be weighted with
other tests at 50% of total assessments.
Appendix A
10
8
6
≤4
Roman
Hero
The student
clearly articulates
which Roman
hero is the
bravest, and
provides at least
3 supported
reasons for their
choice.
The student
mentions which
Roman hero is
the bravest, and
provides at least
2 plausible
reasons for their
choice.
The student
mentions which
Roman hero is
the bravest,
and provides
one plausible
reason for their
choice.
The student does
not choose a Roman
hero or provides no
reasonable
explanation.
Pietas
The student
clearly articulates
the 3 elements of
pietas and which
elements their
chosen hero
displays.
The student
clearly articulates
the 3 elements of
pietas, but does
not articulate
which elements
their chosen hero
displays.
The student
only articulates
the elements of
pietas that their
chosen hero
displays.
The student does
not articulate all
elements of pietas
or does not describe
which elements of
pietas their chosen
hero displays.
Comparison The student
compares their
chosen hero to at
least two other
heroes, including
supports for what
set their hero
apart from the
others.
The student
compares their
chosen hero to at
least two other
heroes, but does
not describe what
plausibly sets
their hero apart
from the others.
The student
compares their
chosen hero to
one hero and
plausibly
describes what
sets them
apart.
The student
compares their
chosen hero to one
hero but does not
plausibly describe
what sets them apart
or does not compare
their chosen hero to
another hero.
Download