COURSE ASSESSMENT: LG 111
LA Academy Cohort
Fall 2009
Date: 1-15-10
Department: Foreign Languages and Literatures
Course:
LG 111
Curriculum or Curricula: LA
PART I. STUDENT LEARNING OBJECTIVES
For Part I, attach the summary report (Tables 1-4) from the QCC Course Objectives Form.
TABLE 1. EDUCATIONAL CONTEXT
LG-111 is the first part of the introductory sequence of foreign language study (a requirement for a successful
transfer to the junior year of a baccalaureate program). It is a foundation course required for (AA) degree in
Liberal Arts and Sciences, Fine Arts, and Business Transfer.
TABLE 2. CURRICULAR OBJECTIVES
Note: Include in this table curriculum-specific objectives that meet Educational Goals 1 and 2:
Curricular objectives addressed by this course:
N/A
TABLE 3. GENERAL EDUCATION OBJECTIVES
Gen Ed
objective’s ID
number from
list (1-10)
General educational objectives addressed by this course: Select from preceding list.
(1) Communicate effectively through reading, writing, listening and speaking.
(2) Use analytical reasoning to identify issues or problems and evaluate evidence in order to
make informed decisions.
TABLE 4: COURSE OBJECTIVES AND STUDENT LEARNING OUTCOMES
Course objectives
Learning outcomes
1. Provide basic information in
Students will be able to talk about themselves by providing their name,
German about yourself, your city,
address, phone number, school schedule. They will also be able to describe
your classmates, your family,
their family or hobbies by answering questions in prompted dialogues.
friends, hobbies, and daily activities.
2. Demonstrate ability to accomplish
simple communicative tasks on
every day topics such as greeting
people or introduce yourself to
others, or describing your life in
school or at work, or during your
spare time.
Students will be able to greet people and/or introduce themselves, or
exchange personal information by participating in communicative tasks
where they will play the role of a new student at QCC or a party guest
trying to make friends.
2
PART II. ASSIGNMENT DESIGN: ALIGNING OUTCOMES, ACTIVITIES, AND
ASSESSMENT TOOLS
For the assessment project, you will be designing one course assignment, which will address at least one general
educational objective, one curricular objective (if applicable), and one or more of the course objectives. Please
identify these in the following table:
TABLE 5: OBJECTIVES ADDRESSED IN ASSESSMENT ASSIGNMENT
Course Objective(s) selected for assessment: (select from Table 4)
1. Provide basic information in German about yourself, your city, your classmates, your family, friends, hobbies,
and daily activities.
2. Demonstrate ability to accomplish simple communicative tasks on every day topics such as greeting people or
introducing yourself to others, or describing your life in school or at work and in your spare time.
Curricular Objective(s) selected for assessment: (select from Table 2)
N/A
General Education Objective(s) addressed in this assessment: (select from Table 3)
1. Communicate effectively through reading, writing, listening and speaking.
2. Use analytical reasoning to identify issues or problems and evaluate evidence in order to make informed
decisions.
In the first row of Table 6 that follows, describe the assignment that has been selected/designed for this project.
In writing the description, keep in mind the course objective(s), curricular objective(s) and the general education
objective(s) identified above,
The assignment should be conceived as an instructional unit to be completed in one class session (such as a lab) or
over several class sessions. Since any one assignment is actually a complex activity, it is likely to require that
students demonstrate several types of knowledge and/or thinking processes.
Also in Table 6, please
a) identify the three to four most important student learning outcomes (1-4) you expect from this assignment
b) describe the types of activities (a – d) students will be involved with for the assignment, and
c) list the type(s) of assessment tool(s) (A-D) you plan to use to evaluate each of the student outcomes.
(Classroom assessment tools may include paper and pencil tests, performance assessments, oral questions,
portfolios, and other options.)
Note: Copies of the actual assignments (written as they will be presented to the students) should be gathered
in an Assessment Portfolio for this course.
3
TABLE 6: ASSIGNMENT, OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS
Briefly describe the assignment that will be assessed:
This project will assess students’ speaking abilities at a novice-mid proficiency level as described in the ACTFL
2006 guidelines. Their conversational abilities will be assessed by engaging them in role-play activities where
both they and the instructor play a specific role. Students may play the role of a new student at QCC, or a party
guests trying to make new friends, or may share some personal information with a close friend (played by the
instructor) about a new boy/girlfriend. Oral tasks will prompt the students in providing personal information, i.e.
their name, address, phone number, or class schedule, or describing their daily activities and hobbies.
Desired student learning outcomes
for the assignment
(Students will…)
List in parentheses the Curricular
Objective(s) and/or General
Education Objective(s) (1-10)
associated with these desired learning
outcomes for the assignment.
Briefly describe the range of
activities student will engage in
for this assignment.
Role-play pretending to talk to a
stranger at a café in Berlin, or to a
brother of a German friend who is
visiting New York City.
What assessment tools will be
used to measure how well
students have met each learning
outcome? (Note: a single
assessment tool may be used to
measure multiple learning
outcomes; some learning
outcomes may be measured using
multiple assessment tools.)
Gen-Ed objective
(1) Communicate effectively through
reading, writing, listening and
speaking.
(2) Use analytical reasoning to
identify issues or problems and
evaluate evidence in order to make
informed decisions.
Curricular objectives
Students will be asked to complete a
task where they need to exchange
personal information with a new
acquaintance met at a café in Berlin,
or with a brother of a German friend
who is visiting New York City and
has poor English skills, or talk to a
friend on the telephone to make
arrangements to meet see a room for
the summer in Berlin (see Appendix
I).
N/A
4
PART III. ASSESSMENT STANDARDS (RUBRICS)
Before the assignment is given, prepare a description of the standards by which students’ performance will be
measured. This could be a checklist, a descriptive holistic scale, or another form. The rubric (or a version of it) may
be given to the students with the assignment so they will know what the instructor’s expectations are for this
assignment.
Please note that while individual student performance is being measured, the assessment project is collecting
performance data ONLY for the student groups as a whole.
TABLE 7: ASSESSMENT STANDARDS (RUBRICS)
Brief description of assignment: (Copy from Table 6 above)
This project will assess students’ speaking abilities at a novice-mid proficiency level as described in ACTFL 2006
guidelines. Their conversational abilities will be assessed by engaging them in role-play activities where both
they and the instructor play a specific role. Students may play the role of a new student at QCC, or a party guest
trying to make new friends, or may share some personal information with a close friend (played by the instructor)
about a new boy/girlfriend. Oral tasks will prompt the students in providing personal information, i.e. their name,
address, phone number, or class schedule, or describing their daily activities and hobbies.
Desired student learning
outcomes from the
assignment: (Copy from
Column 1, Table 6 above;
include Curricular and /or
General Education Objectives
addressed)
Gen-Ed objectives
(1) Communicate effectively
through reading, writing,
listening and speaking.
2) Use analytical reasoning
to identify issues or problems
and evaluate evidence in order
to make informed decisions.
Assessment measures for
each learning outcome:
(Copy from Column 3,Table 6
above)
Students will be asked to complete
a task where they need to
exchange personal information
with a new acquaintance met at a
café in Berlin, or with a brother of
a German friend who is visiting
New York City and has poor
English skills, or talk to a friend
on the telephone to make
arrangements to meet see a room
for the summer in Berlin.
Standards for student performance:
The parameters for measuring students’ speaking
abilities will be to determine whether:
(i) they understand the questions being asked by the
interlocutor,
(ii) they use German syntactic structures and
vocabulary accurately,
(iii) they make themselves understood by using the
correct intonation and pronunciation.
(iv) they speak with some degree of fluidity.
75% of the students tested are anticipated to meet
the course’s expectations as described in the
attached rubric (see Appendix II).
Curricular objectives
N/A
5
PART IV. ASSESSMENT RESULTS
TABLE 8: SUMMARY OF ASSESSMENT RESULTS
Use the following table to report the student results on the assessment. If you prefer, you may report outcomes
using the rubric(s), or other graphical representation. Include a comparison of the outcomes you expected (from
Table 7, Column 3) with the actual results. NOTE: A number of the pilot assessments did not include expected
success rates so there is no comparison of expected and actual outcomes in some of the examples below. However,
projecting outcomes is an important part of the assessment process; comparison between expected and actual
outcomes helps set benchmarks for student performance.
TABLE 8: SUMMARY OF ASSESSMENT RESULTS
Desired student learning outcomes:
(Copy from, Column 1,Table 6 above;
include Curricular and/or General
Education Objectives addressed)
Student achievement: Describe the group achievement of each
desired outcome and the knowledge and cognitive processes
demonstrated.
Gen-Ed objectives
See Table 9
Communicate effectively through
reading, writing, listening and
speaking.
Curricular objectives
N/A
6
TABLE 9. EVALUATION AND RESULTING ACTION PLAN
In the table below, or in a separate attachment, interpret and evaluate the assessment results, and describe the
actions to be taken as a result of the assessment. In the evaluation of achievement, take into account student
success in demonstrating the types of knowledge and the cognitive processes identified in the Course
Objectives.
A. Analysis and interpretation of assessment results:
What does this show about what and how the students learned?
33 students completed the speaking task in German, and their performance was rated
according to five parameters: listening comprehension, fluidity, pronunciation,
vocabulary and grammar. Student performance was scored using the above-mentioned
scoring rubric.
The average score achieved for pronunciation was 3.00. On this parameter, students on
average met expectations (see Chart 1 below):
Chart 1
The average score achieved for fluidity was 2.62 and the average score for vocabulary
was 2.8. In both these parameters students on average almost met expectations (the
average was close to the 3.0 mark). The average score achieved for grammar was 2.4,
which is below expectations, but still within the area of almost reached expectations.
The total points a student could achieve (adding up all five parameters, each worth a
maximum of four points) was 20. The scale agreed upon beforehand by the Assessment
Committee defined the following ranges: 0-5 points= student performance does not meet
expectations; 6 – 10 points= student performance almost meets expectations; 11-15
points= student performance meets expectations; 16-20= student performance exceeds
expectations.
7
The average of total points achieved by students tested in German was 13.55. According
to the scale, this average falls in the 11-15 range, signifying that student overall
performance on the task meets expectations.
The Assessment Committee also predicted that 75% of students tested would achieve a
score indicating that their performance meets expectations. The results for percentage of
students who achieved each performance level in German are the following: 6% of
students scored at Level 1, 0-5 points, and their performance does not meet expectations;
12% scored at Level 2, 6-10 points, and their performance almost meets expectations;
43% scored at Level 3, 11-15 points, and their performance meets but does not exceed
expectations; finally, 39% scored at Level 4, and the quality of their performance exceeds
expectations. Adding together Levels 3 and 4 gives us a total of 82% of students who
meet or exceed expectations, as defined by the scoring rubric. (see Chart 2 below):
Chart 2
B. Evaluation of the assessment process:
What do the results suggest about how well the assignment and the assessment process worked
both to help students learn and to show what they have learned?
B1) Evaluation of students’ results
The results of the speaking task in German show that, as predicted, at least 75% of
students tested meet or exceed expectations on overall performance of the task: the actual
percentage of students tested in Fall 2009 who meet or exceed expectations is 82%. The
largest portion of students (43%) fell into the category rated as “meets expectations,” that
is, achieving 11-15 points out of 20. Adding Levels 1 and 2 together, the percentage of
students whose performance does not meet expectations is 18%.
8
However, if we look at students’ performance according to individual parameters, the
highest average score was in pronunciation (3.0%). The second and third highest scoring
parameters were “fluidity” (2.84%), and “vocabulary” (2.81%). Such high rates may be
due to the fact that these skills require the least analytic skill on the part of the student.
Pronunciation, fluidity and vocabulary are easily leaned and mastered at the elementary
because they are based on memorization and repetition. Students are able to replicate
learned phrases and expressions, while they are not able to create with the language at
this level. In addition to that students have practiced intensely during the entire
assessment process. This positively influenced their control over pronunciation, fluidity
and vocabulary.
The two lowest scoring parameters were ‘listening comprehension’ (2.72%) and
‘grammar’ (2.40%). These parameters differ from each other by .32 points: the listening
comprehension average is only .9 point power than vocabulary and .28 points below the
expected result of 3 points. Listening comprehension is a ‘passive’ skill, but any
deviation in the pronunciation of the speaker or the ignorance of one of the words uttered
in a question can confuse the listeners and consequently interfere with the
comprehension. On the other hand the application grammar rules require analytic skills.
In the context of a speaking task, which also requires listening skills, a weaker
performance on these two particular parameters would be expected.
B2) Evaluation of the assessment tools
The results obtained seem to indicate that the assessment tools used and the assessment
process undertaken have been successful in accurately determining our students’ oral
proficiency level. Not only has it shown that more than ¾ of the students had met the
expectation, but it has also provided them with the appropriate tool that would facilitate
their learning process. The great majority of students, in fact, took the speaking task
assessment very seriously, and studied extensively for it. Since this assessment tool
motivated students to study more, it is an efficient way of achieving the course objectives
at least in the oral skills which were assessed in this case.
The high percentile (82%) of the students assessed who exceeded the expectations or met
them, is reassuring for the instructors. However, the amount of students who exceeded
the expectations in the oral assessment (39%) does not seem realistic, nor does it
correspond to the results achieved in the global evaluation of all skills, reflected in the
results of the final exams. The percentile of students who meet expectations is normally
the highest percentile and could have been much higher than 43% as seen in the students
of German assessment in fall 2009. Therefore, the rubric used in the fall assessment
should be revised to capture in more detail the variations in the proficiency of those
students that perform in the expected level and those who exceed expectations.
9
C. Resulting action plan:
Based on A and B, what changes, if any, do you anticipate making?
Further actions
The overall average achieved by students on the speaking task in German shows a
satisfactory result, with 82% of students meeting expectations, which exceeds by 7% the
75% expected by the Assessment Committee. The averages achieved on each individual
parameter in German show areas of relative strength, as well as two areas of relative
weakness: listening comprehension and grammar. The listening comprehension can be
improved by introducing more oral activities in the classroom. But in order to master
grammar analytical skills required and these skills will improve as the students progress
in the study of the language. After discussion of the results across all languages taught in
the department, it has been decided by the Assessment Committee to postpone any
changes in curriculum emphases or instructional methodology until we have gathered
more complete data, including student performance on written tasks, as well as a second
assessment using the speaking task. After compiling and analyzing data for all
assessment completed in academic year 2009-2010, the Assessment Committee, in
consultation with the faculty of the department, will make recommendations concerning
curriculum and classroom practice for the 2010-2011 academic year.
10
APPENDICES
APPENDIX I
SITUATIONS FOR ORAL ASSESSMENT IN GERMAN
Situation 1: Conversation with a stranger in Berlin
Imagine you are in Berlin sitting in a crowded café on the Kudamm and another person greets you
and asks if s/he can share the table with you. The professor will be playing the role of the other
person. Introduce yourself and tell the person where you are from. Ask the person what her name
is and where s/he comes from. S/he responds and asks you questions including about your
family. Describe your family and what their favorite activities are. Answer 5-8 questions the
person asks. When you realized it is late, excuse yourself and leave.
Possible questions: Wie heißen Sie? Woher kommen Sie? Wo wohnen Sie? Haben Sie ein Hobby? Was ist Ihr
Hobby? Mas machen Sie gern? Haben Sie eine groβe Familie? Wie heißt Ihr Vater? Wie heißt Ihre Mutter? Was
macht Ihr Vater? Was macht Ihrer Mutter? Haben Sie Geschwister? Wie heißen sie? Was machen sie? Wie alt sind
sie? Wo wohnen sie? Wer ist Ihre Lieblingsperson? Sehen Sie Filme gern? Was ist Ihr Lieblingsfilm?
Situation 2: Telephone conversation with a friend
You just arrived in Berlin and you want to spend the summer there. You are looking for a room to
share with a friend. The professor will be playing the role of the friend. You call your friend to find
out if s/he already saw any rooms. The friend tells you that she saw a very good room close to the
subway station and wants you to go and see it too. You will inquire about the details of the room.
After you agree when and where to meet hand up.
Possible questions: Wie geht es? Wo bist du? Was machst du heute? Wann kann ich das Zimmer sehen?
Können wir uns heute treffenn? Um wieviel Uhr sehen wir das Zimmer? Rufst du an? Wie ist das Zimmer? Wo ist das
Zimmer? Wieviel kostet das Zimmer? Was gibt es im Zimmer? Wieviele Stühle gibt es? Gibt es ein Radio? Gibt es
Betten? Gibt es ein Sofa? Wo ist der Sessel? Gibt es Wasser im Zimmer? Ist das Zimmer hell oder dunkel? Gibt es
Fenster? Wo treffen wir uns? Wann treffen wir uns?
11
Situation 3: A blind date
Hanna is your German friend who lives in NYC. Imagine Hanna’s brother Peter from Berlin is
spending a few days in NYC. The three of you are meeting in a restaurant. Your friend Hanna calls
to tell you she is late and describes Peter to you. The professor will be playing the role of Peter.
You see Peter at the bar waiting: greet him, introduce yourself and start a conversation. Ask him
what his favorite activities are; What he likes to eat; if he likes to cook; what his favorite weekend
activities are; which type of music he likes; if he has any hobbies; if he has a big family; where he
lives; etc. The conversation ends when Hanna arrives and you order dinner.
Possible questions: Wie geht es? Sind Sie Peter? Was machen Sie? Woher kommen Sie? Wo wohnen Sie? Was
machen am Wochenende? Haben Sie ein Hobby? Was ist ihr Hobby? Gehen Sie gern ins Theater? Kommen Sie
gern nach NYC? Was essen Sie gern? Kochen Sie gern? Spielen Sie gern Tennis? Arbeiten Sie? Wo arbeiten Sie?
Haben Sie eine groβe Familie? Wer ist Ihre Lieblingsperson? Was ist Ihr Lieblingsfilm? Wie heißen Ihre Eltern
(Vater/Mutter)? Was machen Ihre Eltern ? Wo wohnen sie? Wie alt sind sie?
12
APPENDIX II
HOLISTIC RUBRIC FOR ORAL ASSESSMENT IN GERMAN
Listening
Comprehension
Performance
exceeds
expectations
(4 points)
Performance
meets
expectations
(3 points)
Performance
almost meets
expectations
(2 points)
Performance
does not
meet
expectations
(0 – 1 points)
Fluidity
Pronunciation
Vocabulary
Grammar
Rich use of
vocabulary
Correct use
of basic
language
structures
(1-5 errors)
Does not interfere
with
communication
Adequate and
accurate use of
vocabulary for
this level
Adequate use
of basic
language
structures
(6-10 errors)
Speech
choppy and/or
slow with
frequent
pauses. Few
or incomplete
thoughts
Occasionally
interferes with
communication
Somewhat
inadequate
and/or
inaccurate use
of vocabulary
Emerging use
of basic
language
structures
(11-15 errors)
Speech
halting and
uneven with
long pauses
or incomplete
thoughts
Frequently
interferes with
communication
Inadequate
and/or
inaccurate use
of vocabulary
Inadequate
and/or
inaccurate
use of basic
language
structures
(more than 16
errors)
Student understands
the examiner’s
questions and
responds easily and
without probing
Speech
continuous
with few
pauses or
stumbling
Student understands
the examiner’s
questions and knows
how to respond but
needs occasional
probing
Some
hesitation but
manages to
continue and
to complete
her/his
thoughts
Student only
understands the
examiner’s questions
after probing
Student fails to
understand most
questions even after
probing
Enhances
communication
13
Summary & Conclusions
Generally speaking, data have portrayed a quite uniform acquisition scenario among the six
language groups tested. As shown in Chart 1 below, the majority of students met and exceeded
the standard set up for the course, namely Chinese (97%), French (88%), German (82%),
Hebrew (81%), Italian (86%), and Spanish (88%). The percentage of students that performed
below the expected level was irrelevant, and, in some languages, null. The number of students
that almost reached the expected proficiency level was also uniform among the different
languages ranging from the 3 percentile and the 18 percentile.
Chart 1
Percentage of students in each proficiency level among the six languages tested
70
60
Chinese
50
French
40
German
30
Hebrew
20
Italian
10
Spanish
0
Below
Almost Lev
Level
Exceed Lev
The scenario does not substantially change when we take a closer look at how students
performed with each parameter tested, namely “Listening Comprehension”, “Fluidity”,
“Pronunciation”, “Vocabulary”, and “Grammar”. Results, in fact, do not report significant
discrepancies among these parameters as well as the six language groups. As indicated below,
for each parameter, students reached or slightly exceeded the expected level (3.0).
Chart 2
Accuracy rates of the five parameters among the six languages
4
3.5
Chinese
3
French
2.5
2
German
1.5
Hebrew
Italian
1
0.5
Spanish
0
Listening
Fluidity
Pronunc.
Vocab.
Grammar
Interestingly, the typology and the complexity of the sound pattern of a language do not seem to
delay the development of a particular speaking skill. For instance, the tone system that
14
characterizes languages such as Chinese does not create major acquisition delays to students
whose native phonological system is quite different.
In conclusion, given the overall positive outcomes obtained, we can safely assume that
the instruction time, the teaching methodology practiced and the tools used appear to be effective
in helping our students reach the desired oral ability.
15
SPRING 2010
Date: 6-14-10
Department: Foreign Languages and Literatures
Course:
LG 111
Curriculum or Curricula: LA
PART I. STUDENT LEARNING OBJECTIVES
For Part I, attach the summary report (Tables 1-4) from the QCC Course Objectives Form.
TABLE 1. EDUCATIONAL CONTEXT
LG-111 is the first part of the introductory sequence of foreign language study (a requirement for a
successful transfer to the junior year of a baccalaureate program). It is a foundation course required for
(AA) degree in Liberal Arts and Sciences, Fine Arts, and Business Transfer.
TABLE 2. CURRICULAR OBJECTIVES
Note: Include in this table curriculum-specific objectives that meet Educational Goals 1 and 2:
Curricular objectives addressed by this course:
N/A
TABLE 3. GENERAL EDUCATION OBJECTIVES
Gen Ed
objective’s ID
number from
list (1-10)
General educational objectives addressed by this course: Select from preceding list.
(1) Communicate effectively through reading, writing, listening and speaking.
(2) Use analytical reasoning to identify issues or problems and evaluate evidence in
order to make informed decisions.
16
TABLE 4: COURSE OBJECTIVES AND STUDENT LEARNING OUTCOMES
Course objectives
Learning outcomes
1. Provide basic information in
Chinese about yourself, your
classmates, family, friends and
hobbies.
Students will be able to talk about themselves by providing their
name, age, nationality, phone number and occupation. They will also
be able to describe their family, friends, or hobbies by answering
questions in prompted dialogues.
2. Demonstrate ability to
accomplish simple
communicative tasks on every
day topics such as greeting
people or introducing yourself to
others, or describing what you
like to do in your spare time.
Students will be able to greet people and/or introduce themselves, or
exchange personal information by participating in communicative tasks
where they will play the role of talking to a stranger at a café in Berlin, or
talking on the telephone with a German friend.
17
PART II. ASSIGNMENT DESIGN: ALIGNING OUTCOMES, ACTIVITIES, AND
ASSESSMENT TOOLS
For the assessment project, you will be designing one course assignment, which will address at least one general
educational objective, one curricular objective (if applicable), and one or more of the course objectives. Please
identify these in the following table:
TABLE 5: OBJECTIVES ADDRESSED IN ASSESSMENT ASSIGNMENT
Course Objective(s) selected for assessment: (select from Table 4)
1. Provide basic information in Chinese about yourself, your classmates, family, friends and hobbies.
2. Demonstrate ability to accomplish simple communicative tasks on every day topics such as greeting
people or introducing yourself to others, or describing what you like to do in your spare time.
Curricular Objective(s) selected for assessment: (select from Table 2)
N/A
General Education Objective(s) addressed in this assessment: (select from Table 3)
1. Communicate effectively through reading, writing, listening and speaking.
2. Use analytical reasoning to identify issues or problems and evaluate evidence in order to make
informed decisions.
In the first row of Table 6 that follows, describe the assignment that has been selected/designed for this project.
In writing the description, keep in mind the course objective(s), curricular objective(s) and the general education
objective(s) identified above,
The assignment should be conceived as an instructional unit to be completed in one class session (such as a lab) or
over several class sessions. Since any one assignment is actually a complex activity, it is likely to require that
students demonstrate several types of knowledge and/or thinking processes.
Also in Table 6, please
a) identify the three to four most important student learning outcomes (1-4) you expect from this assignment
b) describe the types of activities (a – d) students will be involved with for the assignment, and
c) list the type(s) of assessment tool(s) (A-D) you plan to use to evaluate each of the student outcomes.
(Classroom assessment tools may include paper and pencil tests, performance assessments, oral questions,
portfolios, and other options.)
Note: Copies of the actual assignments (written as they will be presented to the students) should be gathered
in an Assessment Portfolio for this course.
18
TABLE 6: ASSIGNMENT, OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS
Briefly describe the assignment that will be assessed:
This project will assess students’ speaking abilities at a novice-mid proficiency level as described in
ACTFL 2006 guidelines. Their conversational abilities will be assessed by role-play activities in which
both students and their instructor play a specific role. Students may play the role of a student at QCC,
or a party guest trying to make new friends, or may share some personal information with a classmate
(played by the instructor) about a best friend. Oral tasks will prompt the students to provide personal
information, i.e. their name, age, phone number, nationality, or describing their hobbies.
Desired student learning outcomes
for the assignment
(Students will…)
List in parentheses the Curricular
Objective(s) and/or General
Education Objective(s) (1-10)
associated with these desired learning
outcomes for the assignment.
Briefly describe the range of
activities student will engage in
for this assignment.
Role-play pretending to talk to
a stranger at a café in Berlin, or
to a brother of a German friend
who is visiting New York City.
What assessment tools will be
used to measure how well
students have met each learning
outcome? (Note: a single
assessment tool may be used to
measure multiple learning
outcomes; some learning
outcomes may be measured using
multiple assessment tools.)
Gen-Ed objective
(1) Communicate effectively
through reading, writing, listening
and speaking.
(2) Use analytical reasoning to
identify issues or problems and
evaluate evidence in order to make
informed decisions.
Curricular objectives
N/A
Students will be asked to
complete a task where they
need to exchange personal
information with a new
acquaintance met at a café in
Berlin, or with a brother of a
German friend who is visiting
New York City and has poor
English skills, or talk to a
friend on the telephone to make
arrangements to meet see a
room for the summer in Berlin
(see Appendix I).
19
PART III. ASSESSMENT STANDARDS (RUBRICS)
Before the assignment is given, prepare a description of the standards by which students’ performance will be
measured. This could be a checklist, a descriptive holistic scale, or another form. The rubric (or a version of it) may
be given to the students with the assignment so they will know what the instructor’s expectations are for this
assignment.
Please note that while individual student performance is being measured, the assessment project is collecting
performance data ONLY for the student groups as a whole.
TABLE 7: ASSESSMENT STANDARDS (RUBRICS)
Brief description of assignment: (Copy from Table 6 above)
This project will assess students’ speaking abilities at a novice-mid proficiency level as described in
ACTFL 2006 guidelines. Their conversational abilities will be assessed by role-play activities in which
both students and their instructor play a specific role. Students may play the role of a student at QCC,
or a party guest trying to make new friends, or may share some personal information with a classmate
(played by the instructor) about a best friend. Oral tasks will prompt the students to provide personal
information, i.e. their name, age, phone number, nationality, or describing their hobbies.
Desired student learning
outcomes from the
assignment: (Copy from
Column 1, Table 6 above;
include Curricular and /or
General Education Objectives
addressed)
Gen-Ed objectives
(1) Communicate
effectively through
reading, writing, listening
and speaking.
3) Use analytical
reasoning
to identify issues or
problems and evaluate
evidence in order to make
informed decisions.
Assessment measures for
each learning outcome:
(Copy from Column 3,Table 6
above)
Students will be asked to
complete a task where they
need to exchange personal
information with a new
acquaintance met at a café
in Berlin, or with a brother
of a German friend who is
visiting New York City and
has poor English skills, or
talk to a friend on the
telephone to make
arrangements to meet see a
room for the summer in
Berlin.
Standards for student performance:
The parameters for measuring students’ speaking
abilities will be to determine whether:
(i) they understand the questions being
asked by the interlocutor,
(ii) they are accurate with regard to their
use of Chinese syntactic structures and
vocabulary,
(iii) they are understood by the
interlocutor by using the appropriate
intonation and pronunciation, and
(iv) they speak with some degree of
fluidity.
75% of the students tested are anticipated
to meet the course’s expectations as
described in the attached rubric (see
Appendix II).
Curricular objectives
N/A
20
PART IV. ASSESSMENT RESULTS
TABLE 8: SUMMARY OF ASSESSMENT RESULTS
Use the following table to report the student results on the assessment. If you prefer, you may report outcomes
using the rubric(s), or other graphical representation. Include a comparison of the outcomes you expected (from
Table 7, Column 3) with the actual results. NOTE: A number of the pilot assessments did not include expected
success rates so there is no comparison of expected and actual outcomes in some of the examples below. However,
projecting outcomes is an important part of the assessment process; comparison between expected and actual
outcomes helps set benchmarks for student performance.
TABLE 8: SUMMARY OF ASSESSMENT RESULTS
Desired student learning outcomes:
(Copy from, Column 1,Table 6 above;
include Curricular and/or General
Education Objectives addressed)
Student achievement: Describe the group achievement of each
desired outcome and the knowledge and cognitive processes
demonstrated.
Gen-Ed objectives
See Table 9.
Communicate effectively through
reading, writing, listening and
speaking.
Curricular objectives
N/A
TABLE 9. EVALUATION AND RESULTING ACTION PLAN
In the table below, or in a separate attachment, interpret and evaluate the assessment results, and describe the
actions to be taken as a result of the assessment. In the evaluation of achievement, take into account student
success in demonstrating the types of knowledge and the cognitive processes identified in the Course
Objectives.
A. Analysis and interpretation of assessment results:
What does this show about what and how the students learned?
19 students completed the speaking task in German, and their performance was rated
according to five parameters: listening comprehension, fluidity, pronunciation,
vocabulary and grammar. Student performance was scored using the scoring rubric.
The average score achieved for listening comprehension was 2.63, for pronunciation was
3.0, for fluidity of speech was 2.71, for proficient use of vocabulary was 2.63, and for
accuracy of grammar was 2.37. On all these parameters, most students on average met
expectations (see chart 1).
The total points a student could achieve was 20, after adding up all five parameters, each
worth a maximum of four points. The scale agreed upon beforehand by the Foreign
Languages Assessment Committee defined the following ranges: 0-5 points = student
performance does not meet expectations; 6–10 points = student performance almost
meets expectations; 11-15 points = student performance meets expectations; 16-20=
student performance exceeds expectations.
21
The average of total points achieved by students tested in German was 11.0. According to
the scale, this average falls in the lower end of the 11-15 range, signifying that student
overall performance on the task meets expectations.
The Foreign Language Assessment Committee also predicted that 75% of students tested
would achieve a score indicating that their performance meets expectations. The results
for percentage of students who achieved each performance level in German are the
following: 16% of students scored at Level 2, 6-10 points, and their performance almost
meets expectations; 53% scored at Level 3, 11-15 points, and their performance meets
expectations; finally, 31% scored at Level 4, and the quality of their performance exceeds
expectations. Compared to fall 2009 when 6% of the students scored below expectations,
in spring none of the students scored between 0-5, i.e. below expectation. Levels 3 and 4
add up to a total of 84% of students who meet or exceed expectations, as defined in the
attached rubric.
B. Evaluation of the assessment process:
What do the results suggest about how well the assignment and the assessment process
worked both to help students learn and to show what they have learned?
The results of the speaking task in German show that, as predicted, at least 75% of
students tested meet or exceed expectations on overall performance of the task: the actual
percentage of students tested in Spring 2010 who meet or exceed expectations is 84%.
The largest portion of students (53%) fell into the category rated as meeting expectations;
that is, achieving 11-15 points out of 20. Exceeding Expectations were 31% of the
weakest students almost meeting the expectations were 16%.
If we look at student performance according to individual parameters, the highest average
score was in accuracy of pronunciation (3.0%). Students also performed well in fluidity
(2.71%), listening and vocabulary was at the same level (2.6%). The lowest scoring
parameter was grammar (2.4). Students showed a weakness in grammar due to the fact
that the German grammar at the early stages of language learning is rather complex
because of the cases, and three genders of words.
B2) Evaluation of the assessment
The results obtained seem to indicate that the assessment tools used and the assessment
process undertaken have been successful in accurately determining our students’ oral
proficiency level. Not only has it shown that more than three quarters of the students had
met the expectation, but the study guide and the precise instructions has also provided
them with the appropriate tool that would facilitate their learning process.
22
C. Resulting action plan:
Based on A and B, what changes, if any, do you anticipate making?
The overall average achieved by students on the speaking task in German shows a
satisfactory result, with 84% of students meeting expectations as defined by the Foreign
Language Assessment Committee. These results are slightly better than those obtained in
the Oral Assessment conducted in fall 09, when 82% of the students met the expectations,
but 6% were below expectations. The results of the Spring 2010 show an overall
improvement of most of the students and none performed below expectations. As in fall
2009 the scored for pronunciation was the highest and grammar scored the lowest in
spring 2010. The Foreign Language Assessment Committee, in consultation with the
faculty of the department, will discuss these results and make recommendations
concerning curriculum and classroom practice for the 2010-2011 academic year.
23
APPENDICES
APPENDIX I
SITUATIONS FOR ORAL ASSESSMENT IN GERMAN
Situation 1: Conversation with a stranger in Berlin
Imagine you are in Berlin sitting in a crowded café on the Kudamm and another person greets you
and asks if s/he can share the table with you. The professor will be playing the role of the other
person. Introduce yourself and tell the person where you are from. Ask the person what her name
is and where s/he comes from. S/he responds and asks you questions including about your
family. Describe your family and what their favorite activities are. Answer 5-8 questions the
person asks. When you realized it is late, excuse yourself and leave.
Possible questions: Wie heißen Sie? Woher kommen Sie? Wo wohnen Sie? Haben Sie ein Hobby? Was ist Ihr
Hobby? Mas machen Sie gern? Haben Sie eine groβe Familie? Wie heißt Ihr Vater? Wie heißt Ihre Mutter? Was
macht Ihr Vater? Was macht Ihrer Mutter? Haben Sie Geschwister? Wie heißen sie? Was machen sie? Wie alt sind
sie? Wo wohnen sie? Wer ist Ihre Lieblingsperson? Sehen Sie Filme gern? Was ist Ihr Lieblingsfilm?
Situation 2: Telephone conversation with a friend
You just arrived in Berlin and you want to spend the summer there. You are looking for a room to
share with a friend. The professor will be playing the role of the friend. You call your friend to find
out if s/he already saw any rooms. The friend tells you that she saw a very good room close to the
subway station and wants you to go and see it too. You will inquire about the details of the room.
After you agree when and where to meet hand up.
Possible questions: Wie geht es? Wo bist du? Was machst du heute? Wann kann ich das Zimmer sehen?
Können wir uns heute treffenn? Um wieviel Uhr sehen wir das Zimmer? Rufst du an? Wie ist das Zimmer? Wo ist das
Zimmer? Wieviel kostet das Zimmer? Was gibt es im Zimmer? Wieviele Stühle gibt es? Gibt es ein Radio? Gibt es
Betten? Gibt es ein Sofa? Wo ist der Sessel? Gibt es Wasser im Zimmer? Ist das Zimmer hell oder dunkel? Gibt es
Fenster? Wo treffen wir uns? Wann treffen wir uns?
24
Situation 3: A blind date
Hanna is your German friend who lives in NYC. Imagine Hanna’s brother Peter from Berlin is
spending a few days in NYC. The three of you are meeting in a restaurant. Your friend Hanna calls
to tell you she is late and describes Peter to you. The professor will be playing the role of Peter.
You see Peter at the bar waiting: greet him, introduce yourself and start a conversation. Ask him
what his favorite activities are; What he likes to eat; if he likes to cook; what his favorite weekend
activities are; which type of music he likes; if he has any hobbies; if he has a big family; where he
lives; etc. The conversation ends when Hanna arrives and you order dinner.
Possible questions: Wie geht es? Sind Sie Peter? Was machen Sie? Woher kommen Sie? Wo wohnen Sie? Was
machen am Wochenende? Haben Sie ein Hobby? Was ist ihr Hobby? Gehen Sie gern ins Theater? Kommen Sie
gern nach NYC? Was essen Sie gern? Kochen Sie gern? Spielen Sie gern Tennis? Arbeiten Sie? Wo arbeiten Sie?
Haben Sie eine groβe Familie? Wer ist Ihre Lieblingsperson? Was ist Ihr Lieblingsfilm? Wie heißen Ihre Eltern
(Vater/Mutter)? Was machen Ihre Eltern ? Wo wohnen sie? Wie alt sind sie?
25
APPENDIX II
HOLISTIC RUBRIC FOR ORAL ASSESSMENT IN GERMAN
Listening
Comprehension
Performance
exceeds
expectations
(4 points)
Performance
meets
expectations
(3 points)
Performance
almost meets
expectations
(2 points)
Performance
does not
meet
expectations
(0 – 1 points)
Fluidity
Student understands
the examiner’s
questions and
responds easily and
without probing
Speech
continuous
with few
pauses or
stumbling
Student understands
the examiner’s
questions and knows
how to respond but
needs occasional
probing
Some
hesitation but
manages to
continue and
to complete
her/his
thoughts
Student only
understands the
examiner’s questions
after probing
Student fails to
understand most
questions even after
probing
Pronunciation
Vocabulary
Grammar
Rich use of
vocabulary
Correct use
of basic
language
structures
(1-5 errors)
Does not interfere
with
communication
Adequate and
accurate use of
vocabulary for
this level
Adequate use
of basic
language
structures
(6-10 errors)
Speech
choppy and/or
slow with
frequent
pauses. Few
or incomplete
thoughts
Occasionally
interferes with
communication
Somewhat
inadequate
and/or
inaccurate use
of vocabulary
Emerging use
of basic
language
structures
(11-15 errors)
Speech
halting and
uneven with
long pauses
or incomplete
thoughts
Frequently
interferes with
communication
Inadequate
and/or
inaccurate use
of vocabulary
Inadequate
and/or
inaccurate
use of basic
language
structures
(more than 16
errors)
Enhances
communication
26
LA ACADEMY COHORT
Final Examination
Date 6-14-10
Department: Foreign Languages and Literatures
Course:
LG 111
Curriculum or Curricula: LA
PART I. STUDENT LEARNING OBJECTIVES
For Part I, attach the summary report (Tables 1-4) from the QCC Course Objectives Form.
TABLE 1. EDUCATIONAL CONTEXT
LG111 is the first part of the introductory sequence of foreign language study (a requirement for a
successful transfer to the junior year of a baccalaureate program). It is a foundation course required for
(AA) degree in Liberal Arts and Sciences, Fine Arts, and Business Transfer.
TABLE 2. CURRICULAR OBJECTIVES
Note: Include in this table curriculum-specific objectives that meet Educational Goals 1 and 2:
Curricular objectives addressed by this course:
N/A
TABLE 3. GENERAL EDUCATION OBJECTIVES
Gen Ed
objective’s ID
number from
list (1-10)
General educational objectives addressed by this course: Select from preceding list.
(1) Communicate effectively through reading, writing, listening and speaking.
(2) Use analytical reasoning to identify issues or problems and evaluate evidence in order to
make informed decisions.
TABLE 4: COURSE OBJECTIVES AND STUDENT LEARNING OUTCOMES
Course objectives
Learning outcomes
Read, understand, write simple
texts on familiar topics such as
Students will be able to read, understand and write simple texts on
the mentioned topics such as greetings, family, dates, time and
greetings, family, dates, time hobbies.
and hobbies; express
preferences, likes and dislikes.
27
PART II. ASSIGNMENT DESIGN: ALIGNING OUTCOMES, ACTIVITIES, AND
ASSESSMENT TOOLS
For the assessment project, you will be designing one course assignment, which will address at least one general
educational objective, one curricular objective (if applicable), and one or more of the course objectives. Please
identify these in the following table:
TABLE 5: OBJECTIVES ADDRESSED IN ASSESSMENT ASSIGNMENT
Course Objective(s) selected for assessment: (select from Table 4)
1. Provide basic information in German about yourself, your classmates, family, friends and hobbies.
2. Demonstrate ability to accomplish simple communicative tasks on every day topics such as greeting
people or introducing yourself to others, or describing what you like to do in your spare time.
Curricular Objective(s) selected for assessment: (select from Table 2)
N/A
General Education Objective(s) addressed in this assessment: (select from Table 3)
1. Communicate effectively through reading, writing, listening and speaking.
2. Use analytical reasoning to identify issues or problems and evaluate evidence in order to make
informed decisions.
In the first row of Table 6 that follows, describe the assignment that has been selected/designed for this project.
In writing the description, keep in mind the course objective(s), curricular objective(s) and the general education
objective(s) identified above,
The assignment should be conceived as an instructional unit to be completed in one class session (such as a lab) or
over several class sessions. Since any one assignment is actually a complex activity, it is likely to require that
students demonstrate several types of knowledge and/or thinking processes.
Also in Table 6, please
a) identify the three to four most important student learning outcomes (1-4) you expect from this assignment
b) describe the types of activities (a – d) students will be involved with for the assignment, and
c) list the type(s) of assessment tool(s) (A-D) you plan to use to evaluate each of the student outcomes.
(Classroom assessment tools may include paper and pencil tests, performance assessments, oral questions,
portfolios, and other options.)
Note: Copies of the actual assignments (written as they will be presented to the students) should be gathered
in an Assessment Portfolio for this course.
28
TABLE 6: ASSIGNMENT, OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS
Briefly describe the assignment that will be assessed:
The final exam will be used as the assessment tool to evaluate students’ abilities in listening, reading,
and writing as well as their knowledge in vocabulary and grammar.
Listening comprehension abilities will be evaluated by asking the students to listen to two paragraphs:
one about someone’s personal information, including plans for the weekend; the other one about
purchase at a store. They need to then complete a true or false and a multiple choice exercise.
Reading abilities will be evaluated by asking the students to read a passage about looking for a penpal
and hobbies and answer eight questions.
Vocabulary knowledge will be assessed by asking the students to complete four sections about days of
the week, years, items on a desk and in a room.
Grammar accuracy will be assessed by asking the students to complete six sections using the correct
word endings, verb conjugation, negation, possessive and personal pronuns, etc.
Writing abilities will be assessed by asking the students to describe their room, their dream car or a
person, etc.
Desired student learning outcomes
for the assignment
(Students will…)
List in parentheses the Curricular
Objective(s) and/or General
Education Objective(s) (1-10)
associated with these desired learning
outcomes for the assignment.
Gen-Ed objective
(1) Communicate effectively through
reading, writing, listening and
speaking.
(2) Use analytical reasoning to
identify issues or problems and
evaluate evidence in order to make
informed decisions.
Curricular objectives
N/A
Briefly describe the range of
activities student will engage in
for this assignment.
Listening activity: listen to two
paragraphs read to them and
complete two exercises.
Reading activity: read a
passage about weekend
activities and answer questions.
Vocabulary knowledge:
complete four sections about
days of the week, years, items
on a desk and in a room.
Grammar accuracy: complete
six sections using the correct
word endings, verb
conjugation, negation,
possessive and personal
pronuns, etc.
Writing activities: describe
their room, their dream car or a
person, etc.
What assessment tools will be
used to measure how well
students have met each learning
outcome? (Note: a single
assessment tool may be used to
measure multiple learning
outcomes; some learning
outcomes may be measured using
multiple assessment tools.)
Students will listen and
understand two paragraphs
read to them and complete the
exercises.
Students will read a paragraph
and answer questions.
Students will provide the
correct vocabulary on dates,
activities and hobbies.
Students will provide sentences
using the correct word order
and the measure word.
Students will describe their
room, dream car, or a person.
29
PART III. ASSESSMENT STANDARDS (RUBRICS)
Before the assignment is given, prepare a description of the standards by which students’ performance will be
measured. This could be a checklist, a descriptive holistic scale, or another form. The rubric (or a version of it) may
be given to the students with the assignment so they will know what the instructor’s expectations are for this
assignment.
Please note that while individual student performance is being measured, the assessment project is collecting
performance data ONLY for the student groups as a whole.
TABLE 7: ASSESSMENT STANDARDS (RUBRICS)
Brief description of assignment: (Copy from Table 6 above)
The final exam will be used as the assessment tool to evaluate students’ abilities in listening, reading,
and writing as well as their knowledge in vocabulary and grammar.
Listening comprehension abilities will be evaluated by asking the students to listen to two paragraphs:
one about someone’s personal information, including plans for the weekend; the other one about
purchase at a store. They need to then complete a true or false and a multiple choice exercise.
Reading abilities will be evaluated by asking the students to read a passage about looking for a penpal
and hobbies and answer eight questions.
Vocabulary knowledge will be assessed by asking the students to complete four sections about days of
the week, years, items on a desk and in a room.
Grammar accuracy will be assessed by asking the students to complete six sections using the correct
word endings, verb conjugation, negation, possessive and personal pronuns, etc.
Writing abilities will be assessed by asking the students to describe their room, their dream car or a
person, etc.
Desired student learning
outcomes from the
assignment: (Copy from
Column 1, Table 6 above;
include Curricular and /or
General Education Objectives
addressed)
Gen-Ed objectives
(1) Communicate effectively
through reading, writing,
listening and speaking.
Assessment measures for
each learning outcome:
(Copy from Column 3,Table 6
above)
Students will be asked to
listen and understand two
paragraphs read to them and
complete the exercises.
Standards for student performance:
Students’ listening comprehension ability
will be determined by calculating their
accuracy rates in providing the correct
answers to the questions asked. 75 per
cent of them are expected to meet the
course standards (80% accuracy rate).
Students’ reading comprehension ability
Students will be asked to
will be determined by calculating their
read a paragraph and answer accuracy rates in providing the correct
questions.
4) Use analytical reasoning
answers to the questions asked. 75 per
to identify issues or problems
cent of them are expected to meet the
and evaluate evidence in order Students will be asked to
course standards (80% accuracy rate).
to make informed decisions.
provide the correct
vocabulary on dates,
Students’ vocabulary knowledge will be
Curricular objectives
activities and hobbies.
determined by calculating the accuracy
N/A
Students will be asked to
provide sentences using the
rates in providing the correct words or
phrases in a given situation. 75 per cent of
30
correct word order and the
measure word.
them are expected to meet the course
standards (80% accuracy rate).
Students will be asked to
describe themselves, their
hobbies and their activities.
Students’ grammar knowledge will be
determined by calculating their accuracy
rates in providing the correct forms or
structures. 75 per cent of them are
expected to meet the course standards
(80% accuracy rate) (see Appendix I for
scoring scales).
The parameters used to measure students’
writing abilities will be to determine
whether:
(i) they will provide all the information
they have been asked,
(ii) their writing displays a vocabulary
appropriate to their proficiency level,
(iii)
they show an adequate control of
the syntactic structures for a Chinese
beginning level class,
(iv)
their sentences are fully
developed, even though sporadically
connected.
75% of the students tested are anticipated
to meet the course’s expectations as
described in the attached rubric (see
Appendix II).
31
PART IV. ASSESSMENT RESULTS
TABLE 8: SUMMARY OF ASSESSMENT RESULTS
Use the following table to report the student results on the assessment. If you prefer, you may report outcomes
using the rubric(s), or other graphical representation. Include a comparison of the outcomes you expected (from
Table 7, Column 3) with the actual results. NOTE: A number of the pilot assessments did not include expected
success rates so there is no comparison of expected and actual outcomes in some of the examples below. However,
projecting outcomes is an important part of the assessment process; comparison between expected and actual
outcomes helps set benchmarks for student performance.
TABLE 8: SUMMARY OF ASSESSMENT RESULTS
Desired student learning outcomes:
(Copy from, Column 1,Table 6 above;
include Curricular and/or General
Education Objectives addressed)
Student achievement: Describe the group achievement of each
desired outcome and the knowledge and cognitive processes
demonstrated.
Gen-Ed objectives
See Table 9
Communicate effectively through
reading, writing, listening and
speaking.
Curricular objectives
N/A
32
TABLE 9. EVALUATION AND RESULTING ACTION PLAN
In the table below, or in a separate attachment, interpret and evaluate the assessment results, and describe the
actions to be taken as a result of the assessment. In the evaluation of achievement, take into account student
success in demonstrating the types of knowledge and the cognitive processes identified in the Course
Objectives.
A. Analysis and interpretation of assessment results:
What does this show about what and how the students learned?
The data of 20 students that completed the final exam in German (LG111) was gathered. Their
performance was rated according to five categories: listening comprehension, vocabulary, grammar,
reading and writing.
The results are as follows:
SCORING SHEET‐ LG111 General Performance in E ach Category
Listening Vocabulary Grammar
Reading
Writing Total Range 0‐16 0‐16 0‐36 0‐16 0‐16 0‐100 Average Score # of Students 14.3 12.7 30.0
14.8
12.55
84.4 20 20 20
20
20
20 The average score achieved was 84.4% which is above the expected percentage.
The total points a student could achieve (adding up all five categories) was 100. The scale agreed
upon beforehand by the Foreing language Assessment Committee defined the following ranges: 060points= student performance does not meet expectations; 61–78 points= student performance
almost meets expectations; 79-89 points= student performance meets expectations; 90-100= student
performance exceeds expectations.
Range # 0f Students Percentage OVERALL Student Performance Does not meet expectations Almost meets expectations
0‐60 61‐78 Meets expectations
79‐89 Exceed expectations
90‐100 2 3 7 8 10% 15% 35% 40% The overall student performance shows that 75% of students meet or exceed expectations;
3% almost meets the expectactions; and only 2% did not meet the expectations.
The results for percentage of students in each category are as follows.
Student Performance in LISTENING
Range # of Students Does not meet expectations 0‐9.5 0 Almost meets expectations 10‐12.5 3
Meets expectations
13‐14.5 7
Exceed expectations
15‐16 10 Percentage 0% 15%
35%
50% 33
Student Performance in VOCABULARY
Range # of Students Does not meet expectations 0‐9.5 3 Almost meets expectations 10‐12.5 3
Meets expectations
13‐14.5 8
Exceed expectations
15‐16 6 Percentage 15% 15%
40%
30% Student Performance in GRAMMAR Does not meet Almost meets expectations expectations 0‐21.5 22‐28 Meets expectations
28.5‐32 Exceed expectations
32.5‐36 3 4 4 9 15% 20% 20% 45% Student Performance in READING
Does not meet Almost meets expectations expectations 0‐9.5 10‐12.5 Meets expectations
13‐14.5 Exceed expectations
15‐16 0 2 7 11 0% 10% 35% 55% Student Performance in WRITING
Does not meet Almost meets expectations expectations 0‐9.5 10‐12.5 Meets expectations
13‐14.5 Exceed expectations
15‐16 4 6 3 7 20% 30% 15% 35% Range # of Students Percentage Range # of Students Percentage Range # of Students Percentage B. Evaluation of the assessment process:
What do the results suggest about how well the assignment and the assessment process worked both
to help students learn and to show what they have learned?
The overall average of 84% in German is misleading. The results in the vocabulary,
grammar, and writing categories show that, contrary to the expectation, less than 75% of
students tested meet or exceed expectations on overall performance of the task. The only
categories where the students meet or exceed expectations is in the reading (90%) category
and listening (85%). The actual percentage of students tested who meet or exceed
expectations in vocabulary is 70%, in grammar is 65%, in writing is 50%.
B2) Evaluation of the assessment
The results obtained seem to indicate that the assessment tools used and the assessment
process undertaken have been effective in determining our students’ proficiency level.
34
C. Resulting action plan:
Based on A and B, what changes, if any, do you anticipate making?
The overall average achieved by students on the task in German shows a satisfactory result,
with 84% of students meeting expectations as defined by the Foreing Language Assessment
Committee. However, the averages achieved on each individual category in German show
show areas of strength and weakneses. Except for listening and reading, three areas need to
improve. Vocabulary is slightly below the expected level; the two weakest areas are
grammar, and writing. These results need to be addressed. The Foreign Language
Assessment Committee will meet in the fall to discuss the results and determine
recommendations, which will then be shared with the faculty.
35
APPENDICES
APPENDIX I
SCORING SCALES
Grammar Task Scale (Range: 0 to 36)
GRAMMAR
SECTION
ACCURATE USE OF THE GRAMMATICAL STRUCTURES LEARNED IN
THE COURSE
Performance More than 90% completion of the task
exceeds
(32.5- 36 points)
expectations
Performance Between 79% and 89% completion of the task
meets
(28.5- 32 points)
expectations
Performance Between 64% and 78% completion of the task
almost
(22 – 28 points)
meets
expectations
Performance Between 0% to 60% completion of the task
does not
(0 - 21.5 points)
meet
expectations
36
Vocabulary Task Scale (Range: 0 to 16)
VOCABULARY ACCURATE AND ADEQUATE USE OF THE VOCABULARY
SECTION
LEARNED IN THE COURSE
Performance
exceeds
expectations
More than 90% completion of the task
(15 - 16 points)
Performance
meets
expectations
Between 79% and 89% completion of the task
(13 – 14.5 points)
Performance
almost meets
expectations
Between 62% and 78% completion of the task
(10 – 12.5 points)
Performance
does not meet
expectations
Between 0% to 61% completion of the task
(0- 9.5 points)
Listening Task Scale (Range: 0 to 16)
LISTENING
SECTION
UNDERSTAND PHRASES, EXPRESSIONS AND SHORT MESSAGES
RELATED TO THE TOPICS COVERED IN THE COURSE
Performance
More than 90% completion of the task
exceeds
expectations (15 – 16 points)
Performance Between 79% and 89% completion of the task
meets
(13- 14.5 points)
expectations
Performance
almost
meets
expectations
Performance
does not
meet
expectations
Between 65% and 78% completion of the task
(10 – 12.5 points)
Between 0% to 60% completion of the task
(0 to 9.5 points)
37
Reading Task Scale (Range: 0 to 16)
READING
SECTION
UNDERSTAND SHORT AND SIMPLE MESSAGES
RELATED TO THE TOPICS COVERED IN THE COURSE
Performance
More than 90% completion of the task
exceeds
expectations (15 – 16 points)
Performance Between 79% and 89% completion of the task
meets
(13- 14.5 points)
expectations
Performance Between 65% and 78% completion of the task
almost
(10- 12.5 points)
meets
expectations
Performance Between 0% to 60% completion of the task
does not
(0 to 9.5 points)
meet
expectations
38
APPENDIX II
Writing Task Holistic Rubric
Exceeds
expectations
Meets
expectations
Almost meets
expectations
Does not meet
expectations
Task
Completion
Level of
Discourse
Vocabulary
Grammar
Superior completion
of the task. Ss fully
address the
information
requested, and
provide additional
details
Completion of task.
Ss fully address the
information
provided, but do not
provide additional
details
Partial completion of
task. Ss complete no
more than 60% of
the information
requested
Sentences are fully
developed and
interconnected with
conjunctions (e.g.
AND, BUT, or
BECAUSE )
Rich use of
vocabulary
Perfect control of the
syntactic structures
required (accuracy
level 90% - 100%)
Sentences are fully
developed. Cohesive
devices, however,
are sporadically used
Adequate and
accurate use of
vocabulary
Sentences are
somewhat complete.
Rare use of cohesive
devices
Somewhat
inadequate and/or
inaccurate use of
vocabulary
Adequate control of
the syntactic
structures. Some
grammatical errors
(accuracy level
79% - 89%)
Emerging control of
syntactic structures.
Several grammatical
errors (accuracy
level 61% - 78%)
Minimal completion
of task. Ss complete
less than 40% of the
information
requested.
Sentences are mostly
incomplete. No use
of cohesive devices
Inadequate and/or
inaccurate use of
vocabulary
Minimal control of
syntactic structures.
Numerous
grammatical errors
(accuracy level
0% - 60%)
39
Conclusions & Action Plan
Generally speaking, data portray a uniform acquisition scenario among the foreign languages
evaluated, namely, Chinese, French, German, Hebrew, Italian and Spanish. As Table 1 shows, all
six language groups display similar percentages of students reaching or exceeding the
proficiency levels set up for a Beginning I language course.
Table 1
Percentage of students reaching or exceeding the required proficiency levels for each language group
80
70
Chinese
60
French
50
German
40
Hebrew
30
Italian
20
Spanish
10
0
However, contrary to our predictions, no language group reported more than 75 percent of their
students meeting the proficiency standards of the course. In fact, an average of less than 2/3 of
our learners reached the desired proficiency levels.
In any case, a closer look at the data reveals an interesting acquisition phenomenon. Our L2
learners seem to encounter less difficulty in developing the required speaking and auditory skills
than their reading or writing abilities. Despite their limited amount of instruction and exposure to
the target language, a relatively high number of students (avg. 87%), are able to speak and
converse at the expected level of a beginning course, irrespective of the language they are
learning. Furthermore, their ability to comprehend oral messages seems to be equally developed.
In fact, the percentage of students meeting the standards of the course is 75 percent (see Table 2
below).
40
Table 2
Percentage of students reaching or exceeding the required level of oral, auditory, reading and writing proficiency
for each language group
100
90
80
70
Oral
Auditory
Reading
Writing
60
50
40
30
20
10
0
Chinese
French
Hebrew
German
Italian
Spanish
The reading and writing skills, however, do not seem to follow a similarly expedite
developmental pattern. Data indicate that an average of only 63 percent of the students write at
the level required by the course. If we eliminate the high percentage displayed by the students of
Chinese (92%), the number reaches worrisome levels (55%). These results are quite interesting
and intriguing. Normally, one would expect just opposite outcomes given that performance
limitations of psychological nature (i.e. nervousness, anxiety) are usually more visible in
generating oral than written language.
Such a clear discrepancy in students’ oral and written use of their target language (henceforth:
L2) calls for a re-analysis of the oral assessment tools used, mainly with regard to their reliability
and usefulness in generating accurate data. This re-evaluation should help us eliminate or, at
least reduce, the effects of external factors that might have altered the results obtained. The
personal and oral nature of this assessment process, in fact, may be easily subject to human
errors or bias. With that in mind, the department will organize some training sessions for the
instructors administering the oral test in order to ensure that the assessment tool is used
uniformly and appropriately.
In any case, the low percentage of students meeting the writing standards of the course is also
reflected in the appropriate use of L2 syntactic structures. Data indicate that an average of 44
percent of the students tested were able to reach the accuracy standards required by the course.
And again, if we eliminate the percentage of the students of Chinese from our calculations, the
percentile lowers to 38 percent. Interestingly, such a lower accuracy level is also encountered in
L2 oral production. Across the six language groups, the appropriate use of grammatical
structures is one of the weakest oral abilities. That being the case, one could assume that
41
internalizing L2 grammar rules is problematic. Such difficulties are displayed in both oral and
written language modes.
Fortunately, this does not seem to be case when students need to master the required vocabulary.
Lexical items do not appear to be acquisitionally as problematic as the grammar rules. As Table
3 indicates, the overall percentage of students using the required vocabulary is clearly higher
(62%), even though the Italian and Spanish learners are still showing some problems (33% and
43%, respectively)
Table 3
Percentage of students reaching or exceeding the required levels of lexical and syntactic proficiency for each
language group
80
70
60
50
Grammar
Voc.
40
30
20
10
0
Chinese French Hebrew German
Italian
Spanish
In sum, students have shown to be struggling with the mastering of L2 grammar rules. Their
behavior is quite consistent, equally involving the oral and written use of their target language.
The reasons justifying this lack of accuracy may be various and of different nature. The groups
that have shown greater number of problems are those learning morphologically rich languages
such as Italian, French Spanish and German. At a beginning level, the acquisition of grammar is
mostly morphologically-based. It is well known that morphology is a linguistic component that is
usually acquired very late.
However, the similarly lower number of students of Italian and Spanish reaching the desired
level of lexical knowledge seems to highlight a general behavior of poor study skills and habits.
In fact, L2 learners appear to fall short whenever the skill requires a more attentive participation
and thorough analysis of the language. With this in mind, the department will implement a
greater variety of grammar activities and tasks in the curricula of the beginning language courses
along with a closer monitoring of students' homework and lab assignments.
42