QCC COURSE ASSESSMENT FORM QCC C A

QCC C

OURSE

A

SSESSMENT

F

ORM

(S

HORT

)

QCC COURSE ASSESSMENT FORM

Fall 2004, Rev. 6/15/07

Date: 1-15-10

Department: Foreign Languages and Literatures

Course: LI 111

Curriculum or Curricula: LA

PART I. STUDENT LEARNING OBJECTIVES

For Part I, attach the summary report (Tables 1-4) from the QCC Course Objectives Form.

TABLE 1. EDUCATIONAL CONTEXT

LI-111 is the first part of the introductory sequence of foreign language study (a requirement for a successful transfer to the junior year of a baccalaureate program). It is a foundation course required for (AA) degree in

Liberal Arts and Sciences, Fine Arts, and Business Transfer.

TABLE 2. CURRICULAR OBJECTIVES

Note: Include in this table curriculum-specific objectives that meet Educational Goals 1 and 2:

Curricular objectives addressed by this course:

N/A

TABLE 3. GENERAL EDUCATION OBJECTIVES

Gen Ed objective’s ID number from list (1-10)

General educational objectives addressed by this course: Select from preceding list.

(1) Communicate effectively through reading, writing, listening and speaking.

(2) Use analytical reasoning to identify issues or problems and evaluate evidence in order to make informed decisions.

1(45)

TABLE 4: COURSE OBJECTIVES AND STUDENT LEARNING OUTCOMES

Course objectives

1. Provide basic information in

Italian about yourself, your city, your classmates, your family, friends, hobbies, and daily activities.

2.

Demonstrate ability to accomplish

Learning outcomes

Students will be able to talk about themselves by providing their name, address, phone number, school schedule. They will also be able to describe their family or hobbies by answering questions in prompted dialogues. simple communicative tasks on every day topics such as greeting people or introduce yourself to others, or describing your life in school or at work, or during your spare time.

Students will be able to greet people and/or introduce themselves, or exchange personal information by participating in communicative tasks where they will play the role of a new student at QCC or a party guest trying to make friends.

2(45)

PART II. ASSIGNMENT DESIGN: ALIGNING OUTCOMES, ACTIVITIES, AND

ASSESSMENT TOOLS

For the assessment project, you will be designing one course assignment , which will address at least one general educational objective, one curricular objective (if applicable), and one or more of the course objectives. Please identify these in the following table:

TABLE 5: OBJECTIVES ADDRESSED IN ASSESSMENT ASSIGNMENT

Course Objective(s) selected for assessment: (select from Table 4)

1. Provide basic information in Italian about yourself, your city, your classmates, your family, friends, hobbies, and daily activities.

2. Demonstrate ability to accomplish simple communicative tasks on every day topics such as greeting people or introducing yourself to others, or describing your life in school or at work and in your spare time.

Curricular Objective(s) selected for assessment: (select from Table 2)

N/A

General Education Objective(s) addressed in this assessment: (select from Table 3)

1. Communicate effectively through reading, writing, listening and speaking.

2. Use analytical reasoning to identify issues or problems and evaluate evidence in order to make informed decisions.

In the first row of Table 6 that follows, describe the assignment that has been selected/designed for this project.

In writing the description, keep in mind the course objective(s), curricular objective(s) and the general education objective(s) identified above,

The assignment should be conceived as an instructional unit to be completed in one class session (such as a lab) or over several class sessions. Since any one assignment is actually a complex activity, it is likely to require that students demonstrate several types of knowledge and/or thinking processes.

Also in Table 6, please a) identify the three to four most important student learning outcomes (1-4) you expect from this assignment b) describe the types of activities (a – d) students will be involved with for the assignment, and c) list the type(s) of assessment tool(s) (A-D) you plan to use to evaluate each of the student outcomes.

(Classroom assessment tools may include paper and pencil tests, performance assessments, oral questions, portfolios, and other options.)

Note: Copies of the actual assignments (written as they will be presented to the students) should be gathered in an Assessment Portfolio for this course.

3(45)

TABLE 6: ASSIGNMENT, OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS

Briefly describe the assignment that will be assessed:

This project will assess students’ speaking abilities at a novice-mid proficiency level as described in the ACTFL

2006 guidelines. Their conversational abilities will be assessed by engaging them in role-play activities where both they and the instructor play a specific role. Students may play the role of a new student at QCC, or a party guests trying to make new friends, or may share some personal information with a close friend (played by the instructor) about a new boy/girlfriend. Oral tasks will prompt the students in providing personal information, i.e. their name, address, phone number, or class schedule, or describing their daily activities and hobbies.

Desired student learning outcomes for the assignment

(Students will…)

List in parentheses the Curricular

Objective(s) and/or General

Education Objective(s) (1-10) associated with these desired learning outcomes for the assignment.

Gen-Ed objective



Curricular objectives

N/A

Briefly describe the range of activities student will engage in for this assignment.

Role-play at the Registrar’s office, a friend’s house, or a caffè (Italian coffee shop).

What assessment tools will be used to measure how well students have met each learning

outcome? (Note: a single assessment tool may be used to measure multiple learning outcomes; some learning outcomes may be measured using multiple assessment tools.)

Students will be asked to complete a task where they need to exchange personal information with a new acquaintance met at the Registrar’s office, or in a friend’s house, or discuss information about a new boy/girlfriend’s with a close friend while having something to drink in a nearby caffè (see Appendix I).

4(45)

PART III. ASSESSMENT STANDARDS (RUBRICS)

Before the assignment is given, prepare a description of the standards by which students’ performance will be measured. This could be a checklist, a descriptive holistic scale, or another form. The rubric (or a version of it) may be given to the students with the assignment so they will know what the instructor’s expectations are for this assignment.

Please note that while individual student performance is being measured, the assessment project is collecting performance data ONLY for the student groups as a whole.

TABLE 7: ASSESSMENT STANDARDS (RUBRICS)

Brief description of assignment: (C opy from Table 6 above )

This project will assess students’ speaking abilities at a novice-mid proficiency level as described in ACTFL 2006 guidelines. Their conversational abilities will be assessed by engaging them in role-play activities where both they and the instructor play a specific role. Students may play the role of a new student at QCC, or a party guest trying to make new friends, or may share some personal information with a close friend (played by the instructor) about a new boy/girlfriend. Oral tasks will prompt the students in providing personal information, i.e. their name, address, phone number, or class schedule, or describing their daily activities and hobbies.

Desired student learning outcomes from the assignment: ( Copy from

Column 1, Table 6 above; include Curricular and /or

General Education Objectives addressed)

Gen-Ed objectives


2) Use analytical reasoning to identify issues or problems and evaluate evidence in order to make informed decisions.


N/A

Assessment measures for each learning outcome:

(Copy from Column 3,Table 6 above)

Students will be asked to complete a task where they need to exchange personal information with a new acquaintance met at the Registrar’s office, or in a friend’s house, or discuss information about a new boy/girlfriend’s with a close friend while having something to drink in a nearby caffè .

Standards for student performance:

The parameters for measuring students’ speaking abilities will be to determine whether:

(i) they understand the questions being asked by the interlocutor,

(ii) they use Italian syntactic structures and vocabulary accurately,

(iii) they make themselves understood by using the correct intonation and pronunciation.

(iv) they speak with some degree of fluidity.

75% of the students tested are anticipated to meet the course’s expectations as described in the attached rubric (see Appendix II).

5(45)

PART IV. ASSESSMENT RESULTS

TABLE 8: SUMMARY OF ASSESSMENT RESULTS

Use the following table to report the student results on the assessment. If you prefer, you may report outcomes using the rubric(s), or other graphical representation. Include a comparison of the outcomes you expected (from


Table 7, Column 3) with the actual results. NOTE: A number of the pilot assessments did not include expected success rates so there is no comparison of expected and actual outcomes in some of the examples below. However, projecting outcomes is an important part of the assessment process; comparison between expected and actual outcomes helps set benchmarks for student performance.

Student achievement: Describe the group achievement of each desired outcome and the knowledge and cognitive processes demonstrated.

Desired student learning outcomes:

(Copy from, Column 1,Table 6 above; include Curricular and/or General

Education Objectives addressed)


Communicate effectively through reading, writing, listening and speaking.


N/A

See Table 9

6(45)

TABLE 9. EVALUATION AND RESULTING ACTION PLAN

In the table below, or in a separate attachment, interpret and evaluate the assessment results, and describe the actions to be taken as a result of the assessment. In the evaluation of achievement, take into account student success in demonstrating the types of knowledge and the cognitive processes identified in the Course

Objectives.

A. Analysis and interpretation of assessment results:

What does this show about what and how the students learned?

232 students completed the speaking task in Italian, and their performance was rated according to five parameters: listening comprehension, fluidity, pronunciation, vocabulary and grammar. Student performance was scored using the above-mentioned scoring rubric.

The average score achieved for listening comprehension was 3.24; the average score for pronunciation was 3.35, and the average score for fluidity of speech was 3.10. On these three parameters, students on average met expectations.

The average score for proficient use of vocabulary was 2.98, while the average score for accuracy of grammar was 2.95. In both these parameters students on average almost met expectations (the average was quite close to the 3.0 mark) (see Chart 1 below):

Chart 1

3.4

3.3

3.2

3.1

3

2.9

2.8

2.7

LISTENING FLUIDITY

PRONUNCIATI

ON

VOCABULARY GRAMMAR

3.35

2.98

2.95

OVERALL

Series1 3.24

3.1

3.12

The total points a student could achieve (adding up all five parameters, each worth a maximum of four points) was 20. The scale agreed upon beforehand by the Assessment

Committee defined the following ranges: 0-5 points= student performance does not meet expectations; 6 – 10 points= student performance almost meets expectations; 11-15 points= student performance meets expectations; 16-20= student performance exceeds expectations.

The average of total points achieved by students tested in Italian was 15.62. According to the scale, this average falls in the 11-15 range, signifying that student overall performance

7(45)

on the task meets expectations.

The Assessment Committee also predicted that 75% of students tested would achieve a score indicating that their performance meets expectations. The results for percentage of students who achieved each performance level in Italian are the following: 2% of students scored at Level 1, 0-5 points, and their performance does not meet expectations; 11% scored at Level 2, 6-10 points, and their performance almost meets expectations; 28% scored at Level 3, 11-15 points, and their performance meets but does not exceed expectations; finally, 58% scored at Level 4, and the quality of their performance exceeds expectations. Combining Levels 3 and 4 gives us a total of 86% of students who meet or exceed expectations, as defined by the scoring rubric (see Chart 2 below):

Chart 2

B. Evaluation of the assessment process :

What do the results suggest about how well the assignment and the assessment process worked both to help students learn and to show what they have learned?

B1) Evaluation of students’ results

The results of the speaking task in Italian show that, as predicted, at least 75% of students tested meet or exceed expectations on overall performance of the task: the actual percentage of students tested in Fall 2009 who meet or exceed expectations is 86%. The largest portion of students (58%) fell into the category rated as “exceeding expectations”, that is, achieving 16-20 points out of 20. Combining Levels 1 and 2, the percentage of students whose performance does not meet expectations is 13%.

8(45)

However, if we look at students’ performance according to individual parameters, the highest average score was in pronunciation (3.35%).

The second and third highest scoring parameters were ‘listening comprehension’ (3.24%) and ‘fluidity’ (3.10%). Such high accuracy rates may be due to the fact that these skills require the least analytic skill on the part of the student. Listening comprehension, in fact, is a passive skill and is one the first abilities to be acquired. Another plausible reason could be that students have practiced intensely during the entire assessment process. This could have also positively influenced their speech flow.

The two lowest scoring parameters were ‘vocabulary’ (2.98%) and ‘grammar’ (2.95%).

These parameters require active knowledge of the material learned. Proficient use of vocabulary entails memorization, and accurate application of the rules of grammar requires analytic skills. In the context of a speaking task, a weaker performance on these two particular parameters would be expected.

B2) Evaluation of the assessment tools

The results obtained seem to indicate that the assessment tools used and the assessment process undertaken have been successful in accurately determining our students’ oral proficiency level. Not only has it shown that more than ¾ of the students had met the expectation, but it has also provided them with the appropriate tool that would facilitate their learning process. The great majority of students, in fact, took the speaking task assessment very seriously, and studied extensively for it.

Some faculty members, however, have indicated that there appears to be a wide gap in terms of proficiency between the ‘meets the expectations’ and ‘exceeds the expectations’ levels. It has been speculated that this could be one of the reasons why the data reported such a large number of students at a very high proficiency level. Therefore, it has been suggested to add to the current rubric an intermediate proficiency level that would capture those students that perform slightly above their course level.

C. Resulting action plan:

Based on A and B, what changes, if any, do you anticipate making?

Further actions

The overall average achieved by students on the speaking task in Italian shows a satisfactory result, with 86% of students meeting expectations as defined by the

Assessment Committee. The averages achieved on each individual parameter in Italian show areas of relative strength, as well as two areas of relative weakness: vocabulary and grammar. After discussion of the results across all languages taught in the department, it has been decided by the Assessment Committee to postpone any changes in curriculum emphases or instructional methodology until we have gathered more complete data, including student performance on written tasks, as well as a second assessment using the speaking task. After compiling and analyzing data for all assessment completed in academic year 2009-2010, the Assessment Committee, in consultation with the faculty of

9(45)

the department, will make recommendations concerning curriculum and classroom practice for the 2010-2011 academic year.

10(45)

APPENDICES

APPENDIX I

SITUATIONS FOR ORAL ASSESSMENT IN ITALIAN

GUIDELINES



It is a conversation: teacher–student (about 10 minutes)



Students select at random one of three situations (see study guide)



Students should answer in complete sentences. English is not allowed



Listening comprehension, fluidity, pronunciation, vocabulary and grammar will be evaluated



For top results (see also attached rubric): a.

Listening comprehension: understand the questions and respond easily without probing b.

Fluidity: speak continuously with few pauses or stumbling c.

Pronunciation: pronounce properly d.

Vocabulary: use the appropriate vocabulary. Don’t always use and repeat the same words

STUDY GUIDE e.

Grammar: use correctly the required grammatical structures

PRACTICE the following situations and questions: a.

In class b.

With your classmates c.

With a tutor at the Student Learning Center d.

With your Italian speaking friends or family

11(45)

SITUAZIONE 1: AT QCC

Imagine that you are a new student. You are standing in line at the Registrar and begin a conversation with the student in front of you. (Your teacher will play the role of the other student in line). What would you say, how would you answer his/her questions. Make sure you answer in complete sentences.

Greet each other and exchange names

Personal info: age, nationality, how is he doing/feeling, phone number

Services on campus/location of certain buildings

Daily activities (schedule): courses, activities during the day

Activities during the weekend: likes and dislikes

Plans after college.

Domande possibili:

Come stai? Quanti anni hai? Di dove sei? Qual è il tuo numero di telefono?

Dov’è la mensa? Dov’è la biblioteca? Dov’è….

Che classi frequenti? A che ora è la tua classe di matematica (o un’altra materia)? Che giorno della settimana hai la classe di matematica (o un’altra materia)?

A che ora pranzi? Di solito che cosa prendi per pranzo? A che ora finiscono le tue lezioni? Dove vai dopo le lezioni?

Che cosa fai il weekend? Giochi a calcio? Studi? Lavori? Che cosa fai? Dove lavori? Quando? Dove?

Con chi?

Che cosa ti piace fare nel tuo tempo libero?

SITUAZIONE 2: AT A PARTY

Imagine that you are at a party and you start talking with a person there. (Your teacher will play the role of the other person at the party). What would you say, how would you answer his/her questions. Make sure you answer in complete sentences.


Exchange personal info: age, nationality, how is he doing/feeling, birthday, phone number

Talk about your family

Activities during the week

Plans for next year

Likes and dislikes

Domande possibili :

Come stai? Quanti anni hai? Quand’è il tuo compleanno? Qual è il tuo numero di teléfono? Di dove sei? Dove abiti? Com’è la tua città? Vai spesso a Manhattan? Che cosa fai quando vai a Manhattan?

Qual è il tuo posto preferito a Manhattan?

Di solito che cosa fai il martedì mattina? Che cosa fai il venerdì será? A che ora pranzi durante la settimana? Che cosa mangi? Dove mangi? Con chi mangi?

Hai un hobby? Che cosa ti piace fare nel tuo tempo libero?

12(45)

SITUAZIONE 3: AT YOUR FAVORITE COFFEE SHOP

Imagine that you are in a café talking with your best friend about your new boyfriend/girlfriend. You teacher will play the role of your best friend. What would you say, how would you answer his/her questions. Make sure you answer in complete sentences.

Greet each other

Order food and drinks

Girlfriend/boyfriend info: age, nationality, how is he doing/feeling, birthday, phone number

Activities during the week. What does he/she do?

Plans for the weekend. What are they planning to do during the weekend?

Likes and dislikes. What does he/she like?

Domande possibili :

Come stai? Che cosa prendi? Che cosa preferisci prendere? Vuoi un caffè?

E il tuo ragazzo/la tua ragazza: Quanti anni ha? Di dov’è? Come sta? Dov’è adesso? Quando è il suo compleanno? Qual è il suo numero di teléfono?

Lavora o studia? Dove? Che giorni? Che cosa fa il lunedì mattina? Che cosa fa il sabato será? Che cosa ti piace fare con il tuo ragazzo/la tua ragazza? Quando uscite insieme? Che cosa fate insieme?

Dove andate? Con chi andate?

13(45)

APPENDIX II

HOLISTIC RUBRIC FOR ORAL ASSESSMENT IN ITALIAN

Performance exceeds expectations

Performance meets expectations

Performance almost meets expectations

Performance does not meet expectations

Listening comprehension

Student understands the examiner’s questions and responds easily and without probing

4 points

Student understands the examiner’s questions and knows how to respond but needs occasional probing

3 points

Student only understands the examiner’s questions after probing

2 points

Student fails to understand most questions even after probing

0-1 points

Fluidity Pronunciation Vocabulary Grammar

Speech continuous with few pauses or stumbling

4 points

Some hesitation but manages to continue and to complete her/his thoughts

3 points

Speech choppy and/or slow with frequent pauses. Few or incomplete thoughts

2 points

Speech halting and uneven with long pauses or incomplete thoughts

0-1 points

Enhances communication

4 points

3 points

2 points

Does not interfere with communication

Occasionally interferes with communication

Frequently interferes with communication

0-1 points

Rich use of vocabulary

4 points

Adequate and accurate use of vocabulary for this level

3 points

2 points and/or use of

Somewhat inadequate and/or inaccurate use of vocabulary.

Inadequate inaccurate vocabulary

0-1 points

Correct use of basic language structures

(1-5 errors)

4 points

Adequate use of basic language structures

(6-10 errors)

3 points

Emerging use of basic language structures

(11-15 errors)

2 points

Inadequate and/or inaccurate use of basic language structures

(more than

16 errors)

0-1 points

14(45)

Summary & Conclusion

Generally speaking, data have portrayed a quite uniform acquisition scenario among the six language groups tested. As shown in Chart 1 below, the majority of students met and exceeded the standard set up for the course, namely Chinese (97%), French (88%), German (82%),

Hebrew (81%), Italian (86%), and Spanish (88%). The percentage of students that performed below the expected level was irrelevant, and, in some languages, null. The number of students that almost reached the expected proficiency level was also uniform among the different languages ranging from the 3 percentile and the 18 percentile.

Chart 1

Percentage of students in each proficiency level among the six languages tested

70

60

50

40

30

20

10

Chinese

French

German

Hebrew

Italian

Spanish

0

Below Almost Lev Level Exceed Lev

The scenario does not substantially change when we take a closer look at how students performed with each parameter tested, namely “Listening Comprehension”, “Fluidity”,

“Pronunciation”, “Vocabulary”, and “Grammar”. Results, in fact, do not report significant discrepancies among these parameters as well as the six language groups. As indicated below, for each parameter, students reached or slightly exceeded the expected level (3.0).

Chart 2

Accuracy rates of the five parameters among the six languages

4

3.5

3

2.5

2

1.5

1

0.5

0

Chinese

French

German

Hebrew

Italian

Spanish

Listening Fluidity Pronunc.

Vocab.

Grammar

Interestingly, the typology and the complexity of the sound pattern of a language do not seem to delay the development of a particular speaking skill. For instance, the tone system that

15(45)

characterizes languages such as Chinese does not create major acquisition delays to students whose native phonological system is quite different.

In conclusion, given the overall positive outcomes obtained, we can safely assume that the instruction time, the teaching methodology practiced and the tools used appear to be effective in helping our students reach the desired oral ability.

16(45)

QCC C

OURSE

A

SSESSMENT

F

ORM

(S

HORT

)


Fall 2004, Rev. 6/15/07

Date: 6- 5-10


Course: LI 111










N/A






17(45)



1. Provide basic information in

Italian about yourself, your city, your classmates, your family, friends, hobbies, and daily activities.

2.

Demonstrate ability to accomplish


Students will be able to talk about themselves by providing their name, address, phone number, school schedule. They will also be able to describe their family or hobbies by answering questions in prompted dialogues. simple communicative tasks on every day topics such as greeting people or introduce yourself to others, or describing your life in school or at work, or during your spare time.

Students will be able to greet people and/or introduce themselves, or exchange personal information by participating in communicative tasks where they will play the role of a new student at QCC or a party guest trying to make friends.

18(45)









N/A










19(45)



This project will assess students’ speaking abilities at a novice-mid proficiency level as described in the ACTFL

2006 guidelines. Their conversational abilities will be assessed by engaging them in role-play activities where both they and the instructor play a specific role. Students may play the role of a new student at QCC, or a party guests trying to make new friends, or may share some personal information with a close friend (played by the instructor) about a new boy/girlfriend. Oral tasks will prompt the students in providing personal information, i.e. their name, address, phone number, or class schedule, or describing their daily activities and hobbies.










N/A


Role-play at the Registrar’s office, a friend’s house, or a caffè (Italian coffee shop).



Students will be asked to complete a task where they need to exchange personal information with a new acquaintance met at the Registrar’s office, or in a friend’s house, or discuss information about a new boy/girlfriend’s with a close friend while having something to drink in a nearby caffè .(see Appendix I).

20(45)






This project will assess students’ speaking abilities at a novice-mid proficiency level as described in ACTFL 2006 guidelines. Their conversational abilities will be assessed by engaging them in role-play activities where both they and the instructor play a specific role. Students may play the role of a new student at QCC, or a party guest trying to make new friends, or may share some personal information with a close friend (played by the instructor) about a new boy/girlfriend. Oral tasks will prompt the students in providing personal information, i.e. their name, address, phone number, or class schedule, or describing their daily activities and hobbies.








N/A



Students will be asked to complete a task where they need to exchange personal information with a new acquaintance met at the Registrar’s office, or in a friend’s house, or discuss information about a new boy/girlfriend’s with a close friend while having something to drink in a nearby caffè .


The parameters for measuring students’ speaking abilities will be to determine whether:

(i) they understand the questions being asked by the interlocutor,

(ii) they use Italian syntactic structures and vocabulary accurately,

(iii) they make themselves understood by using the correct intonation and pronunciation.

(iv) they speak with some degree of fluidity.

(See Appendix 2)


21(45)













N/A

See Table 9

22(45)



Objectives.



193 students completed the speaking task in Italian, and their performance was rated according to five parameters: listening comprehension, fluidity, pronunciation, vocabulary and grammar. Student performance was scored using the scoring rubric (see

Appendix 2).

The average score achieved for listening comprehension was 3.08; the average score for pronunciation was 3.20. On these two parameters, students on average met expectations

(see chart 1 below).

The average score for fluidity was 2.84; the average score for proficient use of vocabulary was 2.75; the average score for accuracy of grammar was 2.72. In these parameters students on average almost met expectations (the average was quite close to the 3.0 mark).

The total points a student could achieve (adding up all five parameters, each worth a maximum of four points) was 20. The scale agreed upon beforehand by the Assessment

Committee defined the following ranges: 0-5 points= student performance does not meet expectations; 6 – 10 points= student performance almost meets expectations; 11-15 points= student performance meets expectations; 16-20= student performance exceeds expectations.

The average of total points achieved by students tested in Italian was 14.58. According to the scale, this average falls in the 11-15 range, signifying that student overall performance on the task meets expectations.

The Assessment Committee also predicted that 75% of students tested would achieve a score indicating that their performance meets expectations. The results for percentage of students who achieved each performance level in Italian are the following: 1% of students scored at Level 1, 0-5 points, and their performance does not meet expectations;

13% scored at Level 2, 6-10 points, and their performance almost meets expectations;

42% scored at Level 3, 11-15 points, and their performance meets but does not exceed expectations; finally, 44% scored at Level 4, and the quality of their performance exceeds expectations. Adding together Levels 3 and 4 gives us a total of 86% of students who meet or exceed expectations, as defined by the scoring rubric.



B1) Evaluation of students’ results

The results of the speaking task in Italian show that, as predicted, at least 75% of students tested meet or exceed expectations on overall performance of the task: the actual

23(45)

percentage of students tested in Fall 2009 who meet or exceed expectations is 86%.

Adding Levels 1 and 2 together, the percentage of students whose performance does not meet expectations is 14%.

If we look at students’ performance according to individual parameters, the highest average score was in pronunciation (3.20%).

The second highest scoring parameters were ‘listening comprehension’ (3.08%). The other three parameters: fluidity, vocabulary and grammar all scored very similarly, respectively 2.84, 2.75, and 2.72.

B2) Evaluation of the assessment tools

The results obtained seem to indicate that the assessment tools used and the assessment process undertaken have been successful in accurately determining our students’ oral proficiency level. Not only has it shown that more than ¾ of the students had met the expectation, but it has also provided them with the appropriate tool that would facilitate their learning process. The great majority of students, in fact, took the speaking task assessment very seriously, and studied extensively for it.



Further actions

The overall average achieved by students on the speaking task in Italian shows a satisfactory result, with 86% of students meeting expectations as defined by the

Assessment Committee. These results are very similar to those obtained in the Oral

Assessment conducted in fall 09. The Assessment Committee, in consultation with the faculty of the department, will discuss these results and make recommendations concerning curriculum and classroom practice for the 2010-2011 academic year.

24(45)

APPENDICES

APPENDIX I

SITUATIONS FOR ORAL ASSESSMENT IN ITALIAN

GUIDELINES



It is a conversation: teacher–student (about 10 minutes)



Students select at random one of three situations (see study guide)



Students should answer in complete sentences. English is not allowed



Listening comprehension, fluidity, pronunciation, vocabulary and grammar will be evaluated



For top results (see also attached rubric): f.

Listening comprehension: understand the questions and respond easily without probing g.

Fluidity: speak continuously with few pauses or stumbling h.

Pronunciation: pronounce properly i.

Vocabulary: use the appropriate vocabulary. Don’t always use and repeat the same words

STUDY GUIDE j.

Grammar: use correctly the required grammatical structures

PRACTICE the following situations and questions: e.

In class f.

With your classmates g.

With a tutor at the Student Learning Center h.

With your Italian speaking friends or family

25(45)

SITUAZIONE 1: AT QCC

Imagine that you are a new student. You are standing in line at the Registrar and begin a conversation with the student in front of you. (Your teacher will play the role of the other student in line). What would you say, how would you answer his/her questions. Make sure you answer in complete sentences.


Personal info: age, nationality, how is he doing/feeling, phone number

Services on campus/location of certain buildings

Daily activities (schedule): courses, activities during the day

Activities during the weekend: likes and dislikes

Plans after college.

Domande possibili:

Come stai? Quanti anni hai? Di dove sei? Qual è il tuo numero di telefono?

Dov’è la mensa? Dov’è la biblioteca? Dov’è….

Che classi frequenti? A che ora è la tua classe di matematica (o un’altra materia)? Che giorno della settimana hai la classe di matematica (o un’altra materia)?

A che ora pranzi? Di solito che cosa prendi per pranzo? A che ora finiscono le tue lezioni? Dove vai dopo le lezioni?

Che cosa fai il weekend? Giochi a calcio? Studi? Lavori? Che cosa fai? Dove lavori? Quando? Dove?

Con chi?

Che cosa ti piace fare nel tuo tempo libero?

SITUAZIONE 2: AT A PARTY

Imagine that you are at a party and you start talking with a person there. (Your teacher will play the role of the other person at the party). What would you say, how would you answer his/her questions. Make sure you answer in complete sentences.


Exchange personal info: age, nationality, how is he doing/feeling, birthday, phone number

Talk about your family

Activities during the week

Plans for next year

Likes and dislikes

Domande possibili :

Come stai? Quanti anni hai? Quand’è il tuo compleanno? Qual è il tuo numero di teléfono? Di dove sei? Dove abiti? Com’è la tua città? Vai spesso a Manhattan? Che cosa fai quando vai a Manhattan?

Qual è il tuo posto preferito a Manhattan?

Di solito che cosa fai il martedì mattina? Che cosa fai il venerdì será? A che ora pranzi durante la settimana? Che cosa mangi? Dove mangi? Con chi mangi?

Hai un hobby? Che cosa ti piace fare nel tuo tempo libero?

26(45)

SITUAZIONE 3: AT YOUR FAVORITE COFFEE SHOP

Imagine that you are in a café talking with your best friend about your new boyfriend/girlfriend. You teacher will play the role of your best friend. What would you say, how would you answer his/her questions. Make sure you answer in complete sentences.

Greet each other

Order food and drinks

Girlfriend/boyfriend info: age, nationality, how is he doing/feeling, birthday, phone number

Activities during the week. What does he/she do?

Plans for the weekend. What are they planning to do during the weekend?

Likes and dislikes. What does he/she like?

Domande possibili :

Come stai? Che cosa prendi? Che cosa preferisci prendere? Vuoi un caffè?

E il tuo ragazzo/la tua ragazza: Quanti anni ha? Di dov’è? Come sta? Dov’è adesso? Quando è il suo compleanno? Qual è il suo numero di teléfono?

Lavora o studia? Dove? Che giorni? Che cosa fa il lunedì mattina? Che cosa fa il sabato será? Che cosa ti piace fare con il tuo ragazzo/la tua ragazza? Quando uscite insieme? Che cosa fate insieme?

Dove andate? Con chi andate?

27(45)

APPENDIX II

HOLISTIC RUBRIC FOR ORAL ASSESSMENT IN ITALIAN





Listening comprehension

Student understands the examiner’s questions and responds easily and without probing

4 points

Student understands the examiner’s questions and knows how to respond but needs occasional probing

3 points

Student only understands the examiner’s questions after probing

2 points

Student fails to understand most questions even after probing

0-1 points

Fluidity Pronunciation Vocabulary Grammar

Speech continuous with few pauses or stumbling

4 points

Some hesitation but manages to continue and to complete her/his thoughts

3 points

Speech choppy and/or slow with frequent pauses. Few or incomplete thoughts

2 points

Speech halting and uneven with long pauses or incomplete thoughts

0-1 points

Enhances communication

4 points

3 points

2 points

Does not interfere with communication

Occasionally interferes with communication

Frequently interferes with communication

0-1 points


4 points

Adequate and accurate use of vocabulary for this level

3 points

2 points and/or use of

Somewhat inadequate and/or inaccurate use of vocabulary.

Inadequate inaccurate vocabulary

0-1 points

Correct use of basic language structures

(1-5 errors)

4 points

Adequate use of basic language structures

(6-10 errors)

3 points

Emerging use of basic language structures

(11-15 errors)

2 points

Inadequate and/or inaccurate use of basic language structures

(more than

16 errors)

0-1 points

28(45)

QCC C

OURSE

A

SSESSMENT

F

ORM

(S

HORT

)


Fall 2004, Rev. 6/15/07

Date 6-7-10


Course: LI 111










N/A






29(45)



Read, understand, write simple texts on familiar topics such as greetings, food, daily life, and hobbies; express preferences, likes and dislikes.


Students will be able to read, understand and write simple texts on the mentioned topics such as food, daily life, hobbies, dates and time.

30(45)









N/A










31(45)



The final exam will be used as the assessment tool to evaluate students’ abilities in listening, reading, and writing as well as their knowledge in vocabulary and grammar.

Listening comprehension abilities will be evaluated by asking the students to listen to two paragraphs: one about someone’s daily routine; the other one about two people ordering food in a restaurant. Students need to then complete a true or false section and a multiple choice section.

Reading abilities will be evaluated by asking the students to read two passages: one about a student’s life, the other about the city of Milan. Students need to answer true/false questions in one instance, and multiple choice questions in the other.

Vocabulary knowledge will be assessed by asking the students to complete two sections: one with words indicating directions. The other is a passage describing a young man, his life and his hobbies. Scrambled vocabulary is given that needs to be put in the correct place in the passage.

Grammar accuracy will be assessed by asking the students to complete four sections using the correct 1. adjective forms, 2. verb forms, 3. preposition joint to an article, 4. and finally plural forms of entire sentences in the singular form.

Writing abilities will be assessed by asking the students to describe a typical week of their lives.










N/A


Listening comprehension: listen to two paragraphs and answer true/false and multiple choice questions: one about someone’s daily routine; the other one about two people ordering food in a restaurant.

Reading activity: read two passages and answer questions.

Vocabulary activity: complete two sections with the appropriate vocabulary.

Grammar activities: complete four sections with verbs, adjectives, prepositions joint to an article, and plural forms.

Writing activities: write a composition about Listening activity: listen to two paragraphs and complete two exercises about a typical week.



Students will be asked to listen to two passages and answer questions.

Students will be asked to read two paragraphs and answer questions.

Students will be asked to provide the correct vocabulary in two exercises.

Students will be asked to provide sentences using the correct word order and the measure word.

Students will be asked to describe a typical week

32(45)






The final exam will be used as the assessment tool to evaluate students’ abilities in listening, reading, and writing as well as their knowledge in vocabulary and grammar.

Listening comprehension abilities will be evaluated by asking the students to listen to two paragraphs: one about someone’s daily routine; the other one about two people ordering food in a restaurant. Students need to then complete a true or false section and a multiple choice section.

Reading abilities will be evaluated by asking the students to read two passages: one about a student’s life, the other about the city of Milan. Students need to answer true/false questions in one instance, and multiple choice questions in the other.

Vocabulary knowledge will be assessed by asking the students to complete two sections: one with words indicating directions. The other is a passage describing a young man, his life and his hobbies. Scrambled vocabulary is given that needs to be put in the correct place in the passage.

Grammar accuracy will be assessed by asking the students to complete four sections using the correct 1. adjective forms, 2. verb forms, 3. prepositions joint to an article, 4. and finally plural forms of entire sentences in the singular form.

Writing abilities will be assessed by asking the students to describe a typical week of their lives.








N/A



Students will be asked to listen to two passages and answer questions.

Students will be asked to read two paragraphs and answer questions.

Students will be asked to provide the correct vocabulary in two exercises.

Students will be asked to provide sentences using the correct word order and the measure word.

Students will be asked to describe a typical week


Students’ listening comprehension ability will be determined by calculating their accuracy rates in providing the correct answers to the questions asked. 75 per cent of them are expected to meet the course standards (80% accuracy rate).

Students’ reading comprehension ability will be determined by calculating their accuracy rates in providing the correct answers to the questions asked. 75 per cent of them are expected to meet the course standards (80% accuracy rate).

Students’ vocabulary knowledge will be determined by calculating the accuracy rates in providing the correct words or phrases in a given situation. 75 per cent of them are expected to meet the course standards (80% accuracy rate).

Students’ grammar knowledge will be determined by calculating their accuracy rates in providing the correct forms or structures. 75 per cent of them are expected to meet the course standards (80% accuracy rate) (see Appendix I for scoring scales)

33(45)

34(45)

The parameters used to measure students’ writing abilities will be to determine whether:

(i) they will provide all the information they have been asked,

(ii) their writing displays a vocabulary appropriate to their proficiency level,

(iii) they show an adequate control of the syntactic structures for an Italian beginning level class,

(iv) their sentences are fully developed, even though sporadically connected.














N/A

See Table 9

35(45)



Objectives.



The data of 211 students that completed the final exam (assessment tool) in Italian was gathered.

Their performance was rated according to five categories: listening comprehension, vocabulary, grammar, reading and writing.

The results are as follows:

SCORING SHEET ‐ LI General Performance in E ach Category

Range

Listening

0 ‐ 16

Vocabulary

0 ‐ 16

Grammar

0 ‐ 36

Reading

0 ‐ 16

Writing

0 ‐ 16

Total

0 ‐ 100

Average

Score

# of

Students

14

211

10.9

211

22.2

211

13.5

211

11.8

211

72.5

211

The average score achieved was 72.5%

The total points a student could achieve (adding up all five categories) was 100. The scale agreed upon beforehand by the Assessment Committee defined the following ranges: 0-60points= student performance does not meet expectations; 61–78 points= student performance almost meets expectations; 79-89 points= student performance meets expectations; 90-100= student performance exceeds expectations.

Range

Student overall performance according to each proficiency level

Does not meet expectations Almost meets expectations Meets expectations

0 ‐ 60 61 ‐ 78 79 ‐ 89

# 0f Students 50 78 51

Percentage 24% 37% 24%

Exceed expectations

90 ‐ 100

32

15%

The overall student performance shows that only 39% of students meet o exceed expectations .

36(45)

The results for percentage of students in each category are as follows:

LISTENING.

Student performance …

Range

# of Students

Does not meet expectations 0 ‐ 95

8

Almost meets expectations 10 ‐ 12.5

34

Meets expectations

13.14.5

71

4% 16% 34% Percentage

Range

# of Students

Percentage

Range

# of Students

Percentage

VOCABULARY.


Does not meet expectations 0 ‐ 9.5

72

Almost meets expectations

48

10 ‐ 12.5

34% 23%

GRAMMAR.

Student performance …

Does not meet expectations

0 ‐ 21.5

Almost expectations

22 ‐ 28

meets

93

44%

56

27%

Exceed expectations

15 ‐ 16

98

46%

Meets expectations

13 ‐ 14.5

45

21%

Exceed expectations

15 ‐ 16

46

22%

Meets expectations

28.5

‐ 32

31

15%

Exceed expectations

32.5

‐ 36

31

15%

Range

# of Students

Percentage

Range

# of Students

Percentage

READING.

Student performance …

Does not meet Almost meets expectations

0 ‐ 9.5

expectations

10 ‐ 12.5

8

4%

49

23%

WRITING .


Does not meet expectations

0 ‐ 9.5

Almost expectations

10 ‐ 12.5

meets

44

21%

55

26%

Meets expectations

13 ‐ 14.5

77

36%

Exceed expectations

15 ‐ 16

77

31%

Meets expectations

13 ‐ 14.5

67

32%

Exceed expectations

15 ‐ 16

45

21%



The results in the vocabulary, grammar, reading and writing categories show that, contrary to the prediction, less than 75% of students tested meet or exceed expectations on overall performance of the task. The only category where the students meet or exceed

37(45)

expectations is listening. The actual percentage of students tested who meet or exceed expectations in vocabulary is 43%, in grammar is 30%, in reading is 72%, in writing is

53% and in listening is 80%

B2) Evaluation of the assessment

.

The results obtained seem to indicate that the assessment tools used and the assessment process undertaken have been effective in determining our students’ proficiency level.



The overall average achieved by students on the assessment in Italian shows unsatisfactory results. The averages achieved on each individual category in Italian show that all areas need to improve except for listening. The two weakest areas are vocabulary and grammar.

These results need to be addressed. The committee will meet in the fall to discuss the results and determine recommendations. It will then share and discuss the recommendations with the faculty.

38(45)

APPENDICES

APPENDIX I

SCORING SCALES

Grammar Task Scale (Range: 0 to 36)

GRAMMAR

SECTION





ACCURATE USE OF THE GRAMMATICAL STRUCTURES LEARNED IN

THE COURSE

More than 90% completion of the task

(32.5- 36 points)

Between 79% and 89% completion of the task

(28.5- 32 points)


(22 – 28 points)

Between 0% to 60% completion of the task

(0 - 21.5 points)

39(45)

Vocabulary Task Scale (Range: 0 to 16)

ACCURATE AND ADEQUATE USE OF THE VOCABULARY VOCABULARY

SECTION


LEARNED IN THE COURSE


(15 - 16 points)


(13 – 14.5 points)





(10 – 12.5 points)


(0- 9.5 points)

Listening Task Scale (Range: 0 to 16)

LISTENING

SECTION


UNDERSTAND PHRASES, EXPRESSIONS AND SHORT MESSAGES

RELATED TO THE TOPICS COVERED IN THE COURSE


(15 – 16 points)





(13- 14.5 points)


(10 – 12.5 points)


(0 to 9.5 points)

40(45)

Reading Task Scale (Range: 0 to 16)

UNDERSTAND SHORT AND SIMPLE MESSAGES READING

SECTION


RELATED TO THE TOPICS COVERED IN THE COURSE


(15 – 16 points)


(13- 14.5 points)





(10- 12.5 points)


(0 to 9.5 points)

41(45)

Exceeds expectations

Meets expectations

Almost meets expectations

Does not meet expectations

APPENDIX II

Task

Completion

Superior completion of the task. Ss fully address the information requested, and provide additional details

Completion of task.

Ss fully address the information provided, but do not provide additional details

Partial completion of task. Ss complete no more than 60% of the information requested

Minimal completion of task. Ss complete less than 40% of the information requested.

Writing Task Holistic Rubric

Level of

Discourse

Sentences are fully developed and interconnected with conjunctions (e.g.

AND, BUT, or

BECAUSE )

Sentences are fully developed. Cohesive devices, however, are sporadically used

Sentences are somewhat complete.

Rare use of cohesive devices

Sentences are mostly incomplete. No use of cohesive devices

Vocabulary Grammar


Adequate and accurate use of vocabulary

Somewhat inadequate and/or inaccurate use of vocabulary

Inadequate and/or inaccurate use of vocabulary

Perfect control of the syntactic structures required (accuracy level 90% - 100%)

Adequate control of the syntactic structures. Some grammatical errors

(accuracy level

79% - 89%)

Emerging control of syntactic structures.

Several grammatical errors (accuracy level 61% - 78%)

Minimal control of syntactic structures.

Numerous grammatical errors

(accuracy level

0% - 60%)

42(45)

Conclusions & Action Plan

Generally speaking, data portray a uniform acquisition scenario among the foreign languages evaluated, namely, Chinese, French, German, Hebrew, Italian and Spanish. As Table 1 shows, all six language groups display similar percentages of students reaching or exceeding the proficiency levels set up for a Beginning I language course.

Table 1

Percentage of students reaching or exceeding the required proficiency levels for each language group

80

70

60

50

40

30

20

10

Chinese

French

German

Hebrew

Italian

Spanish

0

However, contrary to our predictions, no language group reported more than 75 percent of their students meeting the proficiency standards of the course. In fact, an average of less than 2/3 of our learners reached the desired proficiency levels.

In any case, a closer look at the data reveals an interesting acquisition phenomenon. Our L2 learners seem to encounter less difficulty in developing the required speaking and auditory skills than their reading or writing abilities. Despite their limited amount of instruction and exposure to the target language, a relatively high number of students (avg. 87%), are able to speak and converse at the expected level of a beginning course, irrespective of the language they are learning. Furthermore, their ability to comprehend oral messages seems to be equally developed.

In fact, the percentage of students meeting the standards of the course is 75 percent (see Table 2 below).

43(45)

Table 2

Percentage of students reaching or exceeding the required level of oral, auditory, reading and writing proficiency for each language group

100

90

80

70

60

50

40

30

20

10

0

Chinese French Hebrew German Italian Spanish

Oral

Auditory

Reading

Writing

The reading and writing skills, however, do not seem to follow a similarly expedite developmental pattern. Data indicate that an average of only 63 percent of the students write at the level required by the course. If we eliminate the high percentage displayed by the students of

Chinese (92%), the number reaches worrisome levels (55%). These results are quite interesting and intriguing. Normally, one would expect just opposite outcomes given that performance limitations of psychological nature (i.e. nervousness, anxiety) are usually more visible in generating oral than written language.

Such a clear discrepancy in students’ oral and written use of their target language (henceforth:

L2) calls for a re-analysis of the oral assessment tools used, mainly with regard to their reliability and usefulness in generating accurate data. This re-evaluation should help us eliminate or, at least reduce, the effects of external factors that might have altered the results obtained. The personal and oral nature of this assessment process, in fact, may be easily subject to human errors or bias. With that in mind, the department will organize some training sessions for the instructors administering the oral test in order to ensure that the assessment tool is used uniformly and appropriately.

In any case, the low percentage of students meeting the writing standards of the course is also reflected in the appropriate use of L2 syntactic structures. Data indicate that an average of 44 percent of the students tested were able to reach the accuracy standards required by the course.

And again, if we eliminate the percentage of the students of Chinese from our calculations, the percentile lowers to 38 percent. Interestingly, such a lower accuracy level is also encountered in

L2 oral production. Across the six language groups, the appropriate use of grammatical structures is one of the weakest oral abilities. That being the case, one could assume that internalizing L2 grammar rules is problematic. Such difficulties are displayed in both oral and written language modes.

44(45)

Fortunately, this does not seem to be case when students need to master the required vocabulary.

Lexical items do not appear to be acquisitionally as problematic as the grammar rules. As Table

3 indicates, the overall percentage of students using the required vocabulary is clearly higher

(62%), even though the Italian and Spanish learners are still showing some problems (33% and

43%, respectively)

Table 3

Percentage of students reaching or exceeding the required levels of lexical and syntactic proficiency for each language group

80

70

60

50

40

30

20

10

0

Chinese French Hebrew German Italian Spanish

Grammar

Voc.

In sum, students have shown to be struggling with the mastering of L2 grammar rules. Their behavior is quite consistent, equally involving the oral and written use of their target language.

The reasons justifying this lack of accuracy may be various and of different nature. The groups that have shown greater number of problems are those learning morphologically rich languages such as Italian, French Spanish and German. At a beginning level, the acquisition of grammar is mostly morphologically-based. It is well known that morphology is a linguistic component that is usually acquired very late.

However, the similarly lower number of students of Italian and Spanish reaching the desired level of lexical knowledge seems to highlight a general behavior of poor study skills and habits.

In fact, L2 learners appear to fall short whenever the skill requires a more attentive participation and thorough analysis of the language. With this in mind, the department will implement a greater variety of grammar activities and tasks in the curricula of the beginning language courses along with a closer monitoring of students' homework and lab assignments.

45(45)

QCC COURSE ASSESSMENT FORM QCC C A

QCC COURSE ASSESSMENT FORM

QCC COURSE ASSESSMENT FORM

QCC COURSE ASSESSMENT FORM

APPENDIX II

Writing Task Holistic Rubric

Related documents

Products

Support

QCC COURSE ASSESSMENT FORM QCC C A

QCC COURSE ASSESSMENT FORM

QCC COURSE ASSESSMENT FORM

QCC COURSE ASSESSMENT FORM

APPENDIX II

Writing Task Holistic Rubric

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib