Date Received Fall 2005 Semester Assessment Report Form Directions: Please complete a form for each of the programs within your department. This form was designed to provide a format for assessment reporting and should not be used to limit the amount of information provided. Each box that is attached to each of the sections is designed to adjust to varying lengths. If you have any questions, please contact Dr. Bea Babbitt at x51506 or via email at: email@example.com. 1. Program Information: Program Italian Studies Department Foreign Languages College Liberal Arts Program Giuseppe Natale Assessment Associate Professor of Italian Coordinator Semester Data Fall 2005 Collected Report Giuseppe Natale Submitted by Associate Professor of Italian Phone/email firstname.lastname@example.org Date Submitted July21, 2006 2. According to the Assessment Plan for this program, what were the planned assessments to be conducted during the 2005-2006 Academic Year? You may want to copy and paste from this program’s assessment plan. Which outcomes for this program were measured? How did you measure the outcomes? What results did you expect? If the students performed well what would their performance look like, i.e. percentages, means, or comparisons to a national standard? __1__outcome out of a total of _1__ outcomes evaluated this semester. Students completing Embedded questions Students scoring 60% (D-) or higher an Italian Studies testing proficiency are proficient at level A1 for Minor should be able were given in quizzes, ITAL113; at level A2 for ITAL 114; to write grammatically midterm, and final at level B1 for ITAL 213. Proficiency and lexically accurate examinations in: ITAL levels are defined in accordance to the Italian at the 113/114/213 guidelines set by the Common intermediate level. (Elementary Italian I European Framework for Languages / They should be able to and II, and Quadro di Riferimento Europeo delle formulate written Intermediate Italian I). Lingue (see attached file) sentences in Italian in Scores were broken a grammatically down for single accurate manner so exercises in order to that it is not a strain ascertain the for native speakers to understand them; should have a sufficient range of language to express ideas on general topics, to narrate events, and to write on literary and cultural topics. development of specific skills: reading, writing, translating, and comprehension. 3. Results, conclusions, and discoveries. What are the results of the planned assessments listed above? What conclusions or discoveries were made from these results? Describe below or attach to the form. Results, conclusions, and discoveries The proficiency guidelines contained in the Common European Framework (CEF) aim at determining the language competence of learners and users against a background of general competences, i.e. competence given by the sum of academic and empirical knowledge. These guidelines reflect the latest functional approaches to foreign language instruction, which call for learning to use the language in different situations. Through its detailed break-down of verbal operations, the CEF guidelines did provide Italian faculty and instructors with means to not only measure stages of learning but also to reflect on current teaching practices and expectations. Before evaluation, the assumption was that students of Italian could attain: Basic User proficiency level after two semesters (Level A1 for students completing ITAL113, Level A2 students completing ITAL114) Independent User level after four semesters (Level B1 for students completing ITAL 213 and Level B2 for students completing 214) Proficient User level after six semesters (for students completing ITAL 301 and ITAL 302). For an accurate assessment, proficiency had to be determined for each level. In Fall 2005 data collection addressed first- and second-year Italian (third-year Italian will be addressed in Spring 2006). Prong was based on sample quizzes, a midterm, and all the final exams for ITAL 113 (5 sections), ITAL114 (2 sections), and ITAL213 (1 section). Analysis was based on a total of 87 exercises and a percentage was calculated for each question. Language competence was then broken down into three main components: a) understanding (oral and written); b) writing (grammar and translation); c) coherence (language in context) Data analysis of tests and exams produced the following results: Score averages ITAL 113 ITAL 114 ITAL 213 79.1 85.3 72.9 Students scoring above 60% ITAL 113 ITAL 114 ITAL 213 80.1 85.7 88.3 A (Understanding) ITAL 113 ITAL 114 ITAL 213 86.6 87.0 95.6 B (Writing) ITAL 113 ITAL 114 ITAL 213 76.3 84.1 68.3 C (Coherence) ITAL 113 ITAL 114 ITAL 213 77.4 83.3 59.5 (data on single exercises and questions is in the attached file) Preliminary analysis Usually language students do increasingly better as they move on to higher level courses. As expected, students of Italian II fared better than those of Italian I, but unexpectedly the lowest scores came from ITAL 213. On the positive side, data overall revealed excellent levels in listening and reading comprehension. In general, understanding proficiency was higher than writing proficiency. This was an expected result given the instructors’ strong emphasis on TL instruction and communication in the classroom. Nonetheless, data confirmed the existence of specific problems in grammar acquisition (prepositions, pronouns, and tense conjugations). These are typical problem areas for English-speaking students of Italian, and therefore lower percentages were not unexpected. Data also showed low proficiency levels for translation and for all those exercises that called for a functional use of language—with lows at 50% or more. This result can be expected in a communicative classroom. Preliminary conclusions By breaking down language competence into separate components it was possible to see how uneven the proficiency levels were. Written production was not on a par with comprehension; knowledge of lexicon and grammar did not always produce accuracy in writing or translating. Also, the formulation of sentences based on a given situational context (language in context) appears to be a problematic area. Most students scored at or above the required proficiency mark for levels A1, A2, and B1, which would indicate that their language competence is at the required level. However, data based on individual exercises and specific tasks seem to indicate that their use of language, tested against a general competence framework, does not always reach the expected level. The data collected reinforced the opinion, already shared by Italian Faculty and instructors, that too much material is being covered in Italian I and II, with not enough time for students to assimilate and master what they memorize. The main source of these problems can be traced to the recent changes in scheduling. With instruction down from three to two days a week, covering the first-year textbook in two semesters has become problematic. To do so, instructors have to sacrifice conversation and other classroom practices aimed at building practical proficiency. Moreover, data also shows a negative rippling effect on the performance of second-year students. While the outcome for exercises on reading and listening comprehension was very pleasing, scores for those exercises requiring extensive language competence were much lower, especially in ITAL 213. 4. Use of Results. What program changes are indicated? How will they be implemented? If none, describe why changes were not needed. Assessment results and prior direct observation indicated the need to restructure the curriculum for first and second year Italian, which entails: Spreading the content of the first year textbook (18 chapters) from two to three semesters (Italian I-II-III). This change, besides benefiting didactically the students, will also bring the Italian program back in line with the other language programs in the Department. Adopting the revised edition of the first-year textbook, featuring a more streamlined, function-driven grammar and vocabulary. Adopting new textbooks for ITAL 213-4. Currently, ITAL 213 has an advanced textbook, entirely in TL, similar to what is used in the 4th semester in other languages. Increasing writing practice and contextualized exercises aimed at improving language in use. As a result of these changes, a new cycle of assessment for first- and second-year Italian will be conducted starting Fall 2006. 5. Dissemination of results, conclusions, and discoveries. How and with whom were the results shared? Data for this Fall 2005 assessment report was assembled by Italian instructors and analyzed by the Italian Studies Coordinator. Results will be made available to FOL faculty and to the College of Liberal Arts Dean’s office.