QCC COURSE ASSESSMENT Date: September 3, 2013 Department: Foreign Languages and Literatures Course: LS 221 Curriculum or Curricula: Liberal Arts PART I. STUDENT LEARNING OBJECTIVES TABLE 1. EDUCATIONAL CONTEXT LS-221 is the first course of the Heritage Spanish sequence of foreign language study. This course partially fulfills the foreign language requirements for the Associate Degree in Liberal Arts and Sciences. It fulfills the language requirements for the Associate Degree in Visual and Performing Arts. It may also fulfill the Liberal Arts and Science and the Humanities elective requirements for all other degree programs TABLE 2. CURRICULAR OBJECTIVES Curricular objectives addressed by this course: N/A TABLE 3. GENERAL EDUCATION OBJECTIVES Gen Ed General educational objectives addressed by this course: Select from objective’ preceding list. (1) Write, read, listen and speak clearly and effectively s ID (2) Use analytical reasoning skills number (3) Use information management skills effectively for academic research and from list lifelong learning (1-10) (4) Integrate knowledge and skills across disciplines (5) Differentiate and make informed decisions about issues based on cultural and political value systems. (6) Use personal and collaborative skills TABLE 4: COURSE OBJECTIVES AND STUDENT LEARNING OUTCOMES Course objectives Learning outcomes By the end of this course: 1) Students will possess the necessary skills to read comfortably in Spanish either for personal, academic or professional reasons. 2) Students will be able to implement the steps of the writing process (brainstorming, outlining, preparing a draft, revising and self-editing) when writing reaction papers in Spanish. 3) Students will be able to apply new vocabulary to their oral interactions and written expression. 4) Students will use analytical, inferential and evaluative skills to sustain informed oral and written discussions on current political, social and cultural aspects affecting the lives of Hispanics and Latinos. 5) Students will have an increased awareness of the vast range of Spanish varieties used throughout the Spanish-speaking world. 6) Students will have an increased awareness of formal and informal oral and written registers. 7) Students will have increased their confidence in their bilingual language strengths. By the end of this course, students will be able to: 1) identify the gender of words; use definite and indefinite articles according to gender 2) identify cognates in Spanish and provide the formal terms for them 3) identify the subject pronoun in a sentence; use formal and informal Spanish pronouns appropriately together with their corresponding verbal forms 4) use and upper and lower case letters in Spanish appropriately 5) use homonyms appropriately according to context 6) use present, preterit and imperfect tenses appropriately (regular and irregular forms) 7) read, analyze and interpret short Spanish texts dealing with cultural, social and political aspects of the Hispanic world 8) write two reaction papers (1-page long) related to the class topics. PART II. ASSIGNMENT DESIGN: ALIGNING OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS TABLE 5: OBJECTIVES ADDRESSED IN ASSESSMENT ASSIGNMENT Course Objective(s) selected for assessment: (select from Table 4) 1) Students will possess the necessary skills to read comfortably in Spanish either for personal, academic or professional reasons 2) Students will be able to implement the steps of the writing process (brainstorming, outlining, preparing a draft, revising and self-editing) when writing reaction papers in Spanish. 3) Students will be able to apply new vocabulary to their written expression. 4) Students will have an increased awareness of formal and informal written registers. Curricular Objective(s) selected for assessment: (select from Table 2) N/A General Education Objective(s) addressed in this assessment: (select from Table 3) 1) Write, read, listen and speak clearly and effectively TABLE 6: ASSIGNMENT, OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS The assessment tool is a comprehensive final exam that addresses the following course objectives and skills: 1. reading comprehension of a text related to the topics discussed in class (Latinos in the United States, language and identity, Latin American culture and society, etc.). 2. writing about one of the topics discussed in class, 3. grammar and orthography activities, 4. vocabulary activities. (See Appendix 1 for a copy of the final exam) Desired student learning outcomes for the assignment Students will… 1) possess the necessary skills to read comfortably in Spanish either for personal, academic or professional reasons (Gen. Ed. Obj. 1 and 2) Briefly describe the What assessment tools will be range of activities used to measure how well students student will engage in for have met each learning outcome? this assignment. 1) Reading comprehension activities 1) Students answer questions about a biographical text provided to them in the exam. 2) be able to implement the steps of the writing process (brainstorming, outlining, preparing a draft, revising and selfediting) when writing reaction papers in Spanish (Gen. Ed. 1) 2) Writing activities 2) Students write responses to questions about one of the class topics (see grading rubrics in Appendix 2). 3) be able to apply new vocabulary to their oral interactions and written expression (Gen. Ed. Obj. 1) 3) Vocabulary activities 3) Students explained some of the vocabulary learned in the course in the context of sentences extracted from the readings. 3) have an increased awareness of formal and informal oral and written registers (Gen. Ed. Obj. 1) 4) Grammar, vocabulary and orthography activities 4) Students answer questions related to the use of definite articles, uppercase letters, the use of written accents, spelling, identification of subject pronouns, the present tense, preterit and imperfect tenses. Students answer vocabulary questions. PART III. ASSESSMENT STANDARDS (RUBRICS) TABLE 7: ASSESSMENT STANDARDS (RUBRICS) The assessment tool is a comprehensive final exam that addresses the following course objectives and skills: 1. reading comprehension of a text related to the topics discussed in class (Latinos in the United States, language and identity, Latin American culture and society, etc.). 2. writing about one of the topics discussed in class, 3. grammar and orthography activities, 4. vocabulary activities. (See Appendix 1 for a copy of the final exam) Desired student Assessment measures Standards for student performance: learning outcomes for each learning from the outcome: assignment: Students will… 1) possess the necessary skills to read comfortably in Spanish either for personal, academic or professional reasons (Gen. Ed. Obj. 1 and 2) 1) Students answer questions about a biographical text provided to them in the exam. 1) Students’ reading comprehension ability will be determined by calculating their accuracy rates in their answers to reading comprehension questions. 75% of the students are expected to meet the course standard. 2) be able to implement the steps of the writing process (brainstorming, outlining, preparing a draft, revising and self-editing) when writing reaction papers in Spanish (Gen. Ed. 1) 2) Students write responses to questions about one of the class topics (see grading rubrics in Appendix 2). 2) The parameters used to measure students’ writing abilities in their essay will be determined by a written rubric (See Appendix 2). 75% of the students are expected to meet the course standard. 3) be able to apply new vocabulary to their oral interactions and written expression (Gen. Ed. Obj. 1) 2) Students answer a series of questions about the vocabulary learned during the course. 2) Students’ vocabulary knowledge will be determined by calculating their accuracy rates in their answers to the vocabulary questions. 75% of the students are expected to meet the course standard. 4) have an increased awareness of formal and informal oral and written registers (Gen. Ed. Obj. 1) 4) Students answer questions related to the use of definite articles, uppercase letters, the use of written accents, spelling, identification of subject pronouns, the present tense, preterit and imperfect tenses. Students answer vocabulary questions. 4) Students’ grammar and ortography knowledge will be determined by calculating their accuracy rates in providing the correct forms or structures. 75% of the students are expected to meet the course standard. Students’ vocabulary knowledge will be determined by calculating their accuracy rates in their answers to the vocabulary questions. 75% of the students are expected to meet the course standard. PART IV. ASSESSMENT RESULTS TABLE 8: SUMMARY OF ASSESSMENT RESULTS Desired student learning outcomes: Students will… 1) possess the necessary skills to read comfortably in Spanish either for personal, academic or professional reasons (Gen. Ed. Obj. 1) 2) be able to implement the steps of the writing process (brainstorming, outlining, preparing a draft, revising and self-editing) when writing reaction papers in Spanish (Gen. Ed. 1) 3) be able to apply new vocabulary to their oral interactions and written expression (Gen. Ed. Obj. 1) 4) have an increased awareness of formal and informal oral and written registers (Gen. Ed. Obj. 1) Student achievement: See Table 9. TABLE 9. EVALUATION AND RESULTING ACTION PLAN A. Analysis and interpretation of assessment results: 1. Written Final Examination The data below correspond to the performance of 47 students in the final exam, which was designed and used as the main assessment tool. The assessment included three categories: vocabulary, grammar and orthography knowledge, reading comprehension, and writing. The results are as follows: Table 1 Vocabulary, grammar and orthography Range Average Score # of Students Reading comprehension Writing Total 0-50 37.44 0-10 9.18 0-20 15.1 47 47 47 0-80 61.78 (77.15%) 47 The total points a student could obtain (adding all three categories) was 80. The Assessment Committee had defined the following performance scale: 0-48 points = student performance does not meet expectations; 49–58.5 points = student performance almost meets expectations; 59-71.2 points = student performance meets expectations; 71.2-80 = student performance exceeds expectations. The students’ average score in this assessment (61.78%) is within the range of 59-71.2 (“student performance meets expectations”). This average score is equivalent to 77.15% in a scale of 100. Table 2 Range 0f 47 Students Percentage Percentage of Students in Each Proficiency Level Does not meet Almost meets Meets expectations expectations expectations 0-48 49-58.5 59-71.2 Exceeds expectations 71.5-80 4 11 21 11 9% 23% 45% 23% The overall student performance shows that a total of 68% of students met or exceeded the expectations of this assessment process. The results by percentage of students in each category are as follows: Table 3 Range # 0f 47 Students Percentage VOCABULARY, GRAMMAR AND ORTOGRAPHY. Student performance … Does not meet Almost meets Meets Exceeds expectations expectations expectations expectations 0-29.5 29.5-36.4 36.5-44.5 44.5-50 4 16 22 5 8% 34% 47% 11% Table 4 Range # 0f 47 Students Percentage READING. Student performance … Does not meet Almost meets Meets expectations expectations expectations 0-6 6,1-7,3 7,4-8,9 Exceeds expectations 9-10 3 4 3 37 6% 9% 6% 79% Table 6 Range WRITING. Student performance … Does not meet Almost meets Meets expectations expectations expectations 0-12 12,25-14,5 14,75-17,75 Exceeds expectations 18-20 # 0f 47 Students Percentage 11 23% 7 15% 10 21% 19 41% B. Evaluation of the assessment process: The percentage of students who meet or exceed expectations in vocabulary, grammar and orthography is 58%. The percentage of students who meet or exceed expectations in reading is 85%. The percentage of students who meet or exceed expectations in writing is 62%. When looking at the assessment results in relation to the Course Objectives and the General Education objectives, we observe the following: Course Objective 1: Students will possess the necessary skills to read comfortably in Spanish either for personal, academic or professional reasons (Gen. Ed. Obj. 1). The assessment tool measured students’ Spanish reading skills by asking literal questions about the reading provided. 85% of students met or exceeded expectations in this assessment area –a percentage above the 75% that the assessment committee considered acceptable. Course Objective 2: Students will be able to implement the steps of the writing process (brainstorming, outlining, preparing a draft, revising and self-editing) when writing reaction papers in Spanish (Gen. Ed. 1). The assessment tool measured students’ Spanish writing skills by asking them questions related to a class reading (see rubrics for evaluating this task in Appendix 2). The performance level in this area (62%) is close to the performance level in the grammar, orthography and vocabulary tasks (58%). Both percentages are below the expected 75%, which suggests that students did not acquire the formal aspects of Spanish writing (grammar, spelling) nor the more conceptual ones (adequate vocabulary and elaborate ideas) at the level expected. Course Objectives 3 and 4: Students will be able to apply new vocabulary to their oral interactions and written expression (Gen. Ed. Obj. 1); students will have an increased awareness of formal and informal oral and written registers (Gen. Ed. Obj. 1) The assessment tool measured students’ acquisition of new vocabulary and awareness of formal and informal oral and written registers by: a) asking grammar and spelling questions, and b) asking vocabulary questions. At 58%, the percentage of students who meet or exceed expectations in the vocabulary, grammar and orthography sections of the assessment is below the expected 75% defined by the assessment committee. C. Resulting action plan: An in-depth examination of the assessment tool and results suggests the following: 1. 68% of the students met or exceeded the objectives, a result close to the 75% expected. The averages of each individual category show that all the areas assessed need to improve except for reading. 2. Although the course syllabus lists several objectives that contain higher-level thinking skills (analytical, inferential and evaluative), the assessment tool does not focus on any of them. Rather, it measures reading by asking literal comprehension questions. The department should consider whether the course is indeed addressing higher-thinking skills. If not, the department must decide whether those skills should continue to be part of the course objectives. If the course is addressing those skills, then one or two of them should be incorporated into the next assessment. 3. Although the course syllabus lists a writing objective that emphasized knowledge of the steps of the writing process, the assessment tool does not measure this knowledge adequately, since it only asks memory questions about one of the course readings. As in the case of the higher thinking skills, the department should consider whether the course is indeed teaching the steps of the writing process. If not, the department must decide whether those skills should continue to be part of the course objectives. 4. The instructors preparing the assessment should pay attention to, and revise if necessary, the progressive level of difficulty in the sequence of assessment tools. The LS 221 reading and writing parts, for instance, appear to be disproportionately easier than the corresponding tasks in LS 222 and 223. Incorporating higher thinking skills to the assessment of LS 221 should improve the continuity among the three levels of heritage classes. 5. It is possible that the reading part of the LS 221 assessment was not challenging enough, which might account for the difference in scores between this part and the rest of the sections. The level of difficulty among the different parts within the assessment tool should be even. Although the assessment committee will meet in the fall of 2013 for follow-up discussions of the assessment tool and the results, a preliminary review of the assessment process suggests the following: 1. The assessment should be repeated after the course objectives are revised and aligned with those of 222 and 223. 2. The assessment tool needs to be revised for a more homogeneous difficulty level among assessment sections and among course levels. 3. Although it is possible that the use of an imperfect assessment tool has distorted the assessment results, the assessment committee should meet with the instructors of this course and discuss ways to improve the writing skills of Spanish heritage speakers. Some ideas include: provide more guided writing time and activities for heritage students, conduct analyses of good writing samples with heritage students, invite students to share writing pieces with their peers, edit and publish a volume with students’ writing, coordinate and train tutors at the Student Learning Center to help with Spanish heritage students’ writing, begin conversations with English composition instructors to: a) compare assessment tools and results, and b) share pedagogical strategies to improve students’ writing.