A TMCC COURSE ASSESSMENT REPORT (CAR) Revised I0/21/2013 Course Prefix, Number and Title: ENG 102-Composition II Division/Unit: Liberal Arts/English Submitted by: Natalie Russell and Erika Bein Contributing Faculty: Assessment Reading Committee Academic Year: 2013-2014 (Spring 2014) Complete and electronically submit your assessment report to your Department Chair/Coordinator/Director. As needed, please attach supporting documents and/or a narrative description of the assessment activities in your course. Coune Outcomes In the boxes below, summarize the outcomes assessed in your course during the year. Outcome#l Standard Written English (SWE) • Employ the conventions of SWE, as evidenced by competency in format, grammar, mechanics, punctuation, and sentence structure. Outcome #2 Students will write a variety of essays that demonstrate progression in college-level writing skills. Assessment Measures Assessment Results Use of Results Effect on Coune In the boxes below, summarize the methods used to assess course outcomes during the last year. In the boxes below, summarize the results of your assessment activities during the last year. In the boxes below, summarize how you are or how you plan to use the results to improve student learning. Based on the results of this assessment, will you revise your outcomes? If so, please summarize how and why in the boxes below: We did not directly assess this outcome although it was indirectly evaluated through reading and scoring the assessment essays, which could not be completely understood without basic student competency in S WE. No direct findings. We do not have plans to change our instruction in this area at this time. This outcome was revised to be attached to the new General Education Leaming Outcomes of Communication, Critical Thinking, and Information Literacy. It was approved by the CAP Committee and the Vice President ofTMCC in I 0/2013. In this current assessment cycle, we evaluated the culminating English 102 assignment, the research paper. This assignment demonstrates the progression of reading and writing skill building through the English composition sequence. It is just one of several types of analytical academic essays that students write during this course. We found that remains one of the most important assignments because it demonstrates the writing and research abilities that are critical to gain in this course. However, we discovered that we need to provide additional training on assignment design and holistic reading/scoring so that we maintain consistency throughout the department. We will be providing more ongoing training to part-time faculty in particular, full-time faculty if requested, on assignment design. This outcome was revised to be attached to the new General Education Leaming Outcomes of Communication, Critical Thinking, and Information Literacy. It was approved by the CAP Committee and the Vice President of TMCC in I 0/2013 . Page I We will also be redesigning the assessment rubric so that it matches the assignment expectations and cannot be misinterpreted. A TMCC COURSE ASSESSMENT REPORT (CAR) Course Prefix, Number and Title: ENG 102-Composition II Division/Unit: Liberal Arts/English Submitted by: Natalie Russell and Erika Bein Contributing Faculty: Assessment Reading Committee Academic Year: 2013-2014 (Spring 2014) Course Outcomes Outcome #3 Students will synthesize critical reading and writing skills in the production of analytical essays that demonstrate synthesis of primary and secondary sources. Outcome #4 Students will synthesize academic research methodologies and college-level writing skills in the production of a research paper. Assessment Measures Assessment Results Use of Results Effect on Course We did not assess any other essay from this course except the research paper. This analysis is explained below. No specific findings in this area. In future assessments, we may choose to drill-down our analysis of research methodologies to study the synthesis of reading and writing and the use of particular types of sources. This outcome was revised to be attached to the new General Education Learning Outcomes of Communication, Critical Thinking, and Information Literacy. It was approved by the CAP Committee and the Vice President ofTMCC in 10/2013. In this assessment cycle, we focused primarily on the culminating research paper. We sampled 35 sections of English I 02, with S essays selected from each class. We studied three main core indicators: evidence of an argument-based thesis, evidence of effective source material, and demonstrated research skills. In each of the three core indicators, students demonstrated an average level of competency. Our numbers slipped slightly from the 20 I I assessment results, showing that diligent work is still needed in these areas. As indicated in the Assessment Report, here is an abbreviated list of our recommendations: I. Refine assessment rubric. 2. Focus instruction on thesis development and on the nature and incorporation of sources. 3. Keep adjunct faculty involved in the process and provide additional training. 4. Continue to collect assignment sheets and provide training in assignment design. Continue to focus on all areas of instruction that maintain the rigorous standards of this course. This outcome was revised to be attached to the new General Education Leaming Outcomes of Communication, Critical Thinking, and Information Literacy. It was approved by the CAP Committee and the Vice President ofTMCC in 10/2013. s. Page2 A TMCC COURSE ASSESSMENT REPORT (CAR) Course Prefix, Number and Title: ENG 102-Composition II Division/Unit: Liberal Arts/English Submitted by: Natalie Russell and Erika Bein Contributing Faculty: Assessment Reading Committee Academic Year: 2013-2014 (Spring 2014) Please enter your name and date below to confirm you have reviewed this report: Title Name Date Department Chair/Coordinator/Director Natalie A. Russell 6/13/201 4 Dean Armida Fruzzetti 6/ 13/2014 Vice President of Academic Affairs Jane Nichols ~~ Q f\,,J (j I Page 3 / ~ 6/25/2014 A TMCC COURSE ASSESSMENT REPORT (CAR) Course Prefix, Number and Title: ENG 102-Composition II Division/Unit: Liberal Arts/English Submitted by: Natalie Russell and Erika Bein Contributing Faculty: Assessment Reading Committee Academic Year: 2013-2014 (Spring 2014) English 102-Spring 2014 Assessment Report The Assessment: The previous English 102 assessment was completed in the spring 2011 semester. Results from this assessment can be found in the 2011 report and CAR fonn. For the 2014 English l 02 assessment process, the department used the same core indicators that had been used in the 2011 assessment, giving us a pattern of results to evaluate and discuss. We used the research paper in 102 as our assessment focus, pulling from 35 sections of English 102 from the fall 2013 semester, selecting 5 essays from each section. This assignment is a typical, required culminating assignment in 102, a thesis-driven research assignment incorporating appropriate source material and fonnatted in a proper (usually MLA) fonnat. New to this assessment cycle was the method of collection: instructors were asked to provide the corresponding assignment sheet along with their designated student essays identified by roster number (similar to our English 101 and 98R department assessments). The assessment committee used three core indicators for this assessment. • Evidence of an argument-based thesis; • Evidence of appropriate/effective use of source material within the essay; • Demonstrated research skills of at least 5 varied/scholarly research materials (as shown in the works cited pages). The assessment readers scored these on a five point scale (O=incompetent and 5==superior). A score of3 on the 5 point scale on the individual indicators was an acceptable passing score. In January of 2014, the 22 assessment readers, both full and part-time instructors, completed a norming session on six essays ranging in quality before the formal reading began. During the formal reading, each essay was read twice, and if discrepancies existed, (a variation of more than 2 points), then the essay was given a third read (37%). For the first time in this assessment process, we also encountered a small number (2%) of fourth reads. The Results: (Average results in all categories) Appropriate Thesis Raw Score 513.4 2011 Average 3.34 2014 Average 2.9 Page4 A TMCC COURSE ASSESSMENT REPORT (CAR) Course Prefix, Number and Title: ENG 102-Composition II Division/Unit: Liberal Arts/English Submitted by: Natalie Russell and Erika Bein Contributing Faculty: Assessment Reading Committee Academic Year: 2013-2014 (Spring 2014) Use of Appropriate Source Material 517.8 3.185 3.0 Evidence of Appropriate Research Materials 608.8 3.77 3.5 The scores for the spring assessment were significant in several ways. Although each of the areas still met the level of "adequate competence" (the thesis section marginally so), the table illustrates that our overall scores have fallen since 2011. There may be multiple reasons for this trend. Throughout both the nonning session and fonnal reading, we discovered that confusion existed in how terms like "thesis" and "research" were defined. Once we noticed ongoing large scoring gaps in these areas, we stopped pulling essays for fourth reads because we decided that it would be unproductive to continue without further evaluating the cause of the discrepancies. In addition, when we reviewed the provided assignment sheets, we also observed that problems existed in assignment design that could have led to the misinterpretation of the expectations of the assignment, therefore skewing assessment results. Even though the downward trend may be discouraging, the scores still indicate average competence ofour students, which is to be expected with an overall sampling of classes. These scores suggest that we still have work to do in these areas, which we will accomplish through the recommendations found below. Recommendations: l . Our primary task before the next assessment should be to refine the rubric. The department will be consulted on how to explain each item up for review and on which core indicators of the class we should focus. 2. Going forward, the English department needs to continue to focus ample class time teaching our students how to craft an effective argument-based thesis and incorporate and cite sources in text. These are skills our students must have leaving the research class, English 102. As mentioned above, though, some of these results could have been affected by a misinterpretation of the nature of the required sources (qualification and incorporation) to be used for this assignment, which should be improved by a reinvention of the rubric. Page 5 A TMCC COURSE ASSESSMENT REPORT (CAR) Course Prefix, Number and Title: ENG I 02-Composition II Division/Unit: Liberal Arts/English Submitted by: Natalie Russell and Erika Bein Contributing Faculty: Assessment Reading Committee Academic Year: 2013-2014 (Spring 2014) 3. The department also recommends keeping the adjunct faculty involved in the assessment process. The integration of full and part-time instructors helps generate a better dissemination of assessment data to all of the instructors and maintain consistent curriculum across classes. However, we discovered that additional training in course expectations and holistic reading/scoring techniques should be provided to those instructors who do not teach English l 02 but would like to be included in the assessment process. We must also streamline the payment process for part-time instructors in order to include them in a fair and ethical way. 4. We also suggest continuing to collect the research paper assignment as part of the required assessment materials from each instructor. Not all of them were received, and some of those collected demonstrated problems in assignment design. We should also provide ongoing professional development and mentoring for instructors to help maintain consistent expectations. 5. While we wish to emphasize the use of an argument-based thesis, appropriate use of scholarly source materials, and research abilities in classes, the department recommends that we continue to focus on all areas, so we can continue to maintain our course standards and work toward improvement in the next assessment cycle. Page 6