Implementing a “Modified” Portfolio Assessment Approach in ESL Writing Courses David Bruno Assistant Professor II & Chairperson ESL Department Camden County College Portfolio Background at CCC 1996: Academic Skills Writing Department pilots portfolios as final assessment in the last Writing course before ENG-101 2009 – Academic Skills Writing Department developed following “Position Statement on Writing” “One piece of writing—even if it is generated under the most desirable conditions—can never serve as an indicator of overall writing ability, particularly for high-stakes decisions. Ideally, writing ability must be assessed by more than one piece of writing, in more than one genre, written on different occasions, for different audiences, and responded to and evaluated by multiple readers as part of a substantial and sustained writing process.” 2013 – Academic Writing and ESL Department Co-Presented at NJCCC Best Practices Conference ESL Department Writing Course Assessment Timeline 2008 Department implements a holistically graded “timed essay” final exam for Level 5 writing course - similar to ENG-101 final assessment Prior to 2006 Individual instructors responsible for creating own assessment measures and final exams 2013 to Present Department implements “portfolio assessment” approach in all writing courses 2006 2010 Department creates standardized final exams for most ESL courses, including writing courses Department expands “timed essay” final exam to Level 4 writing course “Timed Essay” Final Exams Final exam administered 2 weeks prior to end of semester Graded holistically by ESL Department faculty on 1 – 6 scale by two graders Students must pass “ESL Departmental Exam” in order to pass the course. If passing, score counts as 30% of final grade (holistic score is converted) If failing, students automatically fail the course Students offered retest within 2 weeks. Matches current English Composition I final exam method in order to better prepare students Drawbacks to “ESL Departmental Exam” No ongoing ESL Department involvement in the learning process One essay was not enough to properly assess the students’ writing ability All Student Learning Outcomes not being assessed in the exam. Students who are not ready for the next level are still being passed in both lower levels and in courses with “Departmental Exam” Benefits of Portfolio Assessment Increased uniform approach across sections Increased departmental role in sections taught by adjuncts Students received a “realistic” assessment mid-semester in time to make adjustments Increased focus on sentence structure skills – one complaint of English Department Reduce the number of students who are passed on to the next level even though they are not ready Portfolio Assessment Procedure Full-time faculty “mentors” are partnered with 5+ adjunct faculty teaching a combined 10+ sections of writing. Mentors collect and review first-day handouts and 1st day writing samples for verification purposes. Partners communicate via email or in person about academic concerns, process questions, classroom management, essay topics, teaching tips, and grading. All partners have at least three face-to-face meetings per semester: review 1st day writing verifications, to review midterm portfolios and to review final portfolios. Mentors review portfolios at midterm and end of semesters and provide Academic Writing - Portfolio Contents Midterm: Cover letter Revised essay with a draft MyWritingLab (online software) printout Final: Cover letter Two revised essays (one must be peer edited and one teacher edited) open-topic reading-based Timed essay MyWritingLab printout ESL Department Portfolio Contents Beginning of Semester 1st Day Writing Verification (if completed) Midterm Portfolio Review Writing Assignment for Chapter A – All drafts (including final draft graded with departmental rubric) Writing Assignment for Chapter B – All drafts (including final draft graded with departmental rubric) Timed Writing Assignment 1 – graded with departmental rubric. Initial Reflection Homework Assignment (not graded) Graded Quizzes from Chapters A & B Final Portfolio Review All Midterm Portfolio documents Writing Assignment for Chapter C – All drafts (including final draft graded with departmental rubric) Writing Assignment for Chapter D – All drafts (including final draft graded with departmental rubric) Graded Quizzes from Chapters C and D Timed Writing Assignment 2 – graded with departmental rubric. Final Reflection Homework Assignment – not graded “Modified” Approach to Portfolio Assessment 1st Day Verification shows full spectrum of growth Standardized assignments/assessments All sections have the same required assignments to complete and quizzes/tests to administer All writing assignments are scored based on departmentally created rubric for each level Quizzes included to demonstrate skill ability Burden on department, not student Faculty maintains portfolio Incomplete portfolios based on students not completing assignments Copies of work provided for student review Required assignments show all work completed, not just best Factors into final grade breakdown 1st Day Verification Standardized 1st day tests to determine appropriate level Given in all courses (except Oral Communication*) Writing classes – writing prompt FT faculty mentor evaluates writing sample to determine level Students will be moved up levels – never down! ESL Writing V – Departmental Rubric: Spring 2015 Name: __________________________________ Assignment: ____________________________ Date: _________________________________ Content (15 points) The essay fits the assignment. The essay is interesting to read and fully developed. The essay matches on the prewriting & outlining. Total Standardized Rubrics Used Organization (35 Points) The essay has an introduction, body, and conclusion. The essay has a funnel or attention-getting introduction with the thesis statement at the end, and the thesis statement is well written. (it is not too general; it is not just a simple announcement; it avoids stating an obvious fact) Each body paragraph discusses only one main idea, and there are no sentences that are off topic. Each paragraph achieves coherence by repeating key nouns, using transition signals to show relationships among ideas, and using transitions to link paragraphs. The conclusion includes a summary of the main points or paraphrases the thesis statement. Total Grammar & Sentence Structure (40 points) Verb tenses are used correctly. (form and choice of tense) The sentences use the correct word forms. (nouns, pronouns, verbs, adjectives & adverbs, singular vs. plural) The sentences are all complete sentences (no fragments) and there are no fused sentences (run-ons, comma splices). The sentences demonstrate subject-verb agreement. There are a variety of sentence types: simple, compound, complex, and compound-complex. The sentences demonstrate English fluency including standard word order – generally subject-verb-object/complement. Word choice is accurate and descriptive. Phrasal verbs and prepositions are used correctly. Noun determiners (articles, much, many, some, etc.) are used correctly. Modals are used correctly (form & choice of modal). Maximum 5 5 5 15 8 7 10 5 5 35 5 5 8 4 3 4 3 3 3 2 Your Score ESL Department – Spring 2015 Midterm Portfolio Comment Sheet: Writing Name: ___________________________________ Portfolio Assessment Sheets Course Number: ESL-_____________ Writing Assignment #1 Comments: Writing Assignment #2 Comments: Well written! Acceptable Well written! Acceptable Major errors in paragraph/essay structure Major errors in sentence structure Major errors in grammar/mechanics Major errors in paragraph/essay structure Major errors in sentence structure Major errors in grammar/mechanics Timed Writing Assignment #1 Comments: Well written! Acceptable Quizzes Comments: Great job! OK Needs improvement Major errors in paragraph/essay structure Major errors in sentence structure Major errors in grammar/mechanics GENERAL MIDTERM ADVISEMENT (this is NOT included in final grade) This student’s portfolio is: Passing Comments (if needed): Failing Portfolio Reviews All assignments/assessments are scored by the instructor – reviewed by the mentors Midterm and Final Portfolios are due on specific dates Reviewed by FT Faculty Mentors Allows failing & borderline students sufficient time to take additional steps to improve in time for the final portfolio Midterm Portfolio is for “advisement purposes only” Final Portfolio scored as “Passing” or “Failing” If the student passes the Final Portfolio, the grades of the assignments included in the portfolio will count as 40% of the final grade. The 40% is broken down into the final categories as such: 10% for chapter writing assignments 8% for timed writing #1 12% for timed writing #2 10% for the chapter quizzes. All categories are averaged into the final grade separately. If the student fails the Final Portfolio, the student will receive 0 points for each of the above categories in the final grade. As a result, the student will automatically fail the course. Student AND/OR faculty as ability to challenge Final Grade Breakdown ESL Department has established a standardized “final grade breakdown” for all courses in order to build a uniform approach to assessment Students made aware on 1st day of class The final grade breakdown for Writing courses are: 10% - active in-class participation, homework and in-class work 8% - timed writing assignment 1 (included in portfolio assessment) 12% - timed writing assignment 2 (included in portfolio assessment) 10% - chapter writing assignments (included in portfolio assessment) 10% - quizzes (included in portfolio assessment) 20% - MyWritingLab work 30% - ESL Department exam Addendum included with portfolio instructions and grading Portfolio Challenge Procedure If a student/instructor would like to challenge the portfolio, he/she must follow these steps: The faculty member must approve the challenge request. If the faculty member agrees with the failing portfolio grading, the student will not be permitted to challenge and the failing grade will stand. If the faculty member agrees with the student that a challenge is warranted, another full-time faculty member (not the faculty member’s current mentor) will review the portfolio. If a challenge is still warranted, the student will be given the opportunity to complete a timed writing assignment on campus prior to the start of the semester. The ESL Department Chair will review the assignment and share it with the faculty member so that they can both make the decision together. Portfolio Assessment for Mentoring ADVANTAGES All faculty have one person to direct questions to for quick response and discussion. For new faculty, mentors help to adjust individual grading ideas with departmental standards. Midterm portfolios enable partners to review and discuss grading, comments & feedback on writing, format/organization/grammar issues that appear throughout a section, indicating the need for additional teaching or lessons. Exceptionally strong students can be identified for possible next level exemption. Adjuncts feel connected to the department and are informed about policies and changes in a personal manner. Student Learning Outcome Alignment with ESL-027 Upon completion of this course, the student will be able to: Demonstrate the ability to write multiparagraph essays using accurate, meaningful and appropriate sentences. Portfolios allow students to meet their SLOs: Data shows the students are more successful in College-level courses Revised work allows students who need time to revise show how they can use the writing process to reach their writing goals. Demonstrate knowledge of the writing process Demonstrate the ability to detect and analyze errors and use correct forms in writing Demonstrate the ability to recognize and produce various sentence types Demonstrate the ability to react in writing to a reading selection The mix of reading-based and timed essays allow students to showcase their best work on meeting a variety of these goals. Demonstrate the ability to produce grammatical structures relevant to ESL Writing V The quizzes/tests include grammar-based questions The timed essays show how students can edit on their own work independently. Revised essays allow more time to revision and editing. Passing Rate Per Course Before Portfolios: 2010 - 2012 Portfolios: 2013 - 2014 ESL-020: ESL Writing I ESL-020: ESL Writing I ESL-022: ESL Writing II ESL-022: ESL Writing II ESL-023: ESL Writing III ESL-023: ESL Writing III ESL-026: ESL Writing IV ESL-026: ESL Writing IV ESL-027: ESL Writing V ESL-027: ESL Writing V 74% successfully completed 73% successfully completed 77% successfully completed 80% successfully completed 82% successfully completed 64% successfully completed 63% successfully completed 65% successfully completed 65% successfully completed 68% successfully completed Students are no longer being passed unless ready for next level Passing Rate of ESL-027 Before Portfolios: 2010 - 2012 ESL-020: ESL Writing I 72% successfully completed ESL-027 ESL-022: ESL Writing II 73% successfully completed ESL-027 ESL-023: ESL Writing III 73% successfully completed ESL-027 ESL-026: ESL Writing IV 74% successfully completed ESL-027 Portfolios: 2013 - 2014 ESL-020: ESL Writing I No data – students currently taking ESL-027 ESL-022: ESL Writing II 83% successfully completed ESL-027 ESL-023: ESL Writing III 79% successfully completed ESL-027 ESL-026: ESL Writing IV 80% successfully completed ESL-027 Students are better prepared for higher level ESL courses Passing Rate of HIS-101 & PSY-101 Students who successfully completed ESL-026 from 2010 - 2012 Students who successfully completed ESL-026 from 2013 - 2014 75% successfully completed HIS-101 80% successfully completed HIS-101 68% successfully completed PSY-101 78% successfully completed PSY-101 Students are better prepared for College-level courses Portfolio Assessment - Results Pros Students are better prepared for higher level ESL courses Students are better prepared for College-level / Gen-Ed courses More uniform approach of assessment Full snapshot of ability to assess Increased adjunct preparedness Better departmental communication Better data to analyze Cons Enrollment Students losing financial aid or F-1 status Time/Effort to Assess Due Dates Data Analysis Difficulties More needed ENG-101 Institutional Support Key to implementation and sustainability FT faculty compensation (release time/stipends) Work goes above and beyond the normal workload Portfolio documents/requirements updated each semester Time/effort of adjunct faculty Meeting with mentors periodically Preparing portfolios for review Secretarial support Handling portfolio supplies Copying of necessary documents Portfolio Storage (3 years minimum at CCC) The Future of Portfolios at CCC Electronic Portfolios Online Courses Expand to Oral Communication courses for “oral portfolios” Digital capturing of assignments and presentations Attempted in 2013, but removed due to technical issues