To: Carolina Coronado, CIEP Director; and Mike Williams, Academic Support Coordinator From: Lauren Rein, Academic Support Specialist Date: January 14, 2015 CIEP Pass/Fail Rates Report for Academic Years 2011-­‐2012, 2012-­‐2013, and 2013-­‐2014 In EXCEL spreadsheets, passing and failing data was compiled for all three CIEP skills classes in three academic years. The results and discussion are below. Wide variety exists between years, with some possible causes. The differences between 2011-­‐2012 rates and 2013-­‐2014 may be attributed to a few factors such as the UNI Admissions requirement of the minimum paper TOEFL score of 375 (or its IELTS equivalent) being implemented in January 2013. The admittance and performance of the Brazil Scientific Mobility Program (BSMP) students could have also impacted 2013-­‐2014 rates as these students all placed in levels higher than Bridge and often passed their classes at higher rates. Another factor may be the implementation of attendance probation and revised academic probation policies in August of 2011. It’s unlikely that these policies had any impact on the 2011-­‐2012 rates, but as students failing probation were dismissed, successful students would have continued to advance in the program. This could be a factor that led to the increased passing rates in subsequent years. Another limitation of these rates is that tests and exams have been changing in the skill areas of Writing and Grammar to reflect revisions in student learning outcomes. This is a part of the CIEP’s ongoing systematic curriculum review and development based on research results of what UNI faculty require of their students attending academic classes. One source of evaluation of the CIEP curriculum stems from the data CIEP instructor Lauren Rein compiled and analyzed from UNI faculty syllabi and interviews about what they require of their academic students. Review and development of the Reading curriculum is approaching completion. CIEP faculty research in this area has also included selection of readings based on the readability scores of the Flesch Reading Ease and Flesch-Kincaid Grade Level Formula. Revisions are being conducted on standardized midterm and final exams and in chapter tests, quizzes, outside readings, and other assignments. These are now being piloted in several levels of Reading courses. Review and development of the Listening and Speaking curriculum is to begin in 2015. Currently, standardized exams are being reviewed to see that they follow current student learning outcomes. Development will follow the 2012 data and extend to new developments in undergraduate studies. Some assessment tools reflecting past student learning outcomes or no student learning outcomes have been created on some instructors’ own initiatives with the intention to try and improve upon existing assessments. However, these were often administered without any review body or procedure in place. These may have also affected some tests’ reliability and contributed to some variety of pass/fail trends here. Pass/Fail Rates for Listening & Speaking Classes 2011-­‐2012 Levels F P Total % Failing % Passing 1 5 21 26 19.2% 80.8% 2 4 36 40 10.0% 90.0% 3 12 62 74 16.2% 83.8% 4 19 76 95 20.0% 80.0% 5 24 85 109 22.0% 78.0% 6 38 69 107 35.5% 64.5% 7 22 42 64 34.4% 65.6% Total 124 391 515 24.1% 75.9% 2012-­‐2013 Levels F P Total % Failing % Passing 1 7 13 20 35.0% 65.0% 2 5 20 25 20.0% 80.0% 3 16 35 51 31.4% 68.6% 4 10 50 60 16.7% 83.3% 5 20 50 70 28.6% 71.4% 6 26 54 80 32.5% 67.5% 7 28 39 67 41.8% 58.2% Total 112 261 373 30.0% 70.0% 2013-­‐2014 Levels F P Total % Failing % Passing 1 16 19 35 45.7% 54.3% 2 4 28 32 12.5% 87.5% 3 4 36 40 10.0% 90.0% 4 6 58 64 9.4% 90.6% 5 20 80 100 20.0% 80.0% 6 13 82 95 13.7% 86.3% 7 19 53 72 26.4% 73.6% Total 82 356 438 18.7% 81.3% LS Pass/Fail Rates 2011-­‐2012 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 4 Fail 5 6 7 Pass L/S Pass/Fail Rates 2012-­‐2013 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 Fail 4 5 6 7 Pass 100.0% L/S Pass/Fail Rates 2013-­‐2014 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 1 2 3 4 5 6 7 Fail Pass Listening/Speaking Discussion: In the 2011-­‐2012 year, passing rates peaked at level 2 then gradually decreased. In 2012-­‐2013, there were two passing peaks in levels 2 and 4, then decreasing passing rates as the levels progressed. In 2013-­‐2014, Listening & Speaking passing rates stayed at 80% and above for levels 2-­‐6, which does not correspond to similar levels in Writing and Reading. As a result, some students would progress further in Listening & Speaking than in the other skill areas. By the time such students reached LS 7, when more reading and writing complement lectures and critical thinking is required, lower passing rates were recorded. All three years show decreasing Listening & Speaking passing rates starting in levels 4 or 5, when the curriculum begins to focus on note-­‐taking skills and academic caliber content. It is possible many students aren’t prepared for this transition in the curriculum or the standard of the content. As the Listening & Speaking Committee begins reviewing the current curriculum and student learning outcomes revisions in that skill area revisions are expected to improve this transition. Pass/Fail Rates for Writing Classes Wrieng Pass/Fail Rates 2011-­‐2012 2011-­‐2012 Levels F 1 5 2 8 3 21 4 54 5 27 6 28 7 14 Total 157 2012-­‐2013 Levels F 1 6 2 9 3 15 4 29 5 28 6 32 7 15 Total 134 P Total 22 27 41 49 64 85 84 138 96 123 74 102 88 102 469 626 % Failing % Passing 18.5% 81.5% 16.3% 83.7% 24.7% 75.3% 39.1% 60.9% 22.0% 78.0% 27.5% 72.5% 13.7% 86.3% 25.1% 74.9% 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 4 Fail P Total 14 20 21 30 48 63 59 88 53 81 49 81 83 98 327 461 % Failing % Passing 30.0% 70.0% 30.0% 70.0% 23.8% 76.2% 33.0% 67.0% 34.6% 65.4% 39.5% 60.5% 15.3% 84.7% 29.1% 70.9% 5 6 7 Pass Wrieng Pass/Fail Rates 2012-­‐2013 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 Fail 4 5 Pass 6 7 2013-­‐2014 Levels F P Total % Failing % Passing Wrieng Pass/Fail Rates 2013-­‐2014 1 14 19 33 42.4% 57.6% 100.0% 2 16 24 40 40.0% 60.0% 3 9 44 53 17.0% 83.0% 80.0% 4 31 86 117 26.5% 73.5% 60.0% 5 36 86 122 29.5% 70.5% 6 16 69 85 18.8% 81.2% 40.0% 7 8 87 95 8.4% 91.6% 20.0% Total 130 415 545 23.9% 76.1% 0.0% 1 2 3 4 5 6 7 Fail Pass Writing Discussion: The pass/fail rates for Writing indicate diminishing success for students placed in Bridge Level classes. During the academic year 2011-­‐2012, 81.5 percent of Bridge students passed. This continued to decline with 70% passing during 2012-­‐2013, and 57.6% during 2013-­‐2014, a marked decrease. Bridge writing has evolved to a limited extent over the last three years with more focus on grammar and sentence-­‐
level production. This may have affected Bridge student success rates, especially those students who entered the class with little or no previous knowledge or experience in written English. Only limited self-­‐
reporting of Bridge students’ previous reading and writing experiences in their L1s have been available. To address this decline in Bridge success, the CIEP began adding three hours a week of proctored on-­‐line tutorial (Elevate-­‐Reading Horizons) linking English reading literacy with listening and pronunciation of English. In addition, a committee of instructors is investigating how the CIEP might better “bridge” this gap between Bridge and Level 2 courses in all skill areas Writing 7 had 86.3% passing rate in 2011-­‐2012 and 84.7% in 2012-­‐2013 (fairly steady), then an increase to 91.6% in 2013-­‐2014. The curriculum in Writing 7 has incorporated more critical thinking in two of the three assignments in the course. In addition to a short APA-­‐style research paper, students prepare for this by writing two summary-­‐reflection pieces of academic articles. This was a direct result of the 2012 research into academic faculty syllabi and interviews. Introducing more reflective pieces and critical thinking were two of the many factors influencing the changing Writing curriculum. The weighted percentages of various assignments in Writing 7 also changed in Fall 2012 from the previous curriculum as a result of the Writing curriculum review. This might have influenced the pass-­‐fail rates. The three writing assignments and a Midterm Exam now account for 85% of a student’s grade in the course. Prior to Fall 2012, a writing lab component carried 40 percent of the grade with the three writing assignments only carrying 54% of the grade. There was no Midterm Exam in Writing 7 until Fall 2012. There are somewhat similar rates between 2011-­‐2012 and 2012-­‐2013 in levels 3 and 7, but the other levels don’t show close similarities. Both years show a decreased passing rate between 3 and 4 and 5 and 6, indicating levels 4 and 6 were more difficult. Beginning in the fall of 2013, revised writing curriculum began to require students to produce increasingly complex grammar in their writing including some grammar not addressed in the previous CIEP Writing curriculum. Students also begin to learn to cite credible sources in their supporting sentences for their writing topics in Levels 5 and 6. The Focus on Grammar and Focus on Writing series of new textbooks was introduced for levels 2-­‐6 accompanying the curriculum changes. A new rubric system to evaluate writing samples was also initiated based on the 4.0 system of the university. This rubric has undergone revisions as instructors began using it and finding areas that needed realignment with the weighted importance of student learning outcomes. Norming sessions have also been conducted with instructors to improve interpretation and application of the rubric with students’ writing samples. Pass/Fail rates showed a market difference in Writing 2 from the 2011-­‐2012 data with nearly 23% more students failing the course during 2013-­‐2014. One reason could be attributed to the revised curriculum which now requires students in Writing 2 to begin writing unified paragraphs the last four weeks of the course. Previously, Level 2 students only produced collections of unified sentences. More focus on vocabulary, grammar, and correct spelling in Level 2 also may have contributed to this decline. In addition, Bridge students entering Level 2 are only beginning to write at the sentence level. During academic year 2013-­‐2014, 8% more students succeeded in Writing 3 than 2011-­‐2012. This success comes with added requirements such as more variety of paragraphs required and additional grammar to learn. However, Writing 3 during 2013-­‐2014 also began to review some parts of Writing 2 which could have contributed to a higher success rate than those students experienced in Writing 2 (60%). Students faced higher demands in Writing 2 coming from Bridge level. There is another dip between levels 3 and 4; nearly 10% more students succeeded at Writing 3 during 2013-­‐2014 compared to Writing 4. This could be due to the increased challenges of learning and successfully applying such grammar points as present perfect or gerunds and infinitives. Also in 2013-­‐2014, the dip in passing at the higher levels shifts from Writing 6 to Writing 5 from 2011-­‐
2012; students in 2013-­‐2014 appear to be doing better in level 6 by comparison. As essays, clause-­‐level grammar, past perfect verb tense are introduced in Writing 5, this may contribute to the drop in passing. More students may need more time to develop and use clauses and grammar appropriately and within the requirement of the 4/5-­‐paragraph writing style. Pass/Fail Rates for Reading Classes 2011-­‐2012 % % Levels F P Total Failing Passing 1 5 21 26 19.2% 80.8% 2 11 38 49 22.4% 77.6% 3 33 57 90 36.7% 63.3% 4 79 71 150 52.7% 47.3% 5 38 74 112 33.9% 66.1% 6 24 61 85 28.2% 71.8% 7 12 78 90 13.3% 86.7% Total 202 400 602 33.6% 66.4% 2012-­‐2013 % % Levels F P Total Failing Passing 1 7 14 21 33.3% 66.7% 2 11 27 38 28.9% 71.1% 3 7 43 50 14.0% 86.0% 4 37 54 91 40.7% 59.3% 5 37 51 88 42.0% 58.0% 6 26 48 74 35.1% 64.9% 7 16 76 92 17.4% 82.6% Total 141 313 454 31.1% 68.9% 2013-­‐2014 % % Levels F P Total Failing Passing 1 16 20 36 44.4% 55.6% 2 10 26 36 27.8% 72.2% 3 14 30 44 31.8% 68.2% 4 20 55 75 26.7% 73.3% 5 46 80 126 36.5% 63.5% 6 25 71 96 26.0% 74.0% 7 9 93 102 8.8% 91.2% Total 140 375 515 27.2% 72.8% Reading Pass/Fail Rates 2011-­‐2012 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 4 Fail 5 6 7 Pass Reading Pass/Fail Rates 2012-­‐2013 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 4 Fail 5 6 7 Pass Reading Pass/Fail Rates 2013-­‐2014 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 Fail 4 5 Pass 6 7 Reading Discussion: Reading overall shows no consistent trend in the pass/fail rates over the three years at the lower levels (B-­‐3) with one exception: Bridge (a consistent and significant decline in student success). Reading 2 students experienced a slight decline in passing rates over the three years. Reading 3 pass/fail rates demonstrated a dramatic success rate in 2012-­‐2013 at 86%, up from 63% the previous year. During the 2013-­‐2014 period, that success rate plummeted to 68%. Reading curriculum had not changed over the three-­‐year period, However, a new online reading program, Reading Safari was introduced in Fall of 2012, which may partly explain the rise in passing rates in Level 3 in 2012-­‐2013; this program did not continue, so that may explain the regression in 2013-­‐2014. A new program, Reading Horizons-­‐Elevate is now being used in Bridge-­‐Level 2 which incorporates interactive English reading, listening and pronunciation. Data from Reading 4-­‐7 each showed a decline during the second year as well, then increases in success ranging from 5% to 14% the third year. More advanced readability statistics that were recently discovered by the Reading Committee indicated a serious Reading level gap between Bridge and Level 2. Passing rates at Bridge levels are decreasing, from 80.8% in 2011-­‐2012, 66.7% in 2012-­‐2013, and 55.6% in 2013-­‐2014. Students who fail the comprehensive Bridge exit test must retake all Bridge classes. This could indicate that the incoming students are not well prepared to move on to level 2. Perhaps some factors contributing to this decline in Bridge success may be a lack of previous English language instruction, a low level of reading comprehension in students’ L1, lack of sufficient student study skills or time spent studying outside of class, and a need to revise and extend time in the Bridge curriculum. Minor changes in the Bridge Exit test probably don’t account for this decrease, although inconsistent evaluation practices on the part of the Bridge instructor might also have been a factor in the past. The 2011-­‐2012 and 2012-­‐2013 periods (except for dramatic increase in Level 3) show a clear trend of decreasing passing rates in Reading Bridge to Level 4, then an increase through the upper levels. Both 2011-­‐2012 and 2012-­‐2013 show a serious drop in passing rates in level 5, though not in 2013-­‐2014; this improvement could be attributed to the addition of Brazilian students in Spring 2013 who passed at higher rates than previous populations. Academic Year Pass/Fail Rates Across Skills It’s helpful to also look at data separated by year as well as by skill level. Throughout the three academic years analyzed, Listening/Speaking pass/fail rates follow a generally opposite trajectory of decreasing passing towards the higher levels, while Reading and Writing follow a general trend of failure at the lower levels but higher passing rates at the higher levels. This strongly indicates that the Listening/Speaking curriculum needs to be revised at the intermediate levels so that student learning outcomes scaffold and lead to better support of student learning outcomes at the higher levels. 2011-­‐2012 LS Pass/Fail Rates 2011-­‐2012 Wrieng Pass/Fail Rates 2011-­‐2012 100.0% 100.0% 80.0% 80.0% 60.0% 60.0% 40.0% 40.0% 20.0% 20.0% 0.0% 0.0% 1 2 3 Fail 4 5 Pass 6 7 1 2 3 Fail 4 5 6 Pass Reading Pass/Fail Rates 2011-­‐2012 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 1 2 3 4 5 6 7 Fail Pass Discussion: Reading and Writing levels follow a similar “U-­‐shape” trajectory of higher passing rates at beginning and upper levels. The more rigorous academic skills and requirements of mid-­‐level courses in Reading and Writing may influence this curve. Students who succeed in these levels may perform better in the more advanced levels. Those who do not eventually pass in the mid-­‐levels may have left the program. Listening/Speaking has a more convex pattern of lower passing rates at beginning and upper levels. 7 2012-­‐2013 Wrieng Pass/Fail Rates 2012-­‐2013 L/S Pass/Fail Rates 2012-­‐2013 100.0% 100.0% 80.0% 80.0% 60.0% 60.0% 40.0% 40.0% 20.0% 20.0% 0.0% 1 2 3 4 Fail 5 6 0.0% 7 1 Pass 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 2 3 Fail 4 3 Fail Reading Pass/Fail Rates 2012-­‐2013 1 2 5 Pass 6 7 4 5 6 Pass Discussion: Reading and Writing levels follow a similar pattern here as well; an increase in passing at level 3, then a drop in levels 4 and 5, then another increase through level 7. The passing rates in Listening/Speaking are almost opposite, with a drop in passing rates in level 3, an increase in 4, then a gradual decrease through level 7.
7 2013-­‐2014 Wrieng Pass/Fail Rates 2013-­‐2014 L/S Pass/Fail Rates 2013-­‐2014 100.0% 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 1 2 3 4 Fail 5 6 7 2 3 Fail Pass 4 5 6 Pass Reading Pass/Fail Rates 2013-­‐2014 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% 1 2 3 Fail 4 5 Pass 6 7 Discussion: Reading and Writing levels this year no longer show similar trends to the previous Reading and Writing years’ data. Again, part of this may be due to the higher level BSMP participation in the program and the introduction of the new Writing curriculum in Fall 2013. However, both Reading and Writing still show noticeable increases in passing rates from levels 5 and higher. Early and inexperienced use of the new Writing rubric may have resulted in higher scores for student writing samples affecting the writing data. Since then, norming sessions of its use and revisions of the rubric to better reflect student learning outcomes have been conducted. Listening/Speaking rates for 2013-­‐2014 are also the highest of the three years, which is in marked contrast to the passing rates of the other classes in this year. As the Listening/Speaking Curriculum is reviewed, revised, and introduced and the new Reading curriculum has been fully implemented this is hoped to better measure student achievement of student learning outcomes. Currently, comparisons of student achievement across levels indicate variations. For 7 example, student success in Listening/Speaking 2 was 87.5% during this year, in Writing 2, 60%, and Reading 2, 72.2%. Level Group meetings of instructors began last year to review the curriculum across levels, especially student learning outcomes, syllabi, textbooks, standardized Midterm and Final Exams, diagnostics, and other curriculum material. They have provided valuable feedback on areas to adjust to better meet students’ learning needs and for the academic support coordinator to organize their implementation.