B.S. in Family and Consumer Sciences Education Assessment in the Major Report By Dr. Diane Klemme, Program Director 2010 Submitted September, 2011 Table of Contents Overview of the Assessment System .......................................................................................................................................................................1 Overview of the Program .........................................................................................................................................................................................1 PRAXIS I: Pre-Professional Skills Test .................................................................................................................................................................2 PRAXIS II: Content Test Summary .......................................................................................................................................................................4 Benchmark Interview Ratings..................................................................................................................................................................................5 Student Teaching Performance Ratings ...................................................................................................................................................................9 Educational Benchmarking Inventory (EBI) .........................................................................................................................................................10 Alumni Follow-Up Survey ....................................................................................................................................................................................12 Communicating Assessment Data with Constituencies .........................................................................................................................................13 Utilization of Assessment Data to Improve Courses and the Program ..................................................................................................................13 Overview of the Assessment System The University of Wisconsin-Stout School of Education (SOE) has gathered assessment data from fall semester 2003 through December 2010. In the School of Education, data are gathered from several sources to inform unit and program decisions. Data in this report are used to develop program goals, inform curriculum changes, and enhance course delivery in order to improve teacher education candidate learning. This report contains data from the PRAXIS I: Pre-Professional Skills Test, PRAXIS II: Content Test, Benchmark Interviews, Student Teacher Performances, the Educational Benchmarking Inventory (EBI), and Alumni Follow-Up Surveys. This report also describes how assessment data are used to set programmatic goals, improve the program, program curriculum, and delivery of courses. Overview of the Program In 2010, the Family & Consumer Education program consisted of 57 undergraduate students, 3 male and 54 female. See Appendix A for more detailed information. FCSE AIM Report 2010 Page 1 PRAXIS I: Pre-Professional Skills Test The PRAXIS I: Pre-Professional Skills Test (PPST) is required for teacher certification by the Wisconsin Department of Public Instruction. To be accepted into the Teacher Education Program at UW-Stout (Benchmark I), students; typically in their first and second year at UW-Stout, must pass all three tests: reading writing and mathematics. The PPST is administered through the Educational Testing Services (ETS). The test is administered as a traditional hand bubble sheet test or by computer at designated sites. Due to a database conversion in 2009, Datatel to Peoplesoft, we are just now able to start generating data by program. In addition, we are able to disaggregate each test to report data on pen/paper (P) vs. computerized (C) tests. There was a year lapse, 2009, when we were unable to generate any data. Note that the pass rates in the table reflect attempts by all candidates prior to being accepted into the School of Education. Since all are required to pass the PPST to be admitted to the School of Education as part of Benchmark I, the actual pass rate is 100%. Students in the FCSE program percent pass rate exceeded the SOE average on Computerized Math, Computerize and Paper writing and computerized reading. FCSE students combined (computer and paper) rates were: Math 83.5 %, Writing 62%, Reading 55%. The writing and reading pass rates areas for improvement. PPST Attempts and Pass Rates Teacher Education Program FCSE undergraduate TOTALS 2006 # test attempts 16 19 18 2006 # (and %) passed 11 = 69% 7 = 37% 10 = 56% 2007 # test attempts 10 14 12 2007 # (and %) passed 10 14 12 2008 # test attempts 6 10 6 2008 # (and %) passed 5 = 83% 9 = 90% 5 = 83% Math 204 148 = 72.5% 226 191 = 84.5% 130 102 = 78.5% Reading 280 145 = 51.8% 243 184 = 75.7% 150 119 = 79.3% Writing 296 161 = 54.4% 257 200 = 77.8% 138 104 = 75.4% PPST Test Math Reading Writing FCSE AIM Report 2010 Page 2 Teacher Education Program 2010 PPST Test # test # (and %) attempts passed C-Math 8 8 = 100% P-Math 6 4 = 67% B-Math 14 12=86% C-Writing 14 9 = 64% P-Writing 5 3 = 60% FCSE B-Writing 19 12=63% C-Reading 13 7 = 54% P-Reading 7 4 = 57% B-Writing 20 11=55% C-Math 118 93 = 79% P-Math 80 57 = 71% B-Math 198 150=76% C-Writing 116 92 = 79% P-Writing 97 49 = 51% SOE B-Writing 213 141=66% C-Reading 149 88 = 59% P-Reading 94 50 = 53% B-Reading 243 138=57% C= Computerized; P= Pen & Paper Tests; B=Both Computerized and Pen & Paper Tests FCSE AIM Report 2010 Page 3 PRAXIS II: Content Test Summary The PRAXIS II FCSE content exam was revised and the new exam was implemented September 2009. Therefore, old test data is no longer relevant due to the test changing so drastically. Note that all candidates are required to pass the Praxis II to be admitted to student teaching as part of Benchmark II so the pass rate is 100% upon Benchmark II approval. I’m unable to address the 09/10 data because the n is low – no data was reported Content Test from ETS (0121) Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 08/09 18 197 151 166.5 159-171 159 15/18 83% 09/10 4 NA NA NA NA 159 3/4 75% Average Percent Correct (as compared to State and National results) Due to the low number of completers (n=4) in 2009-2010, no score reports are sent from ETS. FCSE Test Category (0121) Points Availa ble Stout 08/09 % State 08/09 % National 08/09 % The Family 15-18 79 78 75 Human Dev. Management 19 17 70 64 71 73 67 72 Nutrition / Food 17 71 78 73 Clothing / Textiles 7 71 68 71 Housing 10 62 68 70 22-25 72 73 69 7-9 60 70 68 FCS Educ. Career, Community FCSE AIM Report 2010 Page 4 Benchmark Interview Ratings Benchmark I Interview Results Family & Consumer Sciences Question Explain personal and professional growth between your initial resume and updated resume. Explain your philosophy of education. Explain three personal characteristics that will make you an effective teacher. Describe yourself as a learner and how that will impact your future teaching. Describe experiences that have impacted your understanding of diversity and human relations and how these might aid you as you work with students and families Explain two subject matter/content artifacts and how these examples illustrate your understanding of the content you will be teaching Completed Alignment Summary Response Unsatisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory 2008 N=14 0% 100% 0% 100% 0% 100% 0% 100% FCSE 2009 N=6 0% 100% 0% 100% 0% 100% 0% 100% 2010 N=6 0% 100% 0% 100% 0% 100% 0% 100% Unsatisfactory 0% 0% 0% 100% 100% 100% Unsatisfactory 0% 0% 0% Satisfactory Unsatisfactory Satisfactory 100% 0% 100% 100% 0% 100% 100% 0% 100% Satisfactory SOE 2010 N=80 1% 99% 0% 100% 0% 100% 1% 99% 0% 100% 1% 99% 0% 100% FCSE students continue to pass the Benchmark one. FCSE AIM Report 2010 Page 5 Benchmark II Interview Results Family & Consumer Education FCSE 2008 2009 Question Response N=14 N=11 Unsatisfactory 0% 0% Emerging 43% 45% Describe your Philosophy of Education and how it has evolved Basic 57% 55% n/a 0% 0% Unsatisfactory 0% 0% Emerging 21% 31% Describe what it means to be a "Reflective Practitioner" Basic 79% 69% n/a 0% 0% Unsatisfactory 0% 0% Emerging 14% 9% Describe the WI Teacher Standard and Domain you feel most competent in Basic 86% 91% n/a 0% 0% Unsatisfactory 0% 0% Describe the WI Teacher Standard and Emerging 36% 36% Domain you have experienced the greatest Basic 64% 64% growth n/a 0% 0% NA NA Provide Portfolio evidence (signed copy of the Unsatisfactory NA NA Instructional Technology Utilization rubric) of Emerging your competence in current instructional Basic NA NA technology n/a NA NA Reviewers choose 2 of the following; discuss portfolio evidence that: Unsatisfactory 0% 0% Emerging 14% 9% demonstrates your content knowledge Basic 43% 9% n/a 43% 82% Unsatisfactory 0% 0% demonstrates your ability to create instructional opportunities adapted to diverse Emerging 7% 9% learners Basic 29% 18% FCSE AIM Report 2010 2010 N=4 0% 50% 50% 0% 25% 0% 75% 0% 0% 75% 25% 0% 0% 75% 25% 0% 0 100% 0% 0% SOE 2010 N=80 1% 41% 58% 0% 1% 34% 64% 0% 0% 31% 69% 0% 0% 32% 68% 0% 0% 52% 48% 0% 0% 50% 50% 0% 0% 100% 0% 2% 27% 71% 0% 3% 40% 57% Page 6 demonstrates your ability to teach effectively demonstrates your ability to assess student learning n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a 64% 0% 0% 7% 93% 0% 0% 7% 93% 73% 0% 0% 18% 82% 0% 27% 9% 64% 0% 0% 100% 0% 0% 0% 50% 50% 0% 0% 4% 25% 71% 0% 2% 41% 57% 0% This is the first time the FCSE program had a student that was unsuccessful on a question. Follow-up will occur with this student. FCSE AIM Report 2010 Page 7 Benchmark III Interview Results Family & Consumer Sciences Question Artifacts from student teaching, reflection ratings Final Student Teaching Assessments and Recommendations from Cooperating Teachers Disposition ratings from student teaching from cooperating & University Supervisors Instructional Technology Utilization Rubric Alignment Summary of artifacts meeting all 10 Wisconsin Teaching Standards & 4 Domains/ Components & reflections/ reflection ratings Response Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a 2008 N=12 0% 0% 42% 58% 0% 0% 0% 17% 83% 0% 0% 0% 17% 83% 0% NA NA NA NA NA 0% 0% 17% 83% 0% FCSE SOE 2009 2010 2010 N=13 N=5 N=138 0% 0% 0% 0% 0% 0% 23% 80% 24% 77% 20% 76% 0% 0% 0% 0% 0% 1% 8% 0% 1% 38% 60% 20% 54% 40% 78% 0% 0% 0% 0% 0% 0% 0% 0% 1% 0% 20% 20% 54% 80% 76% 46% 0% 3% NA 0% 0% NA 0% 1% NA 40% 19% NA 60% 77% NA 0% 3% 0% 0% 0% 38% 20% 14% 0% 0% 8% 62% 80% 75% 0% 0% 3% FCSE students are rated basic or proficient on all items with the exception of the alignment summary – have some emerging rating on this. FCSE AIM Report 2010 Page 8 Student Teaching Performance Ratings Student Teacher Evaluations Family & Consumer Education Rating Scale: 1=Unsatisfactory, 2=Emerging, 3=Basic, 4=Proficient FCSE SOE 2008 2009 2010 2010 N=10 N=13 N=6 N=120 Mean Mean Mean Mean Teachers know the subjects they are teaching 3.90 3.66 3.70 3.78 Teachers know how children grow 3.86 3.73 3.90 3.82 Teachers understand that children learn differently 3.86 3.69 3.50 3.73 Teachers know how to teach 3.62 3.59 3.80 3.84 Teachers know how to manage a classroom 3.46 3.45 3.30 3.65 Teachers communicate well 3.67 3.62 3.60 3.78 Teachers are able to plan different kinds of lessons 3.67 3.77 3.60 3.77 Teachers know how to test for student progress 3.83 3.68 3.40 3.75 Teachers are able to evaluate themselves 3.73 3.74 3.70 3.78 Teachers are connected with other teachers and the community 3.71 3.46 3.40 3.70 Teachers make effective use of instructional technologies to enhance student learning. NA NA 4.00 3.91 FCSE students were rated basic (3) or above; 2 items succeed the SOE means. FCSE AIM Report 2010 Page 9 Educational Benchmarking Inventory (EBI) There were no responses to the 2010/2011 survey. Therefore, nothing was reported from EBI. N=6 FCSE AIM Report 2010 N=17 N=5 N=10/11 Page 10 EBI - Institution Specific Questions Mean Data; Scale (1-Not at all, 4-Moderately, 7-Extremely) FCSE SOE 09/10 10/11 10/11 N=10 N=0 N=87 To what degree were you prepared to create meaningful learning experiences for students based on your content knowledge? To what degree were you prepared to provide instruction that fosters student learning and intellectual, social and personal development? To what degree were you prepared to create instructional experiences adapted for students who learn differently? To what degree were you prepared to use a variety of learning strategies including the use of technology to encourage critical thinking and problem solving? To what degree were you prepared to manage classroom behavior and create a learning environment that encourages positive social interaction, active engagement in learning and self-motivation? To what degree were you prepared to use instructional technology and media to foster active inquiry, collaboration and interaction in the classroom? To what degree were you prepared to plan instruction based on knowledge of subject matter, students, the community and curriculum goals? To what degree were you prepared to use formal and informal assessment strategies to evaluate student progress? To what degree were you prepared to reflect on teaching and evaluate the effects of choices and actions on pupils, parents and others? To what degree were you prepared to foster relationships with colleges, families and the community to support student learning and well-being? 5.20 na 5.48 5.10 na 5.37 4.90 na 5.48 5.70 na 5.51 4.40 na 5.08 5.60 na 5.21 5.10 na 5.43 5.20 na 5.14 5.10 na 5.47 5.10 na 5.38 *We updated our questions beginning in the 2009-2010 school year FCSE AIM Report 2010 Page 11 Alumni Follow-Up Survey The Alumni follow-up surveys are at one-year and five-year intervals. UW-Stout surveys graduates every two years. The next survey will be sent in 2012 for graduates in 2010 and 2006. The sample numbers are typically not large and the employer data typically includes very few responses. Response rates for the five-year (2004) included two individuals, the one- year (2008) has 7 respondents and the employer survey had no responses for the 2008 and 1 for 2004. Due to the low numbers I cannot draw any conclusions from the employer interviews and alumni are very small samples. The two alumni from 2004 have full time positions, one in the field and one in an unrelated field. Both rated their experiences as acceptable and if they had to do it over, they would attend UW- Stout. Of the seven 2008 graduates, five are employed in a directly related field and six have full time positions. Six of the respondent identified that they would “definitely yes” attend Stout if they had to do it over again. The other respondent indicated a “4” in attending Stout. Four would definitely return to the same program, 2 respondents indicated a “4” and 1 indicated a “3”. Eighty –six percent of the respondents rated the overall effectiveness of the program/major as high or very high. The executive summary and full report from the Alumni Follow-Up Study are online. It is located at the following site: http://www.uwstout.edu/static/bpa/ir/afu/2010index.html FCSE AIM Report 2010 Page 12 Communicating Assessment Data with Constituencies Data will be communicated to faculty members through informal and formal means. Program faculty meet during scheduled discipline area work group meetings (DAWG) designed to support ongoing program improvement. In addition, the AIM findings will be shared with the Program Advisory Committee and across program vested publics including technical content instructors. Action plans resulting in desired change will result from these advisory groups and stakeholders. Utilization of Assessment Data to Improve Courses and the Program The findings of the AIM process and report are analyzed and connected with specific program elements (courses, projects, assignments, experiences) that are seen as direct and indirect contributors to the current and future desired outcomes. 2009 Goals 1 Increase the pass rate of the Praxis II exams 2 Increase classroom management ratings 3 Promote the use of career services 4 Assist students with understanding and providing opportunities to succeed on the Technology Utilization Rubric FCSE AIM Report 2010 Action to be taken Evidence towards meeting goals or rationale for abandoning Held study sessions Revised 4 yr plan Curricular focus on classroom management in FCSE 341 Discuss services in all FCSE courses Explained rubric to students/sought ways to have them address these activities during student teaching Data not provided by Praxis due to low numbers – will continue with this goal Need to retain – rating on student teaching continue to be low No data to report – will continue with this goal 4 proficient ratings on student teaching – does not need to be a goal as students are able to meet this w/ student teaching Page 13 2010 Goals 1. Seek ways to improve PRAXIS reading and writing test scores 2. Increase response rate of student to return the EBI 3. Continue to focus on classroom management 4. Continue to promote career services 5. Begin program revision to incorporate new gen ed requirements and review current program 6. Continue recruitment efforts FCSE AIM Report 2010 Action to be taken Stress resources and services available to FCSE students as they prepare for test via classes, SAFCS club discussions and advisement Have upperclassmen talk about the exam in more detail at the advisement day meeting Send a reminder to student teachers to take the take survey. This will be sent out prior to their placement end date. I will address this issue earlier in the program. I have revamped the FCSE 201 classroom observation assignment and will include additional questions relating to this topic. Promote services in FCSE course by taking FCSE 101 class to career services office for presentation Seek input from advisory council meetings Fall/ Spring Submit recruitment plan to Dean by Sept. 30th, 2011 Page 14 Family & Consumer Sciences Education, B.S. Minority enrollment Male Female Total enrollment SCH Student FTE New Freshmen Transfers Number of graduates by year: Number of male graduates Number of female graduates Number of minority graduates Number employed in related major: Number continuing education: Number employed in major: Percent employed: One-Year Rates in Program One-Year Retention Rates - Any Program Six-Year Graduation Rates in Program Six-Year Graduation Rates Any Program Average High School Percentile Average ACT Composite of New Freshmen Average Cumulative GPA Freshmen: 1-29.5 credits Sophomore: 30-59.5 credits Junior: 60-89.5 credits Senior: 90 or more credits Honors Program (FA10) Learning Comm. Partic. Study Abroad Students % of grads who participated in Experiential Learning Salary Average Salary Low Salary High I would attend UW-Stout again I would enroll in the same academic program Three-Year Show Rates - New Freshmen Three-Year Show Rates - New Transfers 10-11 2 4 46 50 710 47 8 4 NA NA NA NA 09-10 08-09 3 3 54 57 823 55 10 7 12 12 - 1 3 42 45 659 44 6 2 16 1 15 8 1 100.0% 66.7% 83.3% 60.0% 70.0% 07-08 2 2 43 45 667 44 2 7 19 19 3 1 4 100.0% 50.0% 50.0% 06-07 04-'05 2004 50.0% 50.0% 45.4% 20.1 NA 10 11 13 16 2 2 100% NA NA NA 48.0% 19.2 2.81 15 13 9 20 NA NA NA 64.0% 23.0 3.31 9 5 7 24 $ $ $ 74.0% 20.5 3.03 5 5 15 20 2003 2002 2001 2000 2008 Grads 26.7% 26.7% 41.7% 58.3% 14.3% 85.7% 28.6% 42.9% 53.5% 21.0 2.70 5 14 11 30 40,000 35,000 47,000 4.86 4.43 56% 76% 2004 Grads 2 2 58 60 906 60 4 2 11 1 10 16 100.0% 0.0% 50.0% 4.50 3.00 Family & Consumer Sciences Education, B.S. FR High School Percentile Rank 10-11 Enrollment Demographics Enrollment New FR and Transfer New FR Enrollment Transfer Enrollment 45.4% 09-10 08-09 50 64.0% 07-08 74.0% 06-07 60 57 48.0% 45 Total Enrollment 45 7 7 10 53.5% FR ACT Avg. Composite score 10-11 4 20.1 3 09-10 19.2 Male 08-09 8 3 2 2 23.0 6 20.5 21.0 58 54 46 42 Female 43 4 10-11 2.81 06-07 09-10 10-11 06-07 07-08 08-09 09-10 10-11 2.70 2 07-08 3.03 2 1 Minority Enrollment 08-09 07-08 06-07 2 3.31 09-10 08-09 2 2 3 10-11 09-10 2 06-07 FR Avg GPA 07-08 06-07 4 08-09 07-08 Family & Consumer Sciences Education, B.S. 2 Employment Numbers - Male graduates 8 1 - 1 - Minority graduates 06-07 - 12 15 19 - - - - One Year Retention Rates Retention Rates in Program Retention Rates Any Program Experiential Learning Percent Employed 10-11 09-10 08-09 07-08 06-07 0.0% 10-11 09-10 08-09 07-08 06-07 0.0% 10 - 1 07-08 - Female graduates 09-10 - - 10-11 - 1 - 3 - 11 06-07 4 19 07-08 - Total graduates by year 08-09 - 08-09 Number continuing education - 09-10 Number employed in related major 16 10-11 Number employed in major Graduates in Program 16 12 Six Year Graduation Rates Graduation Rates In Program Graduation Rates - Any Program 60.0% 66.7% 2004 50.0% 50.0% 42.9% 0.0% 70.0% 2003 26.7% 85.7% 83.3% 50.0% 50.0% Employment Percentages 10-11 09-10 08-09 100% NA 100% 2002 07-08 06-07 100% 100% 2001 2000 For more information on retention/graduation rates go to: http://www2.uwstout.edu/content/bpa/ir/retention/indexstu.html 41.7% 14.3% 58.3% 26.7% 28.6% 50.0% Family & Consumer Sciences Education, B.S. 3 Other Three- Year Show Rates 10-11 10-11 76% 2 Salary Data Salary High 08-09 $47,000 2 Salary Average 08-09 Salary Low 08-09 $40,000 56% $35,000 SCH 710 906 823 659 667 08-09 07-08 Student Credit Hours 10-11 09-10 06-07 Student FTE 47 Honors Learning Study Program Comm. Abroad (FA10) Partic. Students 60 55 44 44 08-09 07-08 FTE Three-Year Three-Year Show Rates - Show Rates New New Freshmen Transfers 10-11 09-10 06-07