B.S. Technology Education Assessment in the Major Report By Dr. David Stricker, Program Director 2010 Submitted October 2011 Table of Contents Overview ..................................................................................................................................................................................................................2 Overview of the Program .........................................................................................................................................................................................2 PRAXIS I: Pre-Professional Skills Test .................................................................................................................................................................3 PRAXIS II: Content Test Summary .......................................................................................................................................................................5 Student Teaching Performance Ratings .................................................................................................................................................................12 Educational Benchmarking Inventory (EBI) .........................................................................................................................................................14 Alumni Follow-up Survey .....................................................................................................................................................................................17 Communicating Assessment Data with Constituencies .........................................................................................................................................23 Utilization of Assessment Data to Improve Courses and the Program ..................................................................................................................23 Technology Education AIM 2010 Page 1 Overview The University of Wisconsin-Stout School of Education (SOE) has gathered assessment data gathered from fall semester 2003 through December 2010. In the School of Education, data are gathered from several sources to inform unit and program decisions. Data in this report are used to develop program goals, inform curriculum changes, and enhance course delivery in order to improve teacher education candidate learning. This report contains data from the: PRAXIS I: Pre-Professional Skills Test PRAXIS II: Content Test Benchmark I, II, and III interviews Student Teaching Performance Ratings Educational Benchmarking Inventory (EBI) 2004 and 2008 Alumni Follow-up Survey This report also describes how assessment data are used to set programmatic goals, improve the program, program curriculum, and delivery of courses. Overview of the Program In 2010, the Technology Education program consisted of 141 undergraduate students, 132 male and 9 female. Over the past five years, the numbers in this program have consistently been decreasing. However, the one-year retention rates within Technology Education have been increasing, and are the highest one-year retention rates within the undergraduate programs in the School of Education. Technology Education AIM 2010 Page 2 PRAXIS I: Pre-Professional Skills Test The PRAXIS I: Pre-Professional Skills Test (PPST) is required for teacher certification by the Wisconsin Department of Public Instruction. Students are considered pre-Technology Education majors until they have passed the PPST. Educational Testing Service (ETS) administers the three tests (reading, writing and mathematics) in either a written format or via computer. Due to a database conversion in 2009, Datatel to Peoplesoft, we just now are able to start generating data by program. In addition, we are able to disaggregate each test to report data on pen/paper (P) vs. computerized (C) tests. There was a year lapse, 2009, when we were unable to generate any data. Note that the pass rates in the table reflect attempts by all candidates prior to being accepted into the School of Education. Since all are required to pass the PPST to be admitted to the School of Education as part of Benchmark I, the actual pass rate is 100%. Table 1. Technology Education Program PPST Attempts and Pass rates. Teacher Education Program TECED undergraduate TOTALS PPST Test Math Reading Writing Math Reading Writing 2006 # test attempts 47 72 92 204 280 296 Technology Education AIM 2010 2006 # (and %) passed 44 = 94% 43 = 60% 45 = 49% 148 = 73% 145 = 52% 161 = 54% 2007 # test attempts 39 38 49 226 243 257 2007 # (and %) passed 36 = 92% 33 = 87% 39 = 80% 191 = 85% 184 = 76% 200 = 78% 2008 # test attempts 21 25 27 130 150 138 2008 # (and %) passed 20 = 95% 21 = 84% 20 = 74% 102 = 79% 119 = 79% 104 = 75% Page 3 Teacher Education Program 2010 PPST Test # test # (and %) attempts passed C-Math 10 7 (70%) P-Math 3 3 (100%) B-Math 13 10 (77%) C-Writing 23 11 (48%) P-Writing 10 3 (30%) TECED B-Writing 33 14 (42%) C-Reading 17 6 (35%) P-Reading 7 1 (14%) B-Reading 24 7 (29%) C-Math 118 93 (79%) P-Math 80 57 (71%) B-Math 198 150 (76%) C-Writing 116 92 (55%) P-Writing 97 49 (51%) SOE B-Writing 213 141 (66%) C-Reading 149 88 (59%) P-Reading 94 50 (53%) B-Reading 243 138 (57%) C= Computerized; P= Pen & Paper Tests; B=Both Computerized and Pen & Paper Tests On average the Technology Education majors show consistent performance in the math portion of the PPST (see Table 1). However, there is a significant decrease in passing rates for the reading and writing portions of the PPST for TE students. More emphasis needs to be placed on encouraging students to enrolling in EDUC 010: PPST Writing Prep Laboratory. EDUC 010 was first offered in 2007 and is now a regularly scheduled course that is designed to help education majors pass the written portion of the PPST. In addition, ASPIRE, a student support program for first-generation college students, provides help with the PPST needs to be encouraged by TE faculty – with special emphasis placed on Advisement Day opportunities. Lastly, tutors have been hired by the School of Education for those students needing help beyond the online tutorials and practice exams. Incoming freshman have been and are currently advised to study for the PPST and seek remediation if their reading/writing skills are weak. Although these strategies have appeared to be effective as students in the Technology Education program pass the PPST at similar or higher rates than all students in education majors in past years, it appears specific effort on the part of faculty to specifically point out these resources is needed. Technology Education AIM 2010 Page 4 PRAXIS II: Content Test Summary Note that all candidates are required to pass the Praxis II to be admitted to student teaching as part of Benchmark II so the pass rate is 100% upon Benchmark II approval. Table 2: Praxis II median score and score ranges Content Test from ETS Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 05/06 06/07 07/08 08/09 09/10 56 750 580 655 630-680 590 55/56 98% 38 720 560 650 630-680 590 35/38 92% 39 720 550 670 630-700 590 37/39 95% 29 710 620 670 650-680 590 29/29 100% 30 730 590 670 640-690 590 30/30 100% Technology Education students’ passing rates (see Table 2) have been above 90% for the past six years with 100% passing in 2009 and 2010. Praxis II median scores and score ranges for Technology Education majors are compiled in Table 2. The data indicates that the rate of passing scores and average Praxis II scores remain relatively stable over time. If students do fail the Praxis II, program faculty are poised to guide them to resources and services that address their particular needs. Specifically, a tutor has been hired from the School of Education and content specific study materials were ordered for students to become better prepared for this exam last Spring (2011). Although specific data have not been collected about their effects on Praxis II scores, the continuing integration of Project Lead the Way curriculum into current course offerings combined with additional math and science requirements should contribute and are intended to increase the rigor of UW-Stout’s Technology Education Program. Technology Education AIM 2010 Page 5 Table 3: Average percent correct by year Wisconsin USA 09/10 09/10 09/10 80 81 78 78 68 73 71 70 70 76 78 81 79 78 74 76 77 80 81 80 78 73 74 76 82 83 82 81 72 UW-Stout 05/06 06/07 07/08 08/09 Tech Ed Test Category Points Available Pedagogy & Professional (T Ed) 33-36 82 76 82 Information & Communication Technology 23-24 73 68 Construct Tech 15-17 80 Manufacturing Tech Energy/Power/Transportati on Tech 20-24 22-24 As categories of the Praxis II are examined more closely (see Table 3), UW-Stout remain higher than the national average in all areas. Technology Education AIM 2010 Page 6 Benchmark Interviews The UW-Stout School of Education Assessment System is designed to review candidates’ progress at three intervals during the program. Students are considered pre-education majors until they have passed the Benchmark I review. In this instance, the review determines a student’s readiness to become a teacher candidate in the Technology Education program. Later, candidates are reviewed again during the Benchmark II interview to determine whether they are equipped to proceed to the student teaching portion of the program. Benchmark III is completed at the end of student teaching and before a candidate is recommended for licensure. This includes the presentation of a capstone “Best Practices” unit for Technology Education, artifacts from student teaching, and recommendations by cooperating teachers. Table 4: Benchmark 1 Interview Data Benchmark I Interview Results Technology Education Question Explain personal and professional growth between your initial resume and updated resume. Explain your philosophy of education. Explain three personal characteristics that will make you an effective teacher. Describe yourself as a learner and how that will impact your future teaching. Describe experiences that have impacted your understanding of diversity and human relations and how these might aid you as you work with students and families Explain two subject matter/content artifacts and how these examples illustrate your understanding of the content you will be teaching Completed Alignment Summary Response Unsatisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory 2008 N=23 0% 100% 0% 100% 0% 100% 0% 100% TECED 2009 N=22 5% 95% 0% 10% 0% 10% 0% 10% 2010 N=7 0% 100% 0% 100% 0% 100% 0% 100% Unsatisfactory 0% 5% 0% 100% 95% 100% Unsatisfactory 0% 5% 0% Satisfactory Unsatisfactory Satisfactory 100% 0% 100% 95% 5% 95% 100% 0% 100% Satisfactory SOE 2010 N=80 1% 99% 0% 100% 0% 100% 1% 99% 0% 100% 1% 99% 0% 100% Technology Education teacher candidates are overwhelmingly successful in moving from pre-education to BSTE program students (see Table 4). Technology Education AIM 2010 Page 7 Benchmark II In order to be considered for a Benchmark II review, Technology Education majors must have completed or be currently enrolled in a pre-student teaching field experience; have completed or be currently enrolled in core education courses and obtained at least a “C” in these same courses which includes: Education Psychology, Multiculturalism, Cross-Cultural Field Experience, Secondary Reading and Language Development, Inclusion, and Lab and Classroom Management. In addition, students must have a cumulative GPA of 2.75 at UW-Stout. Students are rated by two reviewers at an unsatisfactory, emerging, or basic level - with a rating of emerging or basic considered to be sufficient evidence for a student to be allowed to move forward in the program. Table 5: Benchmark II Interview Data Benchmark II Interview Results Technology Education Question Describe your Philosophy of Education and how it has evolved Describe what it means to be a "Reflective Practitioner" Describe the WI Teacher Standard and Domain you feel most competent in Describe the WI Teacher Standard and Domain you have experienced the greatest growth Provide Portfolio evidence (signed copy of the Instructional Technology Utilization rubric) of your competence in current instructional Technology Education AIM 2010 Response Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic 2008 N=32 0% 38% 62% 0% 0% 19% 81% 0% 0% 19% 81% 0% 0% 28% 72% 0% NA NA NA TECED SOE 2009 2010 2010 N=29 N=28 N=80 10% 4% 1% 45% 61% 41% 45% 36% 58% 0% 0% 0% 7% 0% 1% 34% 68% 34% 59% 32% 64% 0% 0% 0% 7% 0% 0% 24% 46% 31% 69% 54% 69% 0% 0% 0% 7% 0% 0% 28% 65% 32% 65% 36% 68% 0% 0% 0% NA 0% 0% NA 54% 52% NA 46% 48% Page 8 technology n/a NA NA Reviewers choose 2 of the following; discuss portfolio evidence that: Unsatisfactory 0% 3% Emerging 9% 21% demonstrates your content knowledge Basic 47% 52% n/a 44% 24% Unsatisfactory 0% 10% demonstrates your ability to create Emerging 19% 34% instructional opportunities adapted to diverse Basic 31% 31% learners n/a 50% 25% Unsatisfactory 0% 0% Emerging 6% 3% demonstrates your ability to teach effectively Basic 6% 10% n/a 88% 87% Unsatisfactory 0% 0% Emerging 9% 14% demonstrates your ability to assess student learning Basic 22% 31% n/a 69% 65% 0% 0% 7% 27% 67% 0% 4% 52% 43% 0% 0% 80% 20% 0% 0% 68% 32% 0% 2% 27% 71% 0% 3% 40% 57% 0% 4% 25% 71% 0% 2% 41% 57% 0% Table 5 highlights that the overwhelming majority of BSTE students passed Benchmark II successfully with either a rating of “emerging” or “basic.” However, in 2010, a small number of students lacked the capacity and dispositions of a candidate who would be successful in the program. These students were either steered toward a program more fitting to their strengths or encouraged to graduate without teacher certification. Technology Education AIM 2010 Page 9 Benchmark III Benchmark III is the culminating assessment for students in the Technology Education program. Benchmark III must be completed before a preservice teacher can be recommended for licensure. The assessment requires that all coursework is completed, that all program-specific requirements are met, a satisfactory student teaching assessment is achieved, and that a complete electronic portfolio receiving a basic or higher proficiency rating is submitted. In order to facilitate this, students complete a student teaching experience at both the middle school and high school level. Four portfolio artifacts (student teaching observations) with accompany reflections and evaluations of the reflections accompany evaluations from cooperating teachers. Student teachers also prepare and present a “Best Practices” unit that is evaluated on planning and preparation, knowledge of resources, use of instructional technologies, use of assessment systems, and reflection on instruction. These artifacts are combined with alignment summaries in the portfolio and disposition ratings from the cooperating teacher/university supervisor to demonstrate that the 10 Wisconsin teaching standards and four Danielson domains have been addressed. Table 6: Benchmark III Interview Results Benchmark III Interview Results Technology Education Question Artifacts from student teaching, reflection ratings Final Student Teaching Assessments and Recommendations from Cooperating Teachers Disposition ratings from student teaching from cooperating & University Supervisors Instructional Technology Utilization Rubric Technology Education AIM 2010 Response Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory 2008 N=28 0% 4% 32% 64% 0% 0% 0% 36% 64% 0% 0% 0% 36% 64% 0% NA TECED SOE 2009 2010 2010 N=37 N=25 N=138 0% 0% 0% 0% 0% 0% 32% 24% 24% 68% 76% 76% 0% 0% 0% 0% 0% 1% 3% 4% 1% 30% 20% 20% 62% 76% 78% 5% 0% 0% 0% 0% 0% 0% 0% 1% 32% 32% 20% 35% 68% 76% 43% 0% 3% NA 0% 0% Page 10 Alignment Summary of artifacts meeting all 10 Wisconsin Teaching Standards & 4 Domains/ Components & reflections/ reflection ratings Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a NA NA NA NA 0% 7% 21% 72% 0% NA NA NA NA 0% 15% 27% 68% 0% 0% 8% 88% 4% 0% 20% 8% 72% 0% 1% 19% 77% 3% 0% 14% 8% 75% 3% During 2010 only 1 TE student ranked at the emerging level on the Final Student Teaching Assessments. Five student teachers ranked at the emerging level for their alignment summaries (see Table 6). Specific clarity regarding the need and use of the alignment summary will be incorporated during Benchmark interviews and Advisement Day opportunities. All 25 student teachers, however, met the requirements to be recommended for licensure. When necessary, university supervisors, cooperating teachers and student teachers work to remediate deficiencies, when possible, to facilitate progress toward recommendation for licensure. Technology Education AIM 2010 Page 11 Student Teaching Performance Ratings Final student teaching evaluations are aligned with the School of Education’s conceptual framework (Danielson’s Framework for Teaching) and the 10 Wisconsin Teaching Standards. Teacher candidates finishing their student teaching experiences are evaluated by their cooperating teachers on a four point scale 1=Unsatisfactory, 2=Emerging, 3=Basic, 4=Proficient. Technology Education students’ rankings compared to the School of Education students’ ranks are compared below. Final student teaching evaluations are aligned with the School of Education’s conceptual framework (Danielson’s Framework for Teaching) and the 10 Wisconsin Teaching Standards. Teacher candidates finishing their student teaching experiences are evaluated by their cooperating teachers on a four point scale. A score of one indicates a lower ranking while a score of 4 indicates a higher ranking. Technology Education students’ rankings compared to the School of Education students’ ranks are compared in Table 7 below. Technology Education AIM 2010 Page 12 Table 7: Final student teaching evaluations Student Teacher Evaluations Technology Education Rating Scale: 1=Unsatisfactory, 2=Emerging, 3=Basic, 4=Proficient TECED SOE 2008 2009 2010 2010 N=27 N=38 N=24 N=120 Mean Mean Mean Mean Teachers know the subjects they are teaching 3.63 3.89 3.83 3.78 Teachers know how children grow 3.51 3.68 3.81 3.82 Teachers understand that children learn differently 3.60 3.83 3.83 3.73 Teachers know how to teach 3.54 3.87 3.75 3.84 Teachers know how to manage a classroom 3.56 3.65 3.69 3.65 Teachers communicate well 3.61 3.76 3.81 3.78 Teachers are able to plan different kinds of lessons 3.59 3.84 3.77 3.77 Teachers know how to test for student progress 3.67 3.76 3.77 3.75 Teachers are able to evaluate themselves 3.69 3.78 3.83 3.78 Teachers are connected with other teachers and the community 3.64 3.69 3.73 3.70 Teachers make effective use of instructional technologies to enhance student learning. NA NA 3.92 3.91 As demonstrated in Table 7, the average Technology Education student teacher performance ratings remained generally consistent 2009 to 2010 with all rankings well above the “Basic” level (3.00) as explained above. Technology Education AIM 2010 Page 13 Educational Benchmarking Inventory (EBI) The Educational Benchmarking Inventory (EBI) has been administered via computer to exiting student teachers during the fall and spring terms since 2003. Eighty-eight questions and fourteen EBI factors are collected for the purpose of unit assessment and are rated on a scale from 1 to 7 with 1= not at all, 4= moderately and 7= extremely). EBI data cannot be published in public domains and are available for internal use only. Table 8: Factor Analysis Trends Technology Education AIM 2010 Page 14 As demonstrated in Table 8, Technology scores have risen in all of the factors and sub-categories. However, knowing how to work with one’s peers, administration, and students’ parents, as well as, understanding educational policy and politics, and encouraging self-motivation and holistic learning continue to be ranked – although remaining above 3.0 - among the lowest. These rankings may be explained by the limitations of a generating an authentic technology education classroom experience within a university classroom. Although simulations have been and will continue to be reengineered to address these matters, until a student is emersed in a classroom and school culture, the contextual richness that is needed to create a powerful learning opportunitiy is not absolutely replicable. Continued efforts are being made to carefully select experiences in school settings that exemplify best practices in the field. Specifically, (beginning in Fall of 2010) during students’ first technology education pedegogy based course, Introduction to Technology Education, all students are travel to visit both a middle and high school classroom, interact with students, and have candid and directed conversations designed to address the issues alluded to above. In addition, during students’ pre-student teaching experience, they are asked to meet with a Union representative and discuss, again using directed questions, matters of politics and policy. Technology Education AIM 2010 Page 15 Table 9: EBI Institution Specific Questions ratings EBI - Institution Specific Questions Mean Data; Scale (1-Not at all, 4-Moderately, 7-Extremely) TECED SOE 09/10 10/11 10/11 N=20 N=16 N=87 To what degree were you prepared to create meaningful learning experiences for students based on your content knowledge? To what degree were you prepared to provide instruction that fosters student learning and intellectual, social and personal development? To what degree were you prepared to create instructional experiences adapted for students who learn differently? To what degree were you prepared to use a variety of learning strategies including the use of technology to encourage critical thinking and problem solving? To what degree were you prepared to manage classroom behavior and create a learning environment that encourages positive social interaction, active engagement in learning and self-motivation? To what degree were you prepared to use instructional technology and media to foster active inquiry, collaboration and interaction in the classroom? To what degree were you prepared to plan instruction based on knowledge of subject matter, students, the community and curriculum goals? To what degree were you prepared to use formal and informal assessment strategies to evaluate student progress? To what degree were you prepared to reflect on teaching and evaluate the effects of choices and actions on pupils, parents and others? To what degree were you prepared to foster relationships with colleges, families and the community to support student learning and well-being? 4.67 5.12 5.48 4.65 5.12 5.37 4.45 5.06 5.48 4.80 5.38 5.51 4.20 5.06 5.08 4.65 5.25 5.21 4.75 5.31 5.43 4.80 5.44 5.14 5.00 5.50 5.47 4.30 5.06 5.38 *We updated our questions beginning in the 2009-2010 school year Technology Education AIM 2010 Page 16 As demonstrated in Table 9, there was an increase in all factors in when compared with 2009/10 data. Technology education student teachers ranked all items above “moderate” (4). Although the low number of respondents could be cited as a problem to making any usable inferences, the data still suggests that technology education students, when compared with School of Education students overall, are generally not connecting what they are learning at Stout with what they are experiencing or being asked to do in technology education classrooms to the degree students in other programs are able. An increase in the number of technology and STEM (Science, Technology, Engineering, and Math) teachers have been and will continue to be added to the program’s advisory committee to inform and increase continuity. Alumni Follow-up Survey The UW-Stout One-Year Follow-up Survey is administered every 2 years by the Budget Planning and Analysis office at UW-Stout. Surveys are sent to graduates receiving and undergraduate degree after one year and after five years. Responses are measured on a five point scale with 1 representing “very poor”/”very dissatisfied”/”very low”/”not valuable”/”definitely no” and 5 representing “very good”/”very satisfied”/”very high”/”very valuable”/”definitely yes”. The next survey will be sent in 2012 for graduates in 2010 and 2006. The executive summary and full report from the Alumni Follow-Up Study are online at the following site: http://www.uwstout.edu/static/bpa/ir/afu/2010index.html Table 10: 2004 Alumni Survey responses (“My education at UW-Stout prepared me to:”) # Question Strongly Disagree 2 1 Understand the content and central concepts of the discipline I teach. 0 2 Create meaningful learning experiences based of my content knowledge. 3 4 Mean 2 6 2 6 16 3.75 0 1 5 6 5 17 3.88 Incorporate reading and language arts into the curriculum Effectively teach math skills in the curriculum 0 2 6 3 5 2 3 7 3 1 16 16 3.19 2.88 5 Provide instruction that supports student learning and their intellectual, social and personal development. 0 2 5 5 5 17 3.76 6 Create instructional experiences adapted for students who learn differently. 0 4 6 2 5 17 3.47 7 Use my knowledge of minority group relations to create appropriate instruction for diverse groups. 1 5 3 4 4 17 3.29 8 Modify curricula when instructing students with disabilities. 0 4 5 3 4 16 3.44 0 2 4 3 8 17 4.00 0 2 5 4 6 17 3.82 Use a variety of learning strategies to encourage critical thinking and problem solving. Create a learning environment that encourages positive social interaction, active 10 engagement in learning and self-motivation. Technology Education AIM 2010 4 Strongly Agree Responses 9 3 Page 17 11 Resolve conflicts between students and between students and staff. 2 1 8 3 3 17 3.24 12 Assist students in learning how to resolve conflicts. 2 3 5 4 3 17 3.18 13 Deal with crises or disruptive situations. 4 4 2 3 3 16 2.81 0 0 6 4 7 17 4.06 0 1 3 6 7 17 4.12 1 1 3 7 5 17 3.82 17 Use formal and informal assessment strategies to evaluate student progress. 0 0 5 6 6 17 4.06 18 Use assessment tools for students with disabilities. 2 2 7 4 2 17 3.12 0 3 5 3 6 17 3.71 0 2 6 3 5 16 3.69 0 2 3 6 5 16 3.88 Use effective communication techniques, media and technology to foster active inquiry, collaboration and supportive interaction in the classroom. 15 Use instructional technology to enhance student learning. 14 16 Plan instruction based on knowledge of subject matter, students, the community and curriculum goals. Use developmental, multiple and measurable assessment tools to assess student learning over time. 20 Use assessment grounded in research and based on best practices in education. 19 21 Use assessment tools with identified benchmarks or levels of proficiency. 22 Analyze students, classroom, and school performance data; make data-driven decisions about strategies for teaching and learning. 0 2 6 7 2 17 3.53 23 Reflect on teaching and evaluate the effects of choices and actions on students, parents and others. 0 1 6 4 6 17 3.88 0 2 5 6 4 17 3.71 1 5 4 2 5 17 3.29 1 4 2 6 4 17 3.47 Assess and analyze student learning, make appropriate adjustments to instruction, 24 monitor student learning, and develop and implement meaningful learning experiences to help all students learn. Foster relationships with colleagues, parents and the community to support student learning and well-being. Identify, apply and promote an understanding of career clusters and pathways within 26 a technology/pre-engineering curriculum. 25 27 Recognize how student organizations provide opportunities for professional growth and leadership for students in a contextualized learning environment. 0 4 5 6 1 16 3.25 28 Maintain and use contemporary tools, instruments and machines safely in a program of study. 3 0 3 4 7 17 3.71 Technology Education AIM 2010 Page 18 Table 11: 2008 Alumni Survey responses # Question Strongly Disagree 2 1 Understand the content and central concepts of the discipline I teach. 1 2 3 Create meaningful learning experiences based of my content knowledge. Incorporate reading and language arts into the curriculum 4 Mean 3 2 1 3 10 3.20 3 3 1 3 3 0 3 2 2 0 10 10 2.60 2.30 Effectively teach math skills in the curriculum 4 3 0 0 3 10 2.50 5 Provide instruction that supports student learning and their intellectual, social and personal development. 0 4 2 3 1 10 3.10 6 Create instructional experiences adapted for students who learn differently. 1 4 0 3 2 10 3.10 7 Use my knowledge of minority group relations to create appropriate instruction for diverse groups. 2 2 1 2 3 10 3.20 1 3 1 2 3 10 3.30 0 3 3 1 3 10 3.40 0 2 5 1 2 10 3.30 11 Resolve conflicts between students and between students and staff. 3 4 1 0 2 10 2.40 12 Assist students in learning how to resolve conflicts. 3 4 1 1 1 10 2.30 13 Deal with crises or disruptive situations. Use effective communication techniques, media and technology to foster active 14 inquiry, collaboration and supportive interaction in the classroom. 3 3 1 2 1 10 2.50 0 1 2 5 2 10 3.80 15 Use instructional technology to enhance student learning. 1 0 2 6 1 10 3.60 1 0 6 1 2 10 3.30 17 Use formal and informal assessment strategies to evaluate student progress. 1 0 3 3 3 10 3.70 18 Use assessment tools for students with disabilities. 1 3 2 3 1 10 3.00 1 0 5 2 2 10 3.40 9 10 16 19 Modify curricula when instructing students with disabilities. Use a variety of learning strategies to encourage critical thinking and problem solving. Create a learning environment that encourages positive social interaction, active engagement in learning and self-motivation. Plan instruction based on knowledge of subject matter, students, the community and curriculum goals. Use developmental, multiple and measurable assessment tools to assess student learning over time. Technology Education AIM 2010 4 Strongly Agree Responses 8 3 Page 19 20 Use assessment grounded in research and based on best practices in education. 1 1 4 3 1 10 3.20 21 Use assessment tools with identified benchmarks or levels of proficiency. 1 2 4 1 2 10 3.10 1 2 2 3 2 10 3.30 1 0 3 3 3 10 3.70 1 0 4 1 4 10 3.70 22 Analyze students, classroom, and school performance data; make data-driven decisions about strategies for teaching and learning. Reflect on teaching and evaluate the effects of choices and actions on students, parents and others. Assess and analyze student learning, make appropriate adjustments to instruction, 24 monitor student learning, and develop and implement meaningful learning experiences to help all students learn. 23 25 Foster relationships with colleagues, parents and the community to support student learning and well-being. 3 1 2 3 1 10 2.80 26 Identify, apply and promote an understanding of career clusters and pathways within a technology/pre-engineering curriculum. 3 3 1 1 2 10 2.60 27 Recognize how student organizations provide opportunities for professional growth and leadership for students in a contextualized learning environment. 2 2 2 2 2 10 3.00 28 Maintain and use contemporary tools, instruments and machines safely in a program of study. 4 1 2 0 3 10 2.70 When comparing the responses of alumni from 2004 against 2008 graduate responses (Tables 10 and 11), the overwhelming majority of means are in the moderate to above moderate range (3 to 4) with the highest rankings in both surveys consistently given to being able to use a variety of learning strategies to encourage critical thinking and problem solving, use effective communication techniques, media and technology to foster active inquiry, collaboration and supportive interaction in the classroom, use instructional technology to enhance student learning, Use formal and informal assessment strategies to evaluate student progress, reflect on teaching and evaluate the effects of choices and actions on students, parents and others, and assess and analyze student learning, make appropriate adjustments to instruction, monitor student learning, and develop and implement meaningful learning experiences to help all students learn. Like the responses to EBI survey items in Table 8, lower rankings in both the 2004 and 2008 data are associated with respondents preparation to deal with political (Foster relationships with colleagues, parents and the community to support student learning and well-being) and classroom environment and culture problems (Resolve conflicts between students and between students and staff; assist students in learning how to resolve conflicts; deal with crises or disruptive situations) lower than other items. Again, although simulations have been and will continue to be reengineered to address these matters, until a student is emersed in a classroom and school culture, the contextual richness that is needed to create a powerful learning opportunitiy is not absolutely replicable. Technology Education AIM 2010 Page 20 Lastly, being able to incorporate reading, language arts, and math effectively into their curriculum was ranked lower in both surveys. As education courses formerly designed to address only technology education teacher candidates are revised to meet the needs of science education and, beginning Fall semester 2011, math education students, the impact on these need areas will be addressed. It is important to note that responses of 2004 alumni were overwhelmingly higher than 2008 respondents. This is to be expected as beginning teachers tend to go through phases that include simple day to day survival and disillusionment with what they thought the job actually entailed (Moir, 1990). Also, teachers that persevere, as evidenced by the responses of the 2004 alumni, tend to report rating that reflect a seeing the forest instead of the trees perspective. Moir describes these stages as being characterized by a sense of rejuvenation and reflection. Table 12: 2004 Technology Education Alumni ratings of the How Important UW-Stout was in contributing to General Education # Question No degree of influence 2 1 Writing effectively 0 2 Speaking or presenting ideas effectively 3 Responses Mean 2 5 5 5 17 3.76 0 1 3 6 7 17 4.12 Listening effectively 2 2 2 9 2 17 3.41 4 Utilization of technologies 0 1 3 4 9 17 4.24 5 6 Using analytic reasoning Creative problem solving 1 0 2 6 6 2 1 6 6 4 17 17 3.35 3.76 7 Critically analyzing information 0 3 3 6 3 15 3.60 8 Maintaining a sense of physical well-being 3 3 6 5 0 17 2.76 9 Appreciating and understanding diversity 2 2 4 5 4 17 3.41 10 Developing a global perspective 11 Appreciating the value of literature and the arts 1 1 7 2 3 4 7 6 1 2 17 17 3.12 2.76 12 Appreciating the natural or physical sciences 1 3 8 2 3 17 3.18 13 Appreciating social, economic and political forces 1 2 8 2 3 16 3.25 14 Appreciating history in context to current issues 5 4 4 3 17 3.18 Technology Education AIM 2010 1 3 4 High degree of influence Page 21 Table 13: 2008 Technology Education Alumni ratings of the How Important UW-Stout was in contributing to General Education # Question No degree of influence 2 1 Writing effectively 2 2 Speaking or presenting ideas effectively 3 Responses Mean 1 4 2 2 11 3.09 0 2 1 6 2 11 3.73 Listening effectively 2 1 1 7 0 11 3.18 4 Utilization of technologies 1 1 3 2 4 11 3.64 5 6 Using analytic reasoning Creative problem solving 2 2 1 3 3 2 2 1 2 4 11 11 3.18 3.36 7 Critically analyzing information 2 1 3 4 1 11 3.09 8 Maintaining a sense of physical well-being 4 2 1 3 1 11 2.55 9 Appreciating and understanding diversity 2 2 1 4 2 11 3.18 10 Developing a global perspective 3 2 2 3 1 11 2.73 11 Appreciating the value of literature and the arts 12 Appreciating the natural or physical sciences 3 2 4 3 1 0 4 3 2 0 11 11 2.18 2.45 13 Appreciating social, economic and political forces 1 5 3 1 0 10 2.40 14 Appreciating history in context to current issues 3 4 0 0 11 2.00 4 3 4 High degree of influence When comparing the responses of alumni from 2004 against 2008 graduate responses (Tables 12 and 13), the majority of means are in the moderate to above moderate range (3 to 4) with the highest rankings given to being able to utilize technologies and lowest given to maintaining a sense of well being, appreciating the value of literature and the arts, and appreciating history in the context to current issues. Although differing numbers of respondents and a program revision in 2007 could be cited as factors for the overall lower rankings, in the literature beginning teachers generally work through times of frustration in their beginning years – which could negatively influence 1 year alumni ratings. Technology Education AIM 2010 Page 22 Communicating Assessment Data with Constituencies Communicating data with faculty members, advisory board members, and students within the program will be done using various methods. The report will be shared with faculty members during scheduled discipline area work group meetings (DAWG) designed to support ongoing program improvement. Program faculty and staff then discuss ways to better meet the needs of students throughout the program. Action plans for desired change result from work group meetings. The B.S in Technology Education program advisory board, which includes students, will receive a copy of the Assessment in the Major report during the fall advisory meeting. Their comments and recommendations for improvement will be encouraged. University supervisors will be asked to share the AIM report with cooperating teachers and solicit feedback from them regarding improvements for the program. The report will be made available to students within the Technology Education program and an opportunity provided for feedback. Data from the AIM report will also be used in program revision processes. Utilization of Assessment Data to Improve Courses and the Program The following are changes or improvements planned for the upcoming years: 2009 Goals 1 2 Increase student exposure to classrooms, teachers, and students. a. School of Education staff and Technology Education faculty and staff are currently identifying schools, contemporary technology education programs, and qualified cooperating teachers that would serve as observation sites for pre-student and student teacher candidates. b. Attention will continue to be focused on developing pre-service teachers’ and Technology Education Collegiate Association’s efforts to work with peers, professionals and students through competitive events, field trips, and lab activities. The Technology Education students, faculty and staff will continue working collaboratively with STEM and other colleges/institutions as improving lab experiences and lab access for students is a key element for students to thoroughly understand their content and, as a result, it’s Technology Education AIM 2010 Evidence towards meeting goals or rationale for abandoning Continued efforts are being made to carefully select experiences in school settings that exemplify best practices in the field. Specifically, (beginning in Fall of 2010) during students’ first technology education pedegogy based course, Introduction to Technology Education, all students travel to visit both a middle and high school classroom, interact with students, and have candid and directed conversations designed to address the issues alluded to above. In addition, the Program Director and the Coordinator of Field Experiences have been visiting and are continuing to schedule meetings with Technology and STEM education programs in Wisconsin and Minnesota to facilitate healthy and lasting relationships with programs that exemplify best practice. A collaborative relationship has been established with the ChetekWeyerhauser (WI) school district to help facilitate a relationship between technology and science education students and teacher faculty to increase the authenticity of STEM problem based learning. Page 23 interdisciplinary power. Specifically, lab experiences and their pedagogical implications have been emphasized. An increase in the number of technology and STEM (Science, Technology, Engineering, and Math) teachers have been and will continue to be added to the program’s advisory committee to inform and increase continuity. Specifically, members from the DPI, The STEM Center at the University on Minnesota – Twin Cities (math education in particular), President of the Minnesota Technology Education Association, and middle and high school administrators have been added. 3 Finally, there will be continued efforts to seeking input from the recently revised advisory committee that includes, among others, current students and alumni, STEM education professionals from other institutions, UW-Stout STEM college faculty, and Technology Education faculty. 1 Increase student exposure to classrooms, teachers, and students. a. School of Education staff and Technology Education faculty and staff are currently identifying schools, contemporary technology education programs, and qualified cooperating teachers that would serve as observation sites for pre-student and student teacher candidates. b. Attention will continue to be focused on developing pre-service teachers’ and Technology Education Collegiate Association’s efforts to work with peers, professionals and students through competitive events, field trips, and lab activities. The Technology Education students, faculty and staff will continue working collaboratively with STEM and other colleges/institutions as improving lab experiences and lab access for students is a key element for students to thoroughly understand their content and, as a result, it’s interdisciplinary power. Finally, there will be continued efforts to seeking input from the recently revised advisory committee that includes, among others, current students and alumni, STEM education professionals from other institutions, UW-Stout STEM college faculty, and Technology Education faculty. 2010 Goals 2 3 Technology Education AIM 2010 Page 24 References Moir, E. (1990). Phases of first-year teaching. Retrieved October 20, 2010 from New Teacher Center at the University of Santa Cruz Web site: http://www.newteachercenter.org/article3.html Technology Education AIM 2010 Page 25 Technology Education, B.S. Minority enrollment Male Female Total enrollment SCH Student FTE New Freshmen Transfers Number of graduates by year: Number of male graduates Number of female graduates Number of minority graduates Number employed in related major: Number continuing education: Number employed in major: Percent employed: One-Year Rates in Program One-Year Retention Rates - Any Program Six-Year Graduation Rates in Program Six-Year Graduation Rates Any Program Average High School Percentile Average ACT Composite of New Freshmen Average Cumulative GPA Freshmen: 1-29.5 credits Sophomore: 30-59.5 credits Junior: 60-89.5 credits Senior: 90 or more credits Honors Program (FA10) Learning Comm. Partic. Study Abroad Students % of grads who participated in Experiential Learning Salary Average Salary Low Salary High I would attend UW-Stout again I would enroll in the same academic program Three-Year Show Rates - New Freshmen Three-Year Show Rates - New Transfers 10-11 2 120 11 131 1,876 125 13 10 NA NA NA NA 09-10 08-09 4 132 9 141 2,053 137 21 13 30 29 1 1 - 2 137 10 147 2,136 142 22 11 27 25 2 1 16 1 100.0% 63.6% 77.3% 71.4% 76.2% 07-08 2 157 12 169 2,491 166 23 12 64 62 2 2 3 23 100.0% 65.2% 82.6% 06-07 04-'05 2004 20.1% 48.3% 59.1% 22.4 NA 26 25 17 63 2 100% NA NA NA 61.9% 21.0 2.48 27 23 35 56 NA NA NA 58.9% 21.1 2.45 30 28 28 61 $ $ $ 63.8% 21.1 3.02 34 34 33 68 2003 2002 2001 2000 2008 Grads 57.1% 71.4% 44.6% 58.9% 52.2% 60.9% 32.1% 63.0% 67.5% 20.2 2.78 37 44 31 97 32,000 29,000 38,000 3.45 3.36 65% 74% 2004 Grads 5 197 12 209 3,041 203 25 19 45 42 3 2 2 2 30 93.0% 68.0% 88.8% 4.00 3.41 Technology Education, B.S. FR High School Percentile Rank 10-11 Enrollment New FR and Transfer New FR Enrollment Transfer Enrollment 209 59.1% 09-10 08-09 Enrollment Demographics 169 61.9% 58.9% 07-08 Total Enrollment 63.8% 06-07 131 141 19 147 25 67.5% FR ACT Avg. Composite score 23 22 10-11 09-10 21.0 08-09 21.1 07-08 21.1 06-07 21 197 22.4 Male 120 132 137 157 13 12 11 10 20.2 12 11 9 12 13 10 Female FR Avg GPA 10-11 09-10 2.48 08-09 2.45 5 06-07 07-08 08-09 09-10 10-11 06-07 07-08 08-09 09-10 10-11 2 06-07 2 07-08 2.78 09-10 06-07 3.02 2 10-11 07-08 Minority Enrollment 08-09 4 Technology Education, B.S. 2 Employment Numbers 64 - 29 25 2 - - 10-11 09-10 08-09 07-08 06-07 0.0% Retention Rates Any Program 10-11 09-10 08-09 07-08 06-07 0.0% 2 45 42 3 2 Minority graduates 1 1 - 06-07 - 07-08 1 2 1 07-08 3 Female graduates 08-09 2 - 09-10 2 - Retention Rates in Program Percent Employed 27 62 Male graduates One Year Retention Rates Experiential Learning 30 - 16 - Total graduates by year 06-07 30 10-11 - 08-09 Number continuing education - 09-10 Number employed in related major 23 10-11 Number employed in major Graduates in Program Six Year Graduation Rates Graduation Rates In Program Graduation Rates - Any Program 71.4% 63.6% 65.2% 68.0% 76.2% 77.3% 82.6% 88.8% Employment Percentages 10-11 09-10 08-09 100% NA 100% 07-08 06-07 100% 93% 2004 63.0% 2003 57.1% 2002 44.6% 2001 2000 For more information on retention/graduation rates go to: http://www2.uwstout.edu/content/bpa/ir/retention/indexstu.html 20.1% 60.9% 58.9% 52.2% 32.1% 71.4% 48.3% Technology Education, B.S. 3 Other Three- Year Show Rates 10-11 10-11 74% Salary Data Salary High 08-09 $38,000 2 Salary Average 08-09 Salary Low 08-09 $32,000 $29,000 SCH 3,041 65% Student Credit Hours 1,876 2,053 2,136 10-11 09-10 08-09 2,491 07-08 06-07 Student FTE 203 - - Honors Learning Study Program Comm. Abroad (FA10) Partic. Students 125 137 142 10-11 09-10 08-09 166 FTE Three-Year Three-Year Show Rates - Show Rates New New Freshmen Transfers 07-08 06-07