University of Wisconsin-Green Bay ASSESSMENT AND TESTING SERVICES Tel: (920) 465-2221 Fax: (920) 465-2430 E-mail: assessment@uwgb.edu Report of Results: UWGB General Education Assessment Program 2004-05 Report of Results: UWGB General Education Assessment Program 2004-05 Academic Year [with extended data analysis] Prepared by Alissa M. Clark, Coordinator Assessment Services May 2005 Table of Contents Acknowledgments................................................................................................... iii Introduction ..............................................................................................................4 Interpretation of Scores ...........................................................................................4 Summary of Results .................................................................................................5 Overall Performance on the College BASE ............................................................................ 5 English........................................................................................................................................ 6 Reading Skills.......................................................................................................................... 6 Understanding Literature ....................................................................................................... 6 Longitudinal Data. .................................................................................................................. 6 Mathematics .............................................................................................................................. 7 General Mathematics .............................................................................................................. 7 Algebra .................................................................................................................................... 7 Geometry ................................................................................................................................. 7 Longitudinal Data. .................................................................................................................. 8 Science ........................................................................................................................................ 9 Laboratory & Field Work ....................................................................................................... 9 Fundamental Concepts ........................................................................................................... 9 Longitudinal Data. .................................................................................................................. 9 Social Studies ........................................................................................................................... 10 History................................................................................................................................... 10 Social Studies ........................................................................................................................ 10 Longitudinal Data. ................................................................................................................ 11 Reasoning Competencies ........................................................................................................ 12 Student Performance by Type of Major ............................................................................... 13 Factors Affecting Test Performance...................................................................................... 13 Student Perceptions of the Experience ................................................................................. 13 Dissemination and Use of Results .........................................................................14 Additional Results ..................................................................................................14 Appendix .................................................................................................................15 Part I: Tabular Summary of Results..................................................................................... 16 Part II: Proficiencies Measured by the College BASE ........................................................ 25 Assessment Services page ii Report: General Education Assessment Program (2004-05) Acknowledgments Implementing a successful assessment program is a difficult and time-consuming task, which cannot be accomplished without the support and assistance of many individuals. Grateful recognition goes to Cliff Abbott, Scott Furlong, Catherine Henze, Don McCartney, Steve Newton, and Linda Peacock-Landrum for their exceptional presentations at the eight orientation and feedback sessions conducted each semester. Assessment Services page iii Report: General Education Assessment Program (2004-05) Introduction The UWGB General Education Assessment Program is in its 14th year. Since spring 1991, more than twelve thousand students (n = 12,233) have completed the program. The purpose of the program is to provide students and UWGB with feedback regarding how well students have developed specific academic skills through their sophomore year. The assessment program has three components: an orientation session, an achievement test, and a feedback session. The one-hour orientation and feedback sessions are voluntary. The orientation session is designed to inform students about the assessment process, help motivate them to do well on the test, and answer questions about the program. Students are told how to interpret their test results and how their performance compares with their peers during the feedback sessions. Besides learning about continuing self-assessment, students choose between two “mini-sessions”: one on developing a portfolio, and the other on steps they should take to ensure career readiness. The achievement test is the College BASE (College Basic Academic Subjects Examination), a commercially published, criterion-referenced test battery that measures twenty-two specific skills in English, mathematics, science and social studies with three different levels of reasoning competency. The test includes 180 multiple-choice items. It provides 39 scores including a measure of proficiency in English, mathematics, science and social studies; nine subscores, called cluster scores, which represent proficiency in specific areas within each curricular area; 22 skill scores distributed throughout the subject areas; and three reasoning competency scores. (See Part II of the Appendix for a complete description of the skills.) Interpretation of Scores As a criterion-referenced examination, the items comprising the College BASE assess specific, clearly delineated content and skills. The criteria and proficiency levels for the skills are based on what a panel of experts agreed should be expected of students who have completed at least two years of college. The College BASE provides two types of test results: numeric scores that range from 40 to 560 points with a mean of 300, and ratings that can be either High, Medium or Low. Numeric scores represent examinees’ familiarity with traditional subject matter (e.g., mathematics) or how well they performed on a group of closely related skills (e.g., reading critically). Numeric scores that fall between 258 and 332 represent an adequate level of skill development and knowledge base. Scores above 332 represent highly developed skills and an extensive knowledge base. Scores below 257 suggest that skills are not well developed and/or that the subject matter knowledge is weak. The proficiency level of UWGB students on each of the 22 curricular skills and 3 reasoning competencies is rated using a three-point scale. A “High” rating suggests that a student has Assessment Services page 4 Report: General Education Assessment Program (2004-05) thoroughly mastered a particular area or skill; students with adequate proficiency receive a “Medium” rating; and a “Low” rating suggests that proficiency levels are not adequate for students in the second half of their college careers. Summary of Results The following results show the overall performance of UWGB students in important areas of general education skill development. However, because the College BASE provides “criterionreferenced” scores, results are best interpreted at the individual student level. By design, the BASE test does not provide “norm-referenced” scores that would allow a comparison of student performance at UWGB with other institutions. Results shown are for the 2004-05 academic year. These results are then placed in the context of longitudinal results, beginning with the 1991-92 academic year. During the 2004-05 academic year, 902 of UWGB’s matriculated students who had earned between 54 and 72 credits completed the College BASE test1. The voluntary orientation and feedback sessions were well attended during the academic year. Fifty-four percent of the eligible students participated in an orientation session and 41 percent either participated in a feedback session or earned the minimum score needed to earn a free credit sans attendance at a feedback session. Overall, results suggest that UWGB students have well developed general education skills by the end of their sophomore year. Reasoning skills, assessed at three distinct levels, are also well developed for a significant majority of students. Overall Performance on the College BASE UWGB students performed adequately on the College BASE in 2004-05. Students performed best on the mathematics portion of the test (mean = 321) followed by science (mean = 284), social studies (mean = 282), and English (mean = 280). The average Composite score (294) approximated the scoring service’s anticipated average Composite score of 300 (see Chart 1). Chart 1 College BASE: Composite Scores (1991-2005) Scores 320 310 300 290 280 Composite 91-92 92-93 93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 306 304 309 305 305 305 304 296 302 295 299 295 295 294 1 The results described in this report are for 885 of the 902 students who took the College BASE. Students for whom relevant data was not available in the UWGB Student Information System or took the exam too late for scoring this year are excluded from this analysis. Assessment Services page 5 Report: General Education Assessment Program (2004-05) English Reading Skills BASE results show that 63 percent of 2004-05 UWGB College BASE test takers can read college level material critically and analytically. The test results also indicated that over one-third of the students (37 percent) had difficulty recognizing assumptions and implications, and evaluating ideas. In addition, 31 percent of the students had problems seeing relationships between form and content when reading a literary text. Understanding Literature Over the past ten years, this skill area has been one of the lowest areas of performance for UWGB students. Almost one-third (30%) of the students received a “Low” rating for their understanding of literature including different literary forms and historical contexts. About half (49%) received a “Medium” rating and another fifth (21%) received a “High” rating. English Skill Ratings Cluster Reading and Literature Writing Skill Area 1. Reading Critically 2. Reading Analytically 3. Understanding Literature 4. Writing as a Process 5. Conventions of Written English % Receiving Ratings High Med Low 16% 47% 37% 15% 54% 31% 21% 49% 30% 22% 60% 18% 11% 72% 18% Longitudinal Data Since 1991-92, UWGB students have typically achieved average scores on the English section of the College BASE. As Chart 2 shows, the overall English score and the scores on the Reading and Literature portion of the exam were the lowest average scores our students have received since the 1991-92 academic year. Interestingly, even though English has often been the lowest scoring area on the College BASE for UWGB students, they tell us on the research question portion of the College BASE that they believe they have strong English skills. Assessment Services page 6 Report: General Education Assessment Program (2004-05) Chart 2 College BASE: English Subject Scores & Subscores (1991-2005) 325 320 315 310 Scores 305 300 295 290 285 280 275 270 91-92 92-93 93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 ENGLISH 294 292 305 301 295 298 292 286 294 289 286 285 282 280 Reading & Lit 295 285 296 284 288 292 287 281 290 283 282 280 277 274 Writing 296 302 314 321 305 307 299 298 303 302 299 300 294 297 Mathematics General Mathematics In the general mathematics cluster, 89 percent of the students that took the exam this year could use mathematical techniques in the solution of real life problems; 92 percent could effectively use the language, notation, and deductive nature of mathematics to express quantitative ideas with precision; and 88 percent could use the techniques of statistical reasoning and recognize common misuses of statistics. Algebra On the algebra portion of the test, more than eight of 10 students could evaluate algebraic and numerical expressions (29% received a “High” rating, 56% a “Medium” rating) and solve equations and inequalities (30% received a “High” rating, 56% a “Medium” rating). Geometry Even though most students had not taken a geometry class since high school (four or more years earlier), they did remarkably well on this portion of the exam. Thirty-three percent received a “High” rating for their ability to recognize two- and three- dimensional figures and their properties. Twenty-one percent of the students received a “High” rating for their ability to use the properties of two- and three-dimensional figures to perform geometrical calculations. Assessment Services page 7 Report: General Education Assessment Program (2004-05) Mathematics Skill Ratings Cluster General Mathematics % Receiving Ratings High Med Low 35% 54% 11% 32% 60% 8% 37% 51% 12% 29% 56% 15% 30% 56% 14% 33% 46% 21% 21% 56% 22% Skill Area 6. Practical Applications 7. Properties and Notations 8. Using Statistics 9. Evaluating Expressions 10. Equations and Inequalities 11. 2 and 3 Dimensional Figures 12. Geometrical Calculations Algebra Geometry Longitudinal Data Since 1991-92, UWGB students have achieved above average scores on the mathematics section of the College BASE. As Chart 3 shows, mathematics scores for UWGB students are on the rise, led by students’ scores on the general mathematics and algebra portions of the exam. Mathematics is consistently the highest scoring area on the College BASE, yet students tell us on the research question portion of the College BASE that they do not believe they have very strong math skills. Chart 3 College BASE: Math Subject Scores & Subscores (1991-2005) 335 330 325 Scores 320 315 310 305 300 295 290 91-92 92-93 93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 MATH 304 308 312 311 314 316 318 310 319 314 322 316 323 321 General 297 302 307 313 315 321 322 320 321 319 325 322 323 330 Algebra 298 300 303 305 309 307 311 301 312 307 313 308 315 323 Geometry 307 304 306 299 307 307 307 300 313 303 313 308 318 324 Assessment Services page 8 Report: General Education Assessment Program (2004-05) Science Laboratory & Field Work About two-thirds (65 percent) of the students tested had a good understanding of how observation and experimentation relate to the development of scientific theories; 81 percent could recognize appropriate procedures for gathering scientific information; and 74 percent were able to interpret and express the results of observation and experimentation. Fundamental Concepts Between 21 and 26 percent of the students had a very good understanding of the fundamental concepts, principles, and theories of the life sciences (21%) and physical sciences (26%). However, a higher percentage of each group received a “Low” rating in these areas (32% and 29% respectively). Science Skill Ratings Cluster Laboratory & Field Work Fundamental Concepts Skill Area 13. Observation/Experimental Design 14. Lab/Field techniques 15. Interpreting Results 16. Life Sciences 17. Physical Sciences % Receiving Ratings High Med Low 22% 43% 36% 22% 59% 18% 19% 55% 25% 21% 47% 32% 26% 45% 29% Longitudinal Data Science scores for UWGB students have declined – in general – since 1991-92, with Laboratory and Field Work scores exceeding Fundamental Concept scores (see Chart 4). However, the Fundamental Concepts average score is higher than it has been for the last four years. Assessment Services page 9 Report: General Education Assessment Program (2004-05) Chart 4 College BASE: Science Subject Scores & Subscores (1991-2005) 315 310 305 Scores 300 295 290 285 280 91-92 92-93 93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 SCIENCE 312 306 308 303 309 307 306 295 300 289 291 284 283 284 Lab & Field 308 308 310 306 310 309 310 300 302 293 295 289 286 288 Fund Concepts 310 299 301 296 302 299 297 288 296 284 286 282 284 296 Social Studies History Results for 2004-05 show that of the students tested, 75 percent have “Medium” to “High” levels of knowledge about the significance of U.S. and 81 percent have “Medium” or “High” levels of knowledge about world events. Overall, students performed somewhat better on the questions relating to world events than on the questions relating to U.S. events. Social Studies Students did best with recognizing basic features and concepts of world geography, with eight of ten students (81%) receiving a “Medium” or “High” rating. Just over one-quarter (29%) received a “Low” rating in the area assessing recognition of basic features and concepts of the world’s political and economic structures. The largest percentage of “Low” ratings (30%) was in the area of recognition of appropriate investigative and interpretive procedures in the social sciences. Assessment Services page 10 Report: General Education Assessment Program (2004-05) Social Studies Skill Ratings Cluster History % Receiving Ratings High Med Low 20% 61% 18% 20% 55% 25% 22% 59% 19% 23% 47% 29% 19% 51% 30% Skill Area 18. Significance of World Events 19. Significance of U. S. Events 20. Geography 21. Political & Economic Structures 22. Social Science Procedures Social Sciences Longitudinal Data Since 1991-92, UWGB students have achieved average to below average scores on the social studies section of the College BASE. Chart 5 shows how social studies scores have been declining since the 1993-94 academic year. Students’ scores on the subject’s two clusters, history and social science, were nearly indistinguishable in the first part of the last decade. In the latter part, history scores have been somewhat more stable than the social sciences scores. Chart 5 College BASE: Social Studies Subject Scores & Subscores (1991-2005) 315 310 305 Scores 300 295 290 285 280 91-92 92-93 93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 SOCIAL STUDIES 310 308 311 305 301 297 298 293 294 290 295 293 291 282 History 308 305 310 304 300 298 300 295 296 295 297 296 295 283 Social Sciences 309 309 309 304 301 297 297 292 294 287 293 291 290 285 Assessment Services page 11 Report: General Education Assessment Program (2004-05) Reasoning Competencies A strong proportion (53% to 90%) of the students displayed competence in the areas of reasoning assessed by the College BASE. The three reasoning competencies assessed by the College BASE are arrayed in order of difficulty, and so it is appropriate that students’ highest achievement in reasoning skill is associated with Interpretive Reasoning (the most straightforward of the three reasoning competencies), followed by Strategic Reasoning, and by Adaptive Reasoning. Importantly, only one in ten students (10%) received a “Low” rating in Interpretive Reasoning. Over half (53%) demonstrated “Medium” or “High” performance in Adaptive Reasoning, the highest level of reasoning assessed by the College BASE. It refers to the ability to synthesize new rules, generate hypotheses and express judgments. As shown by Chart 6, it appears that students’ Interpretive Reasoning scores have increased slightly over time, while students’ Strategic and Adaptive Reasoning scores have declined. Reasoning Skills Ratings % Receiving Ratings High Med Low 42% 48% 10% 20% 58% 22% 11% 42% 47% Reasoning Area Interpretive Reasoning Strategic Reasoning Adaptive Reasoning Chart 6 College BASE: Reasoning Competency Scores (1991-2005) 325 320 315 310 Scores 305 300 295 290 285 280 275 270 91-92 92-93 93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 Interpretive 305 310 314 312 318 317 319 308 321 314 320 313 317 314 Strategic 313 311 316 312 312 315 314 305 306 300 304 298 299 296 Adaptive 297 300 303 294 295 294 291 289 296 284 287 283 277 271 Assessment Services page 12 Report: General Education Assessment Program (2004-05) Student Performance by Type of Major As a group, students that took the College BASE in 2004-05 with majors in the humanities had the highest mean Composite Score. With a score of 300 representing adequately developed skills, the mean composite score for each of five types of majors was: humanities, 325; natural sciences, 317; professional studies, 291; social sciences, 282; and fine arts, 278. Factors Affecting Test Performance Ninety percent of students used 76 or more minutes to work on the exam (the minimum required time is 70 minutes and the maximum is 180 minutes) and, generally, the more time students spent on the test, the higher their scores. Students who took the College BASE during the spring semester of the 2004-05 academic year, completed a brief “post-test” survey in which they self-reported on several aspects of the testing experience2. One question asked students to identify the extent to which they agreed with the statement, “When taking the College BASE, I put forth my best effort.” On a scale of 1-10, where 1 = strongly agree, 49% of the students responding (185 of 399) gave a score of 1, 2, or 3 – indicating that they had put forth their best effort. Seventeen percent (62 of 399) gave a score of 8, 9, or 10 – indicating that they had not put forth their best effort. Students attending a Feedback Session completed a separate survey that asked about intrinsic and extrinsic incentives associated with the test. Of the eight incentives listed, the “opportunity to receive a credit” and “personal satisfaction” were the most significant factors influencing student test performance. Sixty-eight percent rated the credit as “Very Much” a factor and 36 percent said that personal satisfaction was “Very Much” a factor. Recognition that the College BASE assessed important areas of knowledge was an important factor for all but 17 percent of the student. The transcript notation and congratulatory “letter from the Provost” were also cited as very important incentives by 28 percent and 24 percent of the students, respectively. Student Perceptions of the Experience Twenty percent of the students attending a Feedback Session (38 of 194) reported that it had been “very worthwhile” for them to participate in the General Education Assessment Program. Another 72 percent (139 of 194) reported that the experience had been “somewhat worthwhile.” Regarding the Orientation Sessions alone, 73 percent of those attending rated the Orientation Sessions as “Good” or “Excellent.” Regarding the Feedback Sessions alone, 84 percent of those attending rated the Feedback Sessions as “Good” or “Excellent.” ________________________ 2 The students who took the College BASE test during the fall semester filled out a different post-test survey that was created by and sent back to the ARC Scoring service that scores the College BASE tests for UWGB. Thus, only the 399 students who tested during the spring semester, and filled out a survey, are included in the post-test survey results. Assessment Services page 13 Report: General Education Assessment Program (2004-05) Dissemination and Use of Results Individual students and the institution use information obtained through the UWGB General Education Assessment Program in several ways. First, students receive feedback regarding their performance from an individual score report which includes thirty-nine (39) different pieces of information. All students, through a brochure, are encouraged to study their results carefully, consult with their faculty advisors, and take some constructive steps to shore-up areas that may need improvement. In addition, students are asked to attend a voluntary Orientation and Feedback Session. During 2004-05, over half (54%) of the students took advantage of an Orientation Session and 41 percent either attended a Feedback Session or earned the minimum composite score needed (350) to earn the free credit without Feedback session attendance. Second, because the test publisher provides information about student performance individually and in the aggregate at several different levels, the results are available for diagnostic use by the institution as a whole and by individual academic units. A copy of this report is sent to the Provost and Vice Chancellor, academic deans, chairs of the General Education Council and Academic Affairs Council and the director of Institutional Research. Finally, individual test scores are included in the institutional database to allow for additional analysis of the information by various factors such as major, grade point average, courses completed, and the like. Additional Results The balance of this report consists of a two-part Appendix. Part I consists of several tables that provide a detailed summary of test results. A complete listing of tables can be found on the first page of the Appendix. Part II of the Appendix is a detailed description of the proficiencies measured by the College BASE. Assessment Services page 14 Report: General Education Assessment Program (2004-05) Appendix Part I: Tabular Summary of Results Table 1: Background Characteristics of Students Completing College BASE Table 2: Students Completing College BASE by Major Category Table 3: Major Field of Study - Students Completing College BASE Table 4: College BASE Subject and Cluster Scores Table 5: College BASE Skill and Reasoning Competency Scores Table 6: College BASE Subject Scores by Major Table 7: College BASE Subject Scores for Majors with 20+ Students Completing CBASE Table 8: College BASE Subject Scores: Mean Scores for Five Groups of Majors Table 9: Time Spent on Multiple-Choice Portion of College BASE Table 10: College BASE Subject and Cluster Scores by Test Completion Time Table 11: College BASE Subject and Cluster Scores by Transfer Status Table 12: Orientation and Feedback Session Attendance Table 13: Factors Motivating Test Performance Table 14: Orientation and Feedback Sessions: Student Ratings of Overall Quality Part II: Proficiencies Measured by the College BASE Assessment Services page 15 Report: General Education Assessment Program (2004-05) Appendix Part I: Tabular Summary of Results Table 1: Background Characteristics of Students Completing College BASE (Percentage Distribution) Fall 2004 and Spring 2005 Combined Category Sex Male Female Age Up to 21 22 to 25 26 to 29 30 to 40 41+ Ethnicity White Minority n % 289 595 884 32.6% 67.3% 99.9% 544 261 33 33 14 885 61.5% 29.5% 3.7% 3.7% 1.6% 100% 779 103 882 88.3% 11.7% 100% n Category Transfer Student UW College Other College Re-entry Did Not Transfer Year Freshman Sophomore Junior Senior % 126 90 29 636 881 14.3% 10.2% 3.3% 72.2% 100% 2 316 546 17 881 < 1% 35.9% 62.0% 1.9% 100% Table 2: Students Completing College BASE by Major Category (Percentage Distribution) Fall 2004 and Spring 2005 Combined Category Social Sciences Professional Studies Natural Sciences Humanities Fine Arts Pre-majors/Undeclared n 257 204 179 123 110 11 884 Percent 29.1% 23.1% 20.2% 13.9% 12.4% 1.2% 100% Mean* 282 291 317 325 277 308 * Mean is for the mean Composite score for each major category. Assessment Services page 16 Report: General Education Assessment Program (2004-05) Table 3: Major Field of Study - Students Completing College BASE (Percentage Distribution) Fall 2004 and Spring 2005 Combined Area (P) (F) (N) (P) (N) (F) (S) (N) (N) (S) (P) (H) (S) (N) (H) (H) (H) (N) (S) (H) (N) (N) (F) (H) (S) (S) (S) (S) (P) (H) (F) (S) (O) Major Accounting Art Biology Business Administration Chemistry Communication and the Arts Communication Processes Computer Science Earth Science Economics Elementary Education English Environmental Policy & Planning Environmental Sciences French German History Human Biology Human Development Humanistic Studies Information Sciences Mathematics Music & Applied Music Philosophy Political Science Psychology Public Administration Social Change & Development Social Work Spanish Theatre Urban & Regional Studies Undecided, Other TOTAL n Percent 21 23 31 120 11 45 32 22 3 6 35 29 5 10 1 3 44 81 49 18 7 14 17 6 8 89 24 32 28 22 25 12 11 884 2.38% 2.60% 3.51% 13.57% 1.24% 5.09% 3.62% 2.49% 0.34% 0.68% 3.96% 3.28% 0.57% 1.13% 0.11% 0.34% 4.98% 9.16% 5.54% 2.04% 0.79% 1.58% 1.92% 0.69% 0.90% 10.07% 2.71% 3.62% 3.17% 2.49% 2.83% 1.36% 1.24% 100% F=Fine Arts; H=Humanities; N=Natural Sciences; P=Professional Studies; S=Social Sciences; O=Other Assessment Services page 17 Report: General Education Assessment Program (2004-05) Table 4: College BASE Subject and Cluster Scores (Means, Standard Deviations and Range) Fall 2004 and Spring 2005 Combined Assessment Services page 18 Subject/Cluster n mean SD Range Composite Score 884 294 59 138-446 English Subject 885 280 64 73-437 Reading and Literature 885 274 72 86-427 Writing 885 297 50 165-375 Mathematics Subject 885 321 68 140-475 General Mathematics 885 330 57 155-429 Algebra 885 323 67 134-411 Geometry 885 324 79 140-416 Science Subject 885 284 83 85-468 Laboratory and Field Work 885 288 72 105-441 Fundamental Concepts 885 285 82 103-407 Social Studies Subject History 884 884 282 283 50 63 99-471 148-423 Social Sciences 884 285 66 103-438 Reasoning Competencies Interpretive 884 314 68 144-497 Strategic 884 296 67 102-479 Adaptive 884 271 71 105-445 Report: General Education Assessment Program (2004-05) Table 5: College BASE Skill and Reasoning Competency Scores (Percentage Distribution) Highest to Lowest Percentage of “High” Ratings Fall 2004 and Spring 2005 Combined (n=885) #* 6 8 10 9 5 7 4 20 14 18 11 12 19 15 17 21 3 22 2 16 13 1 Description Practical Applications Using Statistics Equations & Inequalities Evaluating Expressions Conventions of Written English Properties & Notations Writing as a Process Geography Lab/Field Techniques Significance of World Events 2 & 3 Dimensional Figures Geometrical Calculations Significance of U.S. Events Interpreting Results Physical Sciences Political/Economic Structures Understanding Literature Social Science Procedures Reading Analytically Life Sciences Observation/Experimental Design Reading Critically Average Percentage Subject Math Math Math Math English Math English Soc St Science Soc St Math Math Soc St Science Science Soc St English Soc St English Science Science English % High 35% 37% 30% 29% 11% 32% 22% 22% 22% 20% 33% 21% 20% 19% 26% 23% 21% 19% 15% 21% 22% % Medium 54% 51% 56% 56% 72% 60% 60% 59% 59% 61% 46% 56% 55% 55% 45% 47% 49% 51% 54% 47% 43% % Low 11% 12% 14% 15% 18% 8% 18% 19% 18% 18% 21% 22% 25% 25% 29% 29% 30% 30% 31% 32% 36% 16% 23.49% 47% 53.84% 37% 22.65% Reasoning Competencies 23 Interpretive 42% 48% 10% 24 Strategic 20% 58% 22% 25 Adaptive 12% 42% 47% * These numbers correspond to the BASE competencies described in Part II of the Appendix. Assessment Services page 19 % High + Medium 89% 88% 86% 85% 83% 82% 82% 81% 81% 81% 79% 77% 75% 74% 71% 70% 70% 70% 69% 68% 65% 63% 76.77% 90% 78% 54% Report: General Education Assessment Program (2004-05) Table 6: College BASE Subject Scores by Major (Mean Scores) Fall 2004 and Spring 2005 Combined Major Accounting Art Biology Business Administration Chemistry Communication and the Arts Communication Processes Computer Science Earth Science Economics Elementary Education English Environmental Policy & Planning Environmental Sciences French German History Human Biology Human Development Humanistic Studies Information Sciences Mathematics Music & Applied Music Philosophy Political Science Psychology Public Administration Social Change & Development Social Work Spanish Theatre Urban & Regional Studies Undecided, Other Assessment Services page 20 n 21 23 31 120 11 45 32 22 3 6 35 29 5 10 1 3 44 81 49 18 7 14 17 6 8 89 24 32 28 22 25 12 11 Composite English Mean Mean 303 271 262 260 306 283 277 261 357 327 276 265 276 264 324 281 313 288 297 256 299 289 296 318 293 296 297 253 388 322 338 355 309 291 308 275 262 259 307 298 302 269 333 290 285 267 346 322 286 285 275 271 305 308 282 273 284 272 297 298 284 266 269 248 301 293 Math Mean 355 267 330 321 371 301 291 378 378 343 333 290 272 311 380 281 317 347 293 317 363 395 320 364 287 298 313 299 324 311 318 285 318 Science Soc Studies Mean Mean 297 291 260 259 313 297 259 268 385 345 269 269 266 281 317 318 317 318 276 313 279 292 269 308 250 354 324 302 443 405 346 371 295 332 325 283 249 248 296 315 280 295 338 308 276 275 330 370 228 343 267 265 279 320 268 288 271 271 291 288 276 277 257 287 292 301 Report: General Education Assessment Program (2004-05) Table 7: College BASE Subject Scores for Majors with 20+ Students Completing College BASE (Mean Scores and Rank*) Fall 2004 and Spring 2005 Combined Major n Accounting Art Biology Business Administration Communication & the Arts Communication Processes Computer Science Elementary Education English History Human Biology Human Development Public Administration Psychology 21 23 31 120 45 32 22 35 29 44 81 49 24 89 Social Change & Development 32 Social Work 28 Spanish 22 Theatre 25 * Composite English Math Science Soc Studies Mean Rank Mean Rank Mean Rank Mean Rank Mean Rank 303 262 306 277 276 276 324 299 296 309 308 262 305 275 282 284 297 284 6 17 4 13 14 15 1 7 9 2 3 18 16 5 12 10 8 11 271 260 283 261 265 264 281 289 318 291 275 259 308 271 273 272 298 266 11 17 6 16 14 15 7 5 1 4 8 18 12 2 9 10 3 13 355 267 330 321 301 291 378 333 290 317 347 293 313 298 299 324 311 318 2 18 5 7 12 16 1 4 17 9 3 15 14 10 13 6 11 8 297 260 313 259 269 266 317 279 269 295 325 249 279 267 268 271 291 276 4 16 3 17 11 15 2 7 12 5 1 18 14 8 13 10 6 9 291 259 297 268 269 281 318 292 308 332 283 248 320 265 288 271 288 277 7 17 5 15 14 11 3 6 4 1 10 18 16 2 9 13 8 12 Ranks are based on the 18 majors that had 20+ students completing the College BASE during the 2004-05 academic year. Table 8: College BASE Subject Scores: Mean Scores for Five Groups of Majors Fall 2004 and Spring 2005 Combined Majors n* Composite English Math Science Social Studies Fine Arts Humanities Natural Sciences Professional Studies Social Sciences 110 123 179 204 257 277 325 317 291 282 264 315 283 273 273 301 323 357 333 298 270 324 323 276 260 270 341 306 280 300 * Represents the number of students with a declared major in each category. Assessment Services page 21 Report: General Education Assessment Program (2004-05) Table 9: Time Spent on Multiple-Choice Portion of College BASE* (Percentage Distribution) Fall 2004 and Spring 2005 Combined Time Used n 70-75 minutes 76-90 minutes 91-105 minutes 106-120 minutes 121-135 minutes 136-150 minutes 151-165 minutes 166-180 minutes Total Missing time data Percent 88 120 89 113 146 115 102 88 861 24 10% 14% 10% 13% 17% 13% 12% 10% 99% * Students must spend at least 70 but not more than 180 minutes on the exam. Table 10: College BASE Subject and Cluster Scores by Test Completion Time* (Mean Scores) Fall 2004 and Spring 2005 Combined Subject/Cluster Areas Amount of Time (Minutes) 70-75 76-90 91-105 106-120 121-135 136-150 221 241 287 301 317 317 Composite Score 205 228 282 286 301 299 English Reading & Lit 184 211 273 280 297 298 Writing 264 274 302 296 306 303 249 275 308 329 343 344 Mathematics General Math 292 303 315 333 340 336 Algebra 244 276 301 322 337 335 Geometry 226 254 302 317 328 342 183 217 273 293 318 317 Science Lab & Field Work 209 238 276 291 316 318 Fund. Concepts 187 215 277 298 312 309 245 243 287 294 303 308 Social Studies History 266 262 292 296 304 306 Social Sciences 240 237 284 293 301 308 Number of Students 88 120 89 113 146 115 * Students must spend at least 70 but not more than 180 minutes on the multiple-choice portion. Assessment Services page 22 151-165 166-180 318 302 296 306 339 333 332 335 324 320 319 305 305 303 102 318 300 304 307 346 341 340 339 317 317 309 310 311 306 88 Report: General Education Assessment Program (2004-05) Table 11: College BASE Subject and Cluster Scores by Transfer Status (Mean Scores) Fall 2004 and Spring 2005 Combined (n = 881) Subject/Cluster Areas Composite Score English Reading & Literature Writing Mathematics General Mathematics Algebra Geometry Science Laboratory & Field Work Fundamental Concepts Social Studies History Social Sciences Number of Students UW Colleges 293 277 268 298 318 325 310 305 287 291 284 289 290 290 126 Transfer Status Other Re-Entry College 287 309 280 299 273 300 294 302 303 326 316 337 293 306 293 321 276 306 282 313 273 293 289 309 301 303 279 312 90 29 No Transfer 290 274 266 294 320 324 316 308 280 286 279 284 291 281 636 Table 12: Orientation and Feedback Session Attendance (Percentage Distribution) Fall 2004 and Spring 2005 Combined (n = 902) Orientation Feedback Attendance n % of Total 494 55% 367* 41% * Includes individuals who did not attend Feedback Session, but who earned free credit by attending an Orientation Session and earning a composite score of 350+. Assessment Services page 23 Report: General Education Assessment Program (2004-05) Table 13: Factors Motivating Test Performance (Percentage Distribution) Fall 2004 and Spring 2005 Combined Factor Opportunity to receive credit Personal satisfaction Transcript notation Letter from Provost Areas tested important Desire to score better than others Orientation session n 193 193 193 192 193 192 193 Extent of Motivation Very Much A Little Not At All 68% 28% <1% 36% 49% 15% 28% 48% 24% 24% 43% 32% 22% 62% 17% 19% 47% 34% 10% 62% 23% Table 14: Orientation and Feedback Sessions: Student Ratings of Overall Quality (Percentage Distribution) Fall 2004 and Spring 2005 Combined Rating Category Excellent Good Fair Poor TOTAL Orientation Session (Spring 2005)* n Percent 76 36% 113 53% 19 9% 5 2% 213 Feedback Session n 29 122 41 1 193 Percent 15% 63% 21% <1% * The data for the fall 2004 Orientation session was not available, so only the spring 2005 session’s data was included here. Assessment Services page 24 Report: General Education Assessment Program (2004-05) Part II: Proficiencies Measured by the College BASE The College BASE measures proficiencies in English, mathematics, science, social studies and three cognitive competencies: interpretive reasoning, strategic reasoning and adaptive reasoning. The specific skills and factual knowledge in the four curricular areas are defined by twenty-two proficiency statements. The twenty-two proficiencies assessed are as follows: English 1. Read accurately and critically by asking pertinent questions about a text, by recognizing assumptions and implications, and by evaluating ideas. 2. Read a literary text analytically, seeing relationships between form and content. 3. Understand a range of literature, rich in quality and representative of different literary forms and historical contexts. 4. Understand the various elements of the writing process, including collecting information regarding formulation of ideas, determining relationships, arranging sentences and paragraphs, establishing transitions, and revising what has been written. 5. Use the conventions of standard written English. Mathematics 6. Use mathematical techniques in the solution of real-life problems. 7. Use the language, notation, and deductive nature of mathematics to express quantitative ideas with precision. 8. Use the techniques of statistical reasoning and recognize common misuses of statistics. 9. Evaluate algebraic and numerical expressions. 10. Solve equations and inequalities. 11. Recognize two- and three-dimensional figures and their properties. 12. Use the properties of two- and three-dimensional figures to perform geometrical calculations. Science 13. Recognize the role of observation and experimentation in the development of scientific theories. 14. Recognize appropriate procedures for gathering scientific information through laboratory and field work. 15. Interpret and express the results of observation and experimentation. 16. Understand the fundamental concepts, principles, and theories of the life sciences. 17. Understand the fundamental concepts, principles, and the theories of the physical sciences. Social Studies 18. Recognize the chronology and significance of major events and movements in world history. 19. Recognize the chronology and significance of major events and movements in United States history. 20. Recognize basic features and concepts of world geography. Assessment Services page 25 Report: General Education Assessment Program (2004-05) 21. 22. Recognize basic features and concepts of the world’s political and economic structures. Recognize appropriate investigative and interpretive procedures in the social sciences. Reasoning Competencies 23. Interpretive Reasoning is a cognitive process by which we translate information, either remembered or immediately observed, into meaningful terms. Typically, this is accomplished by such activities as paraphrasing, summarizing, or explaining the meaning of particular information. 24. Strategic Reasoning establishes boundaries for information through definition, comparison, classification, and analysis. These boundaries lead to inferences or deductions and reveal relationships lying beneath the surface meaning. 25. Adaptive Reasoning involves the ability to synthesize new rules or theories, to hypothesize a means of testing a proposition, to predict the outcome of causal relationships, or express judgments of value, merit or worth. Assessment Services page 26 Report: General Education Assessment Program (2004-05)