STANDARDIZED TESTS Definition: 1. A test with specific tasks and procedures so that comparable measurements may be made by testers working in different geographical areas. 2. A test for which norms on a reference group, ordinarily drawn from many schools or communities, are provided. Examples: Examples of standardized tests being used for assessment: Major Field Achievement Test [MFAT] in cognate areas or for General Education Major field examinations are administered in a variety of disciplines. They often are given to students upon or near completion of their major field of study. These tests assess the ability of students to analyze and solve problems, understand relationships, and interpret material ETS’ Academic Profile ACT's Collegiate Assessment of Academic Proficiency (CAAP) ACT - COMP (College Outcome Measures Program) Measures knowledge and skills acquired by students in general education courses. See http://www.mcli.dist.maricopa.edu/ae0/al_tools.html and http://ericae.net/testcol.htm, for listings of broad-based standardized tests and standardized tests within disciplines. Some specialized standardized tests in: Critical Thinking o Test of Critical Thinking Ability o Watson-Glaser Critical Thinking Appraisal o Ennis, Robert H. “An Annotated List of Critical Thinking Tests” June 2002 http://faculty.ed.uiuc.edu/rhennis/TestListM9Y01.html Critical Thinking/Problem Solving & Writing o NPEC Sourcebook of Assessment Information http://nces.ed.gov/npec/evaltests/ Science/Math o o “Field-tested learning Assessment Guide http://www.flaguide.org/tools/tools.htm Math Forum @ Drexel-Library-Assessment/Testing http://mathforum.org/library/ed_topics/assessment There are also the exams for licensure or certification in specific programs. Costs: Costs vary depending on the selected standardized test. But, keep in mind that there are annual costs associated with administering standardized tests. And, in some ways, using a pre-tested, validated and normed test costs less then the time put into developing, administering and assessing locally developed tests. Advantages: Easy to administer to groups Require minimal training for test administrators Very little professional time is needed beyond faculty efforts to analyze examinations results and develop appropriate curricular changes that address the findings. Tests are traditionally given to students in large numbers and do not require faculty involvement when exams are taken by students. Have documentation of reliability (consistency of results over time) and predictive validity (ability to forecast students' performance on a criterion, such as first-year GPA in graduate school). In most cases, nationally developed tests are devised by experts in the discipline. Institutional comparison of students is possible. Can assist departments in determining programmatic strengths and weaknesses when compared to other programs and national data. In most cases, standardized testing is useful in demonstrating external validity. Funding sources accept them as part of the documentation of program accountability Provide the ability to baseline and benchmark Disadvantages: Potential of teaching to the test and thus narrowing the curriculum Can promote an emphasis on lower-order thinking if the test questions aren’t scrutinized for higher order thinking. May not capture incremental changes in learning over short periods of instructional time Lack of a guarantee that the instrument will cover a program's learning objectives. Students may not be motivated to do well on exams, and incentives may be required. It may be difficult to schedule time for students to take exams. Choosing the best exam may be time-consuming Standardized tests can be expensive to administer on a yearly basis. By their very nature, tend to be generic and not well focused on specific skills or competencies. Language, literacy, and culture are not treated distinctly; that is, they do not tell us whether a learner has trouble with an item because he or she is unfamiliar with the cultural notion underlying the task They do not reflect what has been taught and do not capture all the learning that has taken place, especially in the affective domain. They focus on pencil and paper tasks, and therefore do not provide opportunities for literacy learners to show what they can do in "real life". Implementation Suggestions: The results of testing will have meaning to the learners and instructors only if the test content is related to the goals and content of the instruction and instructional time is sufficient Clearly identify the purpose of the assessment (why the learners are being assessed) and what learning is to be assessed Explore the variety of choices by first reading the commercial literature. Rank order choices and review actual exams. Consider sample size, logistics and incentives. Strive for administrator, faculty and student buy-in. If an institution is using multiple measures, there can be a place in the assessment plan for using standardized tests. Ensure that adequate resources are available to carry out the assessments (e.g., enough materials, comfortable environment, adequately trained administrators and scorers). Be aware of the limitations of the assessments selected. Remember that assessment is not an end in itself, but a means to an end. Share assessment results with learners and instructors, as well as with administrative staff and funders and the results as a basis for decisions. If the standardized instruments do not measure the cognitive areas articulated in mission and purposes statements (e.g., ethnically responsible decision-making), then it’s best to locally develop relevant instruments. Evaluators suggest that passing rates on licensure or certification exams do not per se provide direct evidence of the level of achievement in the specific area of student competencies or skills tested. Unless licensure or certification exam scores are supplemented by information about how well students did in each of the subject matter areas covered in the exam, results are not likely to be useful to academic units that intend to use the scores as an indicator of student learning. Recommendation: Direct measures of student learning yield useful information about the value added to a student's learning by the general education program, the major, or professional program, especially when the results from multiple measures are triangulated and are compared with (1) baseline data and/or with (2) data from other measures taken over time. Bibliography/Resources: “Assessment Instruments and Methods Available to Assess Student Learning in the Major.” UW-Madison Assessment Manual. April 2000. <http://www.wisc.edu/provost/assess/manual/manual2.html> “Direct Measures of Student Learning.” NAU Assessment. Office of the Vice Provost for Undergraduate Studies. January 28, 2002. <http://jan.ucc.nau.edu/~libst-p/assessment/ast_resources/index/amm1.cfm> “Tests (Standardized and Locally Developed.” Institutional Effectiveness. Assessment. University of Kentucky. July 19, 2002. < http://www.uky.edu/Assessment/mtest.shtml> Van Kollenburg, Susan E., ed. A Collection of Papers on Self-Study and Institutional Improvement: Proceedings of the 107th Annual Meeting of the North Central Association: Engaging the Future: Vision, Values, and Validation in the New Educational Marketplace. Chicago, The Higher Learning Commission. 2002.