EDUC 107 ASSESSMENT LEARNING 1 FIRST SEMESTER; SY: 2023-2024 Submitted by: GROUP 2 Jenevieve Dela Cuadra Alburo Emmarie Iyen D. Malinao Carlo Jay T. Maynagcot Gwyneth Rose Untalan Honjielyn S. Gumapac Quintino L. Conlas Jr. Hazel Klenk Daganio Jay Sulaiman Sato Reah A. Bantayan Stella M. Forones Ronila Valentin Vince Amarilla Submitted to: Estela Marie Librea Instructor Section 2 Principles of High Quality Assessment Chapter 3 Appropriateness and Alignment of Assessment methods to Learning Outcomes Overview ● Assessment methods and tools should be parallel to the learning targets or outcomes to provide learners with opportunities that promotes deep learning. Taxonomy of Learning Domains ● Learning outcomes are statements of performance expectations: cognitive, affective and psychomotor. ● Importantly, all learning outcomes must be capable of being assessed and measured. A. Cognitive (knowledge-based) ● ● ● ● A formal, systematic procedure for gathering information. A test is a tool comprised of a set of questions. An instrument used to measure a construct and make decisions. Educational tests may be used to measure the learning progress of a student which is formative in purpose. ● Test are the most dominant form of assessment. ● Test are traditional assessments. B. Psychomotor (Skills-based) ● The psychomotor domain on physical and mechanical skills involving coordination of brain and muscular activity. The combination of the taxonomies built by Simpson, Dave and Harrow. C. Affective (VALUES, ATTITUDE, AND INTEREST) ● It tackles the question, “What actions do I want learners to think or care about?” ● Includes factors such as student motivation, attitudes, appreciation, and values. Krathwohl Affective Domain “The Taxonomy is ordered internalization” according to the principle of Internalization – refers to the process whereby a person’s affect toward an object passes from a general awareness level to a point where the affect is internalized and consistently guides the person’s behavior. TABLE 3.3 TAXANOMY OF AFFECTIVE DOMAIN (Krathwohl, et. al., 1964) Types of Assessment Methods Categorized according to the nature and characteristics of each method. McMillan (2007) identified four categories: 1. Selected - Response 2. Constructed – Response 3. Teacher Observation 4. Student Self – Assessment 1. Selected – Response Format -includes true – false, matching type, and multiple choice. 2. Constructed – Response Format - demands that students create or produce their own answers in response to a question, problem, or task. 2.1. Brief – Constructed Response Items - short response from students. 2.2. Performance Assessment - require students to perform a task. - product – based or skills – oriented. Example of Product based: - Written Reports, Projects, Portfolio, Reflection Papers, and Journals. - Speech, teaching demonstration, debate, recital, and role play. 2.2.1 Authentic or Alternative Assessments - students are required to demonstrate what they through activities, problem, and exercises. can do Crehan (1991) - performance assessments can result to better integration of assessments with instruction, greater focus on higher order thinking skills, increased motivation level in the learning process, and improved instructional and content validity. 2.3. Essays Assessments - involves answering a question or proposition in written form. Restricted – Response - constraints to the content and nature of the responses. Extended - Response - responses are longer and more complex. 2.4. Oral Questioning - is a common assessment method during instruction to check on student understanding. 3. Teacher Observations - are a form of on – going assessment, usually done in combination with oral questioning. 4. Student Self-Assessment - A process where the students are given a chance to reflect and rate their own work and judge how well they have performed in relation to a set of assessment criteria. MATCHING LEARNING TARGETS WITH ASSESSMENT METHODS Constructive Alignment - Constructive alignment provides the “how-to” by verifying that the teachinglearning activities and the assessment task activate the same verbs as in the intended learning outcomes. Learning Target - A learning target is defined as a description of performance that includes what learners should know and be able to do. Knowledge and Simple Understanding - This pertains to mastery of substantive subject matter and procedures. - In the revised Bloom’s taxonomy, this covers the lower order thinking skills of remembering, understanding and applying. Deep Understanding and Reasoning - This involves higher order thinking skills of analyzing, evaluating and synthesizing. Skills - Performance assessment are suited applications with less-structured problems where problem where problem identification; collection, organization, integration and evaluation of information; and originality are emphasized (Miller, Linn & Gronlund, 2009). Product - A product is a substantial and tangible output that showcases a student’s understanding of concepts and skills and their ability to apply, analyze, evaluate and integrate those concepts and skills. Affect - affects pertains to attitudes, interests and values students manifest. - in a study conducted by Stiggins & Popham (2009), there are two affective variables influenced by teachers who employ assessment formatively in their classes; academic efficacy and eagerness to learn. Chapter 4: Validity and Reliability Validity Validity-is a term derived from the Latin word validus, meaning strong. In view of assessment, it is deemed valid if it measures what is supposed to. In contrast to what some teachers believe, it is not a property of test. It pertains to the accuracy of the inference teachers make about students based on the information gathered from assessment. A. Content-Related Evidence - Content-related evidenced for validity pertains to the extent to which the test covers the entire domain of content. Instructional Validity - The extent to which an assessment is systematically sensitive to the nature of instruction offered. TABLE OF SPECIFICATION (TOS) - The TOS is prepared before developing the test. It is the test blueprint that Identifies the content area and describes the learning outcomes at each level of the cognitive domain. Table 4.1 Sample Table of Specifications (Notar et. al., 2004) Table 4.2 Time Requirements for Certain Assessment Tasks Six Elements in TOS Development 1. Balance among the goals selected for the examination. 2. Balance among the levels of learning. 3. The test format. 4. The total number of items. 5. The number of items for each goal and level of learning. 6. The enabling skills to be selected from each goal framework. B. Criterion-Related Evidence ● It refers to the degree to which test scores agree with an external criterion as such, it is related to the external validity. ● It examines the relationship between an assessment and another measure of the same trait. Three Types of Criteria 1. Achievement test scores 2. Ratings, grades and other numerical judgements made by the teacher. 3. Carrere data Two Types of Criterion-Related Evidence 1. Concurrent Validity - provides an estimate of a student's current performance in relation to a previously validated or established measure. 2. Predictive Validity - pertains to the power or usefulness of test scores to predict future performance. C. Construct – Related Evidence ● It is an assessment of the quality of the instrument used. ● It measures the extent to which the assessment is meaningful measure of an unobservable trait or characteristics. (McMillan, 2007) Types of construct-related evidence ❑ Theoretical ❑ Logical ❑ Statistical Two Methods of Establishing Construct Validity ❑ Convergent validity - occurs when measures of constructs that are related are in fact observed to be related. ❑ Divergent validity -occurs when constructs that are unrelated are in observed not to be. Unified concept of Validity ❑ integrates considerations of content, criteria, and consequences into a construct framework for the empirical testing of rational hypotheses about score meaning and theoretically relevant relationships. (Messick, 1995) Six distinct aspects of construct validity ▪ ▪ ▪ ▪ ▪ ▪ Content - to content related evidence which calls for content relevance and representativeness. Substantive - pertain to the theoretical construct and empirical evidences. Structural – assess how well the scoring structure matches the construct domain. Generalizability – examine how score properties and interpretations generalize to and across populations groups, context and tasks. External – include convergent and discriminant evidences taken from multitraitMultimethod studies. Consequential – pertain to the intended and unintended effects of assessment on teaching and learning. Validity of Assessment Methods 1. The selected performance should reflect a valued activity. 2. The completion of performance assessments should provide a valuable learning experience. 3. The statement of goals and objectives should be clearly aligned with the measurable outcomes of the performance activity. 4. The task should not examine extraneous or unintended variables. 5. Performance assessments should be fair and free from bias. Treats to Validity 1. Unclear test directions 2. Complicated vocabulary and sentence structure 3. Ambiguous statements 4. Inadequate time limits 5. Inappropriate level of difficulty of test items 6. Poorly constructed test items 7. Inappropriate test items for outcomes being measured 8. Short test 9. Improper arrangement of items 10. Identifiable pattern of answers. Suggestion for enhancing validity • • • • • • • Ask others to judge the clarity of what you are assessing. Check to see if different ways of assessing the same thing give the same result. Sample a sufficient number of examples of what is being assessed. Prepare a detailed table of specifications. Ask others to judge the match between the assessment items and the objectives of the assessment. Compare groups known to differ on what is being assessed. Compare scores taken before to those taken after instruction. • • • • • • • Compare predicted consequences to actual consequences. Compare scores on similar, but different traits. Provide adequate time to complete the assessment. Ensure appropriate vocabulary, sentence structure and item difficulty. Ask easy question first. Use different methods to assess the same thing. Use only for intended purposes. Reliability ● It talks about reproducibility band consistency in methods and criteria. ● An assessment is said to be reliable if it produced the same results if given to an examinee on two occasions. Types of Reliability 1. Internal reliability - assesses the consistency of result across item within the test. 2. External reliability - gauges the extent to which a measure varies from one use to another. Source of reliability evidence 1. Stability - the test-retest reliability correlates score obtained from two administrations of the same test over a period of time. 2. Equivalence - parallel form of reliability ascertain the equivalency of forms. 3. Internal consistency - a student who has a mastery learning will get all or most of the items correctly while a student who knows little or nothing about the subject matter will get all or most of the items wrong. 4. Scorer or rater consistency - People do not necessarily rate in a similar way. 5. Decision consistency - Decision consistency describe how consistent the classification are rather than how consistent the score are ( Nitko & Brookhart, 2011). Measurement Errors - Measurements errors can be caused by examinee-specific factors like fatigue, boredom, lack of motivation, momentarily lapse of senses and carelessness. Reliability of Assessment - Between a well constructive objectives test and performance assessment, the former has better reliability (Miller, Linn & Gronlund 2009; Harris,1997). Ways to improve reliability of assessment result • • • • • • • • • Lengthen the assessment procedure by providing more time, more questions, and more observation whenever practical. Broaden the scope of the procedure by assessing all the significant aspects of the largest learning performance. Improve objectively by using systematic and more formal procedure for scoring students’ performance. Use multiple marker by employing inter-rater reliability Combined results from several assessment especially when making crucial educational decision. Provide sufficient time to students in completing the assessment procedure. Teach students how to perform their best by providing practice and training to students and motivating them. Match the assessment difficulty to the students ability level by providing task that are neither too easy nor too difficult. Differentiate among students by selecting assessment task that are distinguish or discriminate the best from the least able students. Chapter 5: Practicality and Efficiency Practicality - means “useful” - it can be used to improve classroom instruction and for outcomes assessment purposes. - it also refers to intelligent utilization of classroom time. Efficiency - is the ability to do something without wasting materials, time or energy. Factors on Practicality and Efficiency that contribute to High Quality Assessment Familiarity with the Method - the teacher should be familiarized with the method of assessment. - a rule of thumb is that simple learning targets require simple assessment, complex learning targets demands complex assessment (Nitko & Brookhart, 2011). Time Required - it is best that assessments are quick to develop but not to the point of reckless construction - assessment should allow students to respond readily but not hastily. - assessment should also be scored promptly but not without basis. - assessments’ reusability. Ease in Administration - easy to administer - administration should have clear instructions and steps, and less time and effort should be required. Ease of Scoring - scoring must correspond to your strategy and objectives. - performance assessment it is more practical to use rating scales and checklists rather than writing extended individualized evaluations. Ease of Interpretation - refers to how easily the assessment results can be understood and analyzed. - This is important because educators and stakeholders need to be able to make sense of the data and use it to inform decision-making. Note: Assessment that are clear and provide meaningful feedback make it easier identify areas of strength and areas that are need improvement. Cost - is another crucial consideration in assessment. - Developing and administering assessments can be resource-intensive, so it's important to consider the financial implications. - Cost-effective assessments allow educational institutions to allocate resources efficiently without compromising the quality of the assessment process. In summary, assessments that are easy to interpret and cost-effective provide valuable insights into learning outcomes while optimizing available resources. Such assessments enable educators to make informed decisions and support student growth effectively. Chapter 6: Ethics Overview "Teachers' assessments have important long-term and short-term consequences for students; thus, teachers have an ethical responsibility to make decisions using the most valid and reliable information possible" (Russell & Airasian, 2012, p.21). Students' Knowledge of Learning Targets and Assessments - transparency is defined here as the disclosure of information to students about assessments. - informing students regarding the assessment details, they can adequately prepare and recognize the importance of the assessment. Opportunity to Learn - Fair assessments are aligned with instruction that provides adequate time and opportunities for all students to learn (McMillan, 2007). Prerequisite Knowledge and Skills - - Students may perform poorly in an assessment if they do not possess background knowledge and skills. Classroom illustration: A grade school pupils taught about inverse proportion. Even if they memorize the meaning of proportion, they would not be able to develop a schema if they are not able to connect new information with previous knowledge. A stereotype is a generalization of a group of people based on inconclusive observation of a small sample of this group. Stereotyping is caused by preconceived judgements of a people ones comes in contact with which are sometimes unintended Classroom Illustration: A female student who was told that females are academically interior in Mathematics may feel a certain level of anxiety, and the negative expectation may cause her to underperform in the assessment. Avoiding Bias in Assessment Task and Procedures - Assessment must be free from bias. Fairness demands that all learners are given equal chances to do well (from the task) and get a good assessment (from the rater). There are two forms of assessment: 1. Offensiveness - happens if test-takers get distressed, upset or distracted about how an individual or a particular group is portrayed in the test. 2. Unfair Penalization - harms students’ performance due to test content, not because items and offensive but rather, the content caters to some particular groups from the same economic class race gender etc. leaving other groups at a loss or disadvantage. Accommodating Special Needs - Teachers need to be sensitive to the needs of student. Certain accommodation must be given especially for those who are physically or mentally challenge. The legal basis for accommodation is contained in Sec. 12 of Republic Act 7277 entitled "An Act Providing for the Rehabilitation, Self-Development and Self-Reliance of Disabled Person and their Integration into Mainstream of Society and for other purpose" Six Categorizing of Accommodation: ▪ Presentation (repeat directions, read aloud, used large print, braille) ▪ Response (mark answers in test booklet, permit responses via digital recorder or computer, use reference material like dictionary) ▪ Setting (study carrel, separate room, preferential seating, individual or small group, special lighting) ▪ Timing (extended time, frequent breaks, unlimited time) ▪ Scheduling (specific time of day, subtests in different order, administer test in several timed sessions) ▪ Others (special test preparation techniques and out-of-level test)