ANALYZING ASSESSMENTS CARRIE KIRK G R A N D C A N YO N U N I V E R S I T Y : R E A 5 1 0 - 0 5 0 0 APRIL 22, 2020 TEAL AND SLT RUBRIC T E A L THESIS EVID ENCE ANALYSIS LANGUAGE In M at h , w e say CL A IM In M at h , w e say EV ID EN CE In M at h , w e say Reason in g In M at h , w e say Voc ab u lar y of t h e D isc ip lin e In English, we say THESIS In English, we say In English, we say In Science, we say CLAIM In Science, we say EVIDENCE In History, w e sa y THESIS TEXTUAL EVIDENCE In History, w e sa y EVIDENCE ANALYSIS In Science, we say Justification & Reasoning In Science, we say Concepts In History, w e sa y ANALYSIS In History, w e use ACADEM IC (p rim a ry & second a ry sources) In ________, we say ___________ In ________, we say ___________ In English, we use lit er ar y & r het or ical devices In ________, we say ___________ & Vocabular y & FORM AL La ng ua g e In ________, we say ___________ Sult ana High School 2019 VALIDITY AND RELIABILITY • Reliability • Validity – This is a valid assessment as measures what it intends to measure (Rubin, B, 2018) the ability to relate literacy to writing. – This rubric focuses on the different vocabulary used across the disciplines in order to score writing. – It is valid as it is scoring a student’s ability to write within each discipline and in turn can be used by all contents to see if a student has mastery or weakness in their writing. (Rubin, B, 2018) – This is a reliable assessment it is consistent, as this rubric to looking for how a student can write about a specific topic but across different disciplines. – The rubric is reliable because it adheres to the idea of test-retest reliability, which is characterized by the replicability of results. That is to say, if a group of students takes a test twice, both the results for individual students, as well as the relationship among students’ results, should be similar across tests (Rubin, B, 2018). BIAS AND CONCERNS • This assessment does not lend to having any biases however it can become present whenever one or more items on a test offend or unfairly penalize students because of those students’ personal characteristics such as race, gender, socioeconomic status, or religion(Popham, W. J., 2012). • A concern regarding biases comes with teacher’s biasness about a student or a student’s abilities. If a teacher is not properly trained on how to use the rubric to assess their subject, the data may become skewed as to how it was originally designed IREADY VALIDITY AND RELIABILITY • Validity – The Validity of this assesses is sound as it fits into the idea of a construct validity. It ensures the interpretability of results, thereby paving the way for effective and efficient data-based decision making by school leaders (Rubin, B, 2018). – The data reported from this test allows the teachers to determine what next steps in instruction to take with their students. • Reliability – Reliability is sensitive to the stability of extraneous influences, such as a student’s mood (Rubin, B, 2018). – iReady is not totally reliable as student motivation and attitude can highly skew the results. – Many times in conversations with students they explain that they rushed, guessed or do not try . – As a result the data is not a true determination of their ability BIAS AND CONCERNS • Since this assessment is ability based kindergarten through 12th grade, and adapts as the student shows mastery there is not much bias to the assessment. • The greatest concern is students not taking the assessment seriously and not showing their true abilities. • Also as the students show mastery in different literacy areas, the test gets progressively harder and students start to show burn out. • Another issue is teacher buy in. When the teachers “just do it” the students do not value the information. • Additionally, students are able to have the assessment reset, this in turn does not give equal time to all students. CAASPP: CALIFORINA ASSESSMENT OF STUDENT PERFORMANCE AND PROGRESS VALIDITY AND RELIABILITY • Validity – The CAASPP is a valid assessment as it assesses a specific content area and is gathering data basic on that content. – There is an accuracy of the inferences that are drawn about the test taker • Reliability – Split-Half Reliability • – The data is based on evidence for content validity, predictive validity, construct validity (Understanding Standardized Assessment, 2020). – Scores are compared over a period of time, a student’s growth is seen if the student has been in the same district of a period of time. This allows for teachers to follow up and see a “true level” of a student’s abilities. Administer a single test to a group of students, create two scores by dividing the test in half, and measure the extent to which the rankings change from one half to the other (Understanding Standardized Assessment, 2020) – Test-Retest Reliability • • Administer the same test to the same people on two occasions and measure the extent to which the rankings change over time (Understanding Standardized Assessment, 2020) – Alternate-Form Reliability • • Administer two equivalent forms of a test to the same group of students at the same time and compare the results (Understanding Standardized Assessment, 2020) BIAS AND CONCERNS • The CAASPP is low on bias because every person who takes the test responds to the same items under the same conditions (Understanding Standardized Assessment, 2020). • The answers are evaluated according to the same scoring standards (Understanding Standardized Assessment, 2020) • The scores are interpreted through comparison to the scores obtained from a group that took the same test under the same conditions or through comparison to a predetermined standard Understanding Standardized Assessment, 2020). • Some concerns with the CASAPP is the how much of a HIGH STAKES assessment it is. Teachers, administrators as well as the students are under a lot of pressure to perform well. RECOMMENDATION AS MOST EFFECTIVE, RELIABLE , VALID, AND BIAS-FREE • TEAL/SLT Rubric would be the most effective, reliable, valid and bias-free assessments as they are given in a more familiar environment and more frequently. The assessment is used in everyday writing as well as data collecting assessments, therefore students are more comfortable with it and do not feel the pleasure of a high stakes test. • Schools interested in establishing a culture of data are advised to come up with a plan before going off to collect it. They need to first determine what their ultimate goal is and what achievement of that goal looks like. An understanding of the definition of success allows the school to ask focused questions to help measure that success, which may be answered with the data(Rubin, B, 2018). The TEAL/SLT allows for this. REFERENCES • Popham, W. J. (2012) MASTERING ASSESSMENT: A SELF-SERVICE SYSTEM FOR EDUCATORS Assessment Bias: How to Banish It Second Edition Retrieved from http://ljournal.ru/wp-content/uploads/2016/08/d-2016-154.pdf. (2016). doi: 10.18411/d-2016154 • Rubin, B. (2018, September 10). Importance of Validity and Reliability in Classroom Assessments. Retrieved from https://www.thegraidenetwork.com/blog-all/2018/8/1/the-twokeys-to-quality-testing-reliability-and-validity • Understanding Standardized Assessment(2020) Retrieved from https://www.shsu.edu/aao004/documents/15_006.pdf
