Joanna Chappell Results of a unit test for an 8th-grade French II class Nom: ______________________________________________ Date: _____________________ Points : ________ / 56 ________% Examen Unité 3: La santé Part I. Cerclez la meilleure réponse pour répondre à la question ou compléter la phrase. Circle the best response to answer the question or complete the sentence. (2 points per correct response) 1. You are walking down the street in France and see this sign pictured below. What establishment does it indicate? Unit 3: Global Health Challenges A. B. C. D. A doctor’s office A hospital A pharmacy A Red Cross headquarters French II • 18 Questions • 56 points total • 19 select-response items • 9 supply-response items 2. Il a le nez qui coule. Il a besoin d’__________________________. A. B. C. D. un mouchoir une allergie une infection un pantalon 3. Elle a de la température. Elle a ____________________________. A. B. C. D. une sinusite une angine un rhume une fièvre 4. Je ne peux pas manger des cacahuètes*. J’ai ______________________. A. B. C. D. une infection la grippe une allergie mal à la tète *cacahuète Valid and reliable Changes made based on instruction All Correct Incorrect 20 0 2 18 20 0 19 1 19 1 20 0 20 0 7 13 (avg point loss: 0.92/2) 1 19 (avg point loss: 0.76/2) 13 7 (avg point loss: 1.2/2) 11 9 (avg point loss: 0.94/2) 5 15 (avg point loss: 1.3/2) 8 12 (avg point loss: 0.96/2) 5 15 (avg point loss: 1.3/2) 20 0 (everyone got all 7 correct) 6 14 (avg point loss: 2.5/5 points) 1 19 (avg point loss: 2.3/8 points) 2 18 (avg point loss: 1.6/8) average median min max With #2 Without #2 44 46 69 71 82 84 79 81 85 87 75 75 96 98 87 87 85 87 90 92 93 95 85 87 66 68 66 68 82 84 88 90 84 86 52 54 71 73 84 86 78.15 79.95 83 85 44 46 96 98 A B C D F 3 9 3 3 2 Average Score: 78.15% Median Score: 83% Highest Score: 96% Lowest Score: 44% A (90-100) 3 B (80-89) 9 C (70-79) 3 D (60-69) 3 F 2 (0-59) 1. 100% 12. 53% * 2. 11% 13. 71% * 3. 100% 14. 54% * 4. 95% 15. 100% * 5. 95% 16. 65% * • Select-response vs. supply-response 6. 100% 17. 71 * • Questions with multiple points 7. 100% 18. 81% * 8. 70%* 9. 55% * 10. 79% * 11. 79% * * Indicates the average score on test items that had multiple points per item. COGNITIVE LEVEL CONTENT Body Parts Vocabulary Knowledge Comprehension 2, 3, 4, 16 Evaluation 17, 18 + (understand) 5, 6, 7 + (describe, ask about, answer) 8, 9, 10, 11, 17, 18 + (describe, ask about, answer) Past tense of verbs with “être” Eating habits Synthesis 17, 18 + (express, ask about, answer) + (identify) Personal Care Routine Vocabulary and Reflexive Verbs (present and past tense) Analysis (ask about, answer) + (identify) 15 Health Issues and Disease Vocabulary Application 12, 13, 14 (ask about, answer) (identify) Cultural Differences in Healthcare + (compare, connect, ask, answer) 1 Formal/Informal Register (use) (identify) 17, 18 (interpersonal/presentational) Oral and written communications in French (interpretive) Importance of learning French/Use of French outside the classroom (ask about, answer) (distinguish between) (exchange, sustain) Oral and written communications in French 18 17 (understand) Students demonstrated mastery of the ability to recall and understand relevant vocabulary and verb structures. • Students were not as successful when called upon to apply their knowledge of verb structures. (compare) 17, 18 + (develop awareness) • (use verbal/nonverbal cues) (explore why) (identify reasons why) • Dr & Mrs Vandertramp verbs • Reflexives followed by a direct object Allison Score: 75% (C) Items missed: 8 (75%), 9 (25%), 10 (50%), 11 (50%), 12 (25%), 13 (75%), 14 (25%), 16 (60%), 17 (81%), 18 (63%) Theresa Score: 93% (A) Items missed: 2, 9 (75%), 17 (94%), 18 (88%) Jonathan Score: 84% (B) Items missed: 2, 8 (75%), 10 (50%), 11 (75%), 16 (60%), 17 (75%), 18 (88%) * The percentages indicate each student’s score on an item where multiple points were awarded for a single question. COGNITIVE LEVEL CONTENT Body Parts Vocabulary Knowledge Comprehension 2 2, 3, 4, 16 16 + (understand) 5, 6, 7 Past tense of verbs with “être” Eating habits Synthesis Evaluation 17 17 17, 18 18 18 + (express, ask about, answer) + (identify) Personal Care Routine Vocabulary and Reflexive Verbs (present and past tense) Analysis (ask about, answer) + (identify) 15 Health Issues and Disease Vocabulary Application 17 17 17, 18 18 18 + (describe, ask about, answer) 8 8, 9 9, 10 10, 11 11, 17 17 17, 18 18 18 + (describe, ask about, answer) 12, 13, 14 (ask about, answer) (identify) • (compare) 17 17 17, 18 18 18 + (develop awareness) Cultural Differences in Healthcare + (compare, connect, ask, answer) 1 Formal/Informal Register (use) (identify) 17 17 17, 18 18 18 (interpersonal/presentational) Oral and written communications in French (interpretive) Importance of learning French/Use of French outside the classroom (ask about, answer) (distinguish between) (exchange, sustain) Oral and written communications in French 18 18 18 17 17 18 (understand) Key Allison Theresa Jonathan (use verbal/nonverbal cues) (explore why) (identify reasons why) Commonalities • All of the students struggled with the higher-order supplyresponse items. • Vocabulary issues (#2, #16) Threats to reliability Interrupted instruction and loss of class time due to snow days. I designed the test, but my CT taught the unit. Item #1 was not printed in color. Threats to validity Changes were made to the test based on changes in instruction. The original item #18 (asking students to describe Doctors without Borders) was removed. Cultural approaches to healthcare was still represented by item #1, but not to the extent indicated by the ILOs. Future Assessments Put grading system for passé compose (#8-14) and question section (#17) on the test itself Use holistic grading for #17-18 Print in color or project image of #1 on the board in color Language tests and the focus on accuracy Many schools are moving to Integrated Performance Assessments instead of traditional unit tests. Did students learn? 90% pass rate Assessment provided information on the degree to which students have or have not mastered the vocabulary and structures from the unit. Instructional Decisions Short-term Review Dr & Mrs Vandertramp verb series (#12-14) Review past tense of reflexive verbs, focusing on past participle agreement (#8-11) Review question formation (#17-18) Long-term Allow more time in the future for instruction of different forms of passé composé. Review and recycle vocabulary constantly throughout the unit. Plan more opportunities for students to use higher-order thinking skills and produce language in class. Integrate more culture throughout the unit. This assessment, which was created for an 8th-grade French II class as a unit test, initially had high validity and reliability. Over the course of the unit, and with the assessment itself, certain threats to validity and reliability developed, but were handled by the instructor or did not severely threaten possible inferences about student learning. As a result the assessment maintained a reasonably high degree of validity and reliability. Because of this we can make reasonably accurate inferences about the nature and degree of student learning, and we can use those inferences to better shape future instruction. By completing this test construction and analysis project I came away with a better understanding of what makes a test valid and reliable and what specifically I can do to improve my instruction and student learning.