WRITING MULTIPLE CHOICE EXAM ITEMS DOUGLAS A. BERNSTEIN University of South Florida Southampton University www.douglasbernstein.com douglas.bernstein@worldnet.att.net SOME MYTHS ABOUT MCQs USEFUL MAINLY FOR TESTING BASIC, FACTUAL INFORMATION NOT USEFUL FOR ASSESSING LEARNING AT HIGHER COGNITIVE LEVELS BLOOM’S TAXONOMY OF COGNITIVE SKILLS SOME MYTHS ABOUT MCQs USEFUL MAINLY FOR TESTING BASIC, FACTUAL INFORMATION NOT USEFUL FOR ASSESSING LEARNING AT HIGHER COGNITIVE LEVELS NOT AS GOOD OVERALL AS ESSAY EXAMS EASY TO WRITE NO SPECIAL PRACTICE NEEDED THE TRUTH ABOUT MCQs THEY CAN, INDEED, TEST MEMORY OF FACTUAL INFORMATION BLOOM’S LEVEL Remembering DEFINITION Recall or recognize information, ideas, and principles as presented. SAMPLE VERBS List, label, name SAMPLE QUESTIONS What are the three main parts of _____?; What are the steps involved in____? A “REMEMBERING” ITEM Who wrote War and Peace? a. Rachmoninov b. Tolstoy c. Dostoyevsky d. Solzhenitsyn THE TRUTH ABOUT MCQs THEY CAN, INDEED, TEST MEMORY OF FACTUAL INFORMATION THEY CAN ALSO TEST THE ABILITY TO UNDERSTAND, APPLY, ANALYZE, AND EVALUATE INFORMATION MCQs FOR ALL LEVELS LEVEL REMEMBERING UNDERSTANDING DEFINITION Recall or recognize information, ideas, and principles as presented. Comprehend or interpret information SAMPLE VERBS List, label, name Explain, summarize describe, illustrate SAMPLE QUESTIONS What are the three main parts of _____; What are the steps involved in____? How is __ related to __?; What themes do you see in __? AN “UNDERSTANDING” ITEM During a sleep study, you notice that your research participant’s EEG shows brain activity known as spindles and K-complexes. The participant is probably in stage ___ sleep. a. 1 b. 2 c. 3 d. 4 MCQs FOR ALL LEVELS LEVEL REMEMBERING UNDERSTANDING APPLYING DEFINITION Recall or recognize information, ideas, and principles as presented. SAMPLE VERBS List, label, name SAMPLE QUESTIONS What are the three main parts of _____?; What are the steps involved in____? Translate, comprehend, or interpret information based on prior learning. Explain, summarize, describe, illustrate How is __ related to __?; What themes do you see in __? Select, transfer, and use data and principles to complete a problem or task. Use, compute, solve, demonstrate How would you __ on the basis of __ theory? AN “APPLYING” ITEM Naomi asked seven randomly selected people the following question, "On a scale from 0 (not at all) to 9 (definitely), how strongly do you believe in ghosts?" Their responses were as follows: 0, 0, 1, 1, 9, 9, 9. Which measure of central tendency should Naomi use if she wants to convince her friends that most people strongly believe in ghosts? A. Mode C. Median B. Mean D. Range LEVEL DEFINITION SAMPLE VERBS REMEMBERING Recall or recognize information, ideas, and principles as presented. List, label, name UNDERSTANDING Translate, comprehend, or interpret information based on prior learning. Explain, summarize, describe, illustrate APPLYING Select, transfer, and use data and principles to complete a problem or task. Use, compute, solve, demonstrate ANALYZING Distinguish, classify, and relate the assumptions, hypotheses, evidence, or structure of a statement or question. Categorize compare contrast separate SAMPLE QUESTIONS What are the three main parts of _____; What are the steps involved in____? How is __ related to __?; What themes do you see in __? How would you __ on the basis of __ theory? How would you classify __?; What evidence supports __? AN “ANALYZING” ITEM The difference between state and trait anxiety is that a. The former refers to general anxiety and the latter refers to situational anxiety. b. The former refers to situational anxiety and the latter refers to general anxiety. c. The former refers to physiological responses and the latter refers to questionnaire responses. d. The former refers to questionnaire responses and the latter refers to physiological responses. LEVEL DEFINITION REMEMBERING Recall or recognize information, ideas, and principles as presented. UNDERSTANDING Translate, comprehend, or interpret information based on prior learning. APPLYING Select, transfer, and use data and principles to complete a problem or task. ANALYZING Distinguish, classify, and relate the assumptions, hypotheses, evidence, or structure of a statement or question. EVALUATING Appraise, assess, based on standards and criteria. SAMPLE VERBS List, label, name Explain, summarize, describe, illustrate Use, compute, solve, demonstrate Categorize, compare, contrast, separate SAMPLE QUESTIONS What are the three main parts of _____; What are the steps involved in____? How is __ related to __?; What themes do you see in __? How would you __ on the basis of __ theory? How would you classify __?; What evidence supports __? Judge, Which approach recomto __ would mend be most critique, effective, justify and why? AN “EVALUATING” ITEM Injuries to belted passengers increase wherever seat belt laws are passed. Which of the following serves as a logical counterargument to repealing these laws? a. Injuries will fall once people learn to wear belts correctly. b. More people are wearing belts when accidents occur. c. Though injuries increase, fatalities decrease. d. Both B and C THE TRUTH ABOUT MCQs THEY CAN, INDEED, TEST MEMORY OF FACTUAL INFORMATION THEY CAN ALSO TEST THE ABILITY TO UNDERSTAND, APPLY, ANALYZE, AND EVALUATE INFORMATION THEY CAN EVEN TEST CREATIVITY LEVEL REMEMBERING UNDERSTANDING APPLYING ANALYZING EVALUATING CREATING DEFINITION Recall or recognize information, ideas, and principles as presented. SAMPLE VERBS List, label, name SAMPLE QUESTIONS What are the three main parts of _____; What are the steps involved in____? Translate, comprehend, or interpret information based on prior learning. Explain, summarize, describe, illustrate How is __ related to __?; What themes do you see in __? Select, transfer, and use data and principles to complete a problem or task. Use, compute, solve, demonstrate, apply Distinguish, classify, and relate the assumptions, hypotheses, evidence, or structure of a statement or question. Categorize, compare, contrast, separate How would you classify __?; What evidence supports __? Appraise, assess, based on standards and criteria. Judge, recommend, critique, justify Which approach to __ would be most effective, and why? Originate, integrate, and combine ideas into a new product, plan or proposal. Create, design, invent, develop Write an MCQ item at the ___ level of Bloom’s taxonomy. How would you __ on the basis of __ theory? THE TRUTH ABOUT MCQs THEY CAN, INDEED, TEST MEMORY OF FACTUAL INFORMATION THEY CAN ALSO TEST THE ABILITY TO UNDERSTAND, APPLY, ANALYZE, AND EVALUATE INFORMATION THEY CAN EVEN TEST CREATIVITY THEY CAN BE AS GOOD AS ESSAYS GOOD ONES ARE NOT EASY TO WRITE Two researchers tested the hypothesis that college students’ grades and stress are related. One researcher operationally defined stress as the number of minutes spent arguing with others. The other researcher defined stress as the amount of tension at any point in time measured on a 10-point scale. Which of the following statements is accurate? a. Because their hypotheses are identical, the two researchers should have similar results. b. The only valid definition is the number of minutes spent arguing with others because it is the only objective measure. c. The difference in operational definitions of stress could lead to quite different results. THE TRUTH ABOUT MCQs THEY CAN, INDEED, TEST MEMORY OF FACTUAL INFORMATION THEY CAN ALSO TEST THE ABILITY TO UNDERSTAND, APPLY, ANALYZE, AND EVALUATE INFORMATION THEY CAN EVEN TEST CREATIVITY THEY CAN BE AS GOOD AS ESSAYS GOOD ONES ARE NOT EASY TO WRITE USEFUL GUIDELINES ARE AVAILABLE BASIC CHARACTERISTICS OF GOOD MCQ TESTS CONTAIN AT LEAST 60 ITEMS (FOR MIDTERM AND FINAL EXAMS) CAREFULLY PLANNED Fritz's computer screeches like a monkey every time he gets new e-mail. One day, Fritz is watching "Three Monkeys and a Banana" on HBO, but when a monkey screeches on the show, Fritz does not go to check his e-mail because he realizes the sound is coming from the TV set, not his computer. In other words, Fritz has made a A. hit. C. false alarm. B. miss. D. correct rejection. A TABLE OF SPECIFICATION COGNITIVE LEVEL CONTENT REMEMBERING UNDERSTANDING APPLYING Classical Conditioning 1 Shaping 1 1 1 Reinforcement 1 Observational Learning 1 Latent Learning Cognitive Processes in Learning 2 1 1 1 1 BASIC CHARACTERISTICS OF GOOD MCQ TESTS CONTAIN AT LEAST 60 ITEMS (FOR MIDTERM AND FINAL EXAMS) CAREFULLY PLANNED WRITTEN CAREFULLY CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS NOT SO GOOD William James a. pioneered the functionalist school in psychology b. developed introspection to measure consciousness. c. was concerned with perceptual principles. d. was the founder of the behaviorist approach. BETTER William James is best known for founding the _______ approach to psychology. a. functionalist b. structuralist c. behaviorist d. evolutionary CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS RELEVANT INFORMATION NOT SO GOOD Shanelle and Audre were each graduating from high school. During their commencement speeches, Shanelle spoke about how excited she was to be going to Harvard because she had worked so hard to reach her goals and was looking forward to moving far away to "spread her wings." Audre said she could not have "made it" without her family's and friends' support and was delighted to be able to attend a nearby university so she can stay close to home. According to your text, Shanelle is most likely _________ and Audre is most likely __________. A. from a collectivist culture; from an individualist culture B. from an individualist culture; from a collectivist culture C. resolving the identity crisis; resolving the intimacy crisis D. resolving the intimacy crisis; resolving the identity crisis BETTER Shanelle is excited to go to a faraway university so that she can “spread her wings" and “be herself.” Audre is happy to study at a local university where she can be sure of having the support of her family and friends. According to your text, Shanelle is most likely ___________ and Audre is most likely _____________. A. from a collectivist culture; from an individualist culture B. from an individualist culture; from a collectivist culture C. resolving the identity crisis; resolving the intimacy crisis D. resolving the intimacy crisis; resolving the identity crisis CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS RELEVANT INFORMATION MINIMAL USE OF NEGATIVES NOT SO GOOD Tyler is two days old, healthy and wide awake at the moment. Tyler is unlikely to A. be able to support his weight by hanging with his hands from your fingers. B. pay particular attention to moving faces with high contrast. C. hear the difference between musical notes on a scale. D. turn away and cry when spoken to in an expressive manner. BETTER Tyler is two days old. He is likely to do all of the following EXCEPT A. be able to hang from your finger by one hand. B. pay particular attention to human faces. C. hear the difference between musical notes on a scale. D. turn away and cry when spoken to in an expressive manner. CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS RELEVANT INFORMATION MINIMAL USE OF NEGATIVES GRAMMATICALLY CONSISTENT NOT SO GOOD The neurotransmitters associated with schizophrenia and depression are A. GABA B. Glutamate C. Dopamine and serotonin D. Norepinephrine BETTER The neurotransmitters associated with schizophrenia and depression are A. GABA and glutamate B. Dopamine and GABA C. Dopamine and serotonin D. Serotonin and norepinephrine CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS RELEVANT INFORMATION MINIMAL USE OF NEGATIVES GRAMMATICALLY CONSISTENT BALANCED CORRECT OPTIONS ONLY ONE CORRECT OPTION NOT SO GOOD Azenett was interested in the relationship between temperature and domestic violence. She found a positive correlation between daily high temperatures and number of calls to a domestic violence hotline over a one year period. Which of the following correlation coefficients best describes the relationship between temperature and crisis line calls? A. -0.78 C. +0.83 B. 0.00 D. +1.00 BETTER Azenett found a positive correlation between daily high temperatures and number of calls to a domestic violence hotline over a one year period. Which of the following correlation coefficients best describes the relationship between temperature and crisis line calls? A. -0.78 C. +0.83 B. 0.00 D. -1.00 CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS RELEVANT INFORMATION MINIMAL USE OF NEGATIVES GRAMMATICALLY CONSISTENT BALANCED CORRECT OPTIONS ONLY ONE CORRECT OPTION EQUALLY PLAUSIBLE OPTIONS NOT SO GOOD The founder of Gestalt therapy was A. Karen Horney B. Fritz Perls C. Osama Bin Laden D. George Kelly BETTER The founder of Gestalt therapy was A. Karen Horney B. Fritz Perls C. Carl Rogers D. George Kelly NOT SO GOOD Avoiding shock by pressing a button when a warning tone sounds is an example of A. escape conditioning B. avoidance conditioning C. delayed conditioning D. backward conditioning BETTER In a laboratory experiment, Peter learned that if he pushed a button at the sound of a warning tone, he would not be shocked. This learning is an example of _______ conditioning. A. escape B. delayed C. backward D. avoidance NOT SO GOOD The phenomenon of looming occurs A. when light is poorly focused. B. at the blind spot. C. when the visual image of an approaching object expands about equally in all directions on the retina until it fills the visual field. D. only at night. BETTER The phenomenon of looming occurs A. when light is poorly focused on the retina, preventing transmission of a clear image to the brain. B. at the blind spot, where axons from ganglion cells exit the eyeball and eventually reach the lateral geniculate nucleus in the thalamus. C. when the visual image of an approaching object expands about equally in all directions on the retina until it fills the visual field. D. only at night, because looming depends on the activity of photoreceptors called rods, which are activated even in very low light conditions. CHARACTERISTICS OF GOOD MULTIPLE CHOICE ITEMS SUBSTANTIAL, MEANINGFUL STEMS RELEVANT INFORMATION MINIMAL USE OF NEGATIVES GRAMMATICALLY CONSISTENT BALANCED CORRECT OPTIONS ONLY ONE CORRECT OPTION EQUALLY PLAUSIBLE OPTIONS CLEARLY WORDED NOT SO GOOD Susan is applying to graduate school in psychology. She has always been interested in the influences of advertising on consumer behavior. What department of psychology would be the best fit with Susan's interests? A. Educational C. Social B. Clinical D. Industrial BETTER Susan is applying to graduate school in psychology. She has always been interested in the influences of advertising on consumer behavior. Which of the following psychology graduate programs would most likely provide courses and research opportunities that best fit with Susan's interests? A. Educational C. Social B. Clinical D. Developmental LET’S WORK ON SOME ITEMS CAN YOU IMPROVE THIS ITEM? Use the phylogenetic tree to order the domains from youngest to oldest. Eukaria Bacteria Archaea Origin of life a. Bacteria, Archaea, Eukarya b. Archaea, Bacteria, Eukarya *c. Eukarya, Archaea, Bacteria d. Eukarya, Bacteria, Archaea e. They are all the same age Can You Improve This Item? Noam Chomsky is *a. professor at Harvard University b. professor at WWU c. an economics expert d. professor at MIT Can You Improve This Item? According to the socioemotional theory of friendship change in later life, a. elderly people want larger and more diverse friendship networks but are unable to have them due to reduced mobility and inhibiting health problems b. elderly people experience important developmental changes that reduce their motivation to have friendships and cause them to withdraw into their own world *c. elderly people are more “present-oriented” rather than “future-oriented” and as a result are less interested in large diverse networks of friends d. none of the above Can You Improve This Item? Which of the following is NOT true about how friendships change during midlife? a. As romantic relationships develop, individuals tend to withdraw from their friendship networks b. Women’s friendships are more significantly affected than are men’s friendships by the development of a romantic relationship. *c. Friendship networks get broader (more friends) and more intense (more intimacy) after marriage d. After children leave home, their parents’ friendship networks tend to increase It’s Time to Write Some Great Items! REMINDER ABOUT MCQ LEVELS LEVEL REMEMBERING UNDERSTANDING APPLYING DEFINITION Recall or recognize information, ideas, and principles as presented. Translate, comprehend, or interpret information based on prior learning. Use data and principles to complete a problem or task. SAMPLE VERBS List, label, name Explain, summarize, describe, illustrate Use, compute, solve, demonstrate SAMPLE QUESTIONS What are the three main parts of _____?; What are the steps involved in____? How is __ related to __?; What themes do you see in __? How would you __ on the basis of __ theory? SOME TIPS FOR REVIEWING MC EXAMS CREATE A LIST OF THE “TOP TEN” MISSED ITEMS REVIEW THOSE ITEMS FIRST DON’T ARGUE DISTRIBUTE ITEM-REVIEW FORMS REQUEST TO REVIEW GRADING OF AN EXAM ITEM Name:__________________Student ID#:________ Test form (A or B): ______Item #:______ I believe that response alternative ___ should also be considered correct because: __________________________________________ __________________________________________ __________________________________________ __________________________________________ __________________________________________ __________________________________________ __________________________________________ _________________________________________ I found supporting evidence on page ____ in the textbook. SOME TIPS FOR REVIEWING MC EXAMS CREATE A LIST OF THE “TOP TEN” MISSED ITEMS REVIEW THOSE ITEMS FIRST DON’T ARGUE DISTRIBUTE ITEM-REVIEW FORMS PROVIDE FEEDBACK AT NEXT CLASS WRITING MULTIPLE CHOICE EXAM ITEMS DOUGLAS A. BERNSTEIN University of South Florida Southampton University www.douglasbernstein.com douglas.bernstein@worldnet.att.net CAN YOU IMPROVE THIS ITEM? Jack Chang believes that his success in math is based upon hard work, but he believes his difficulties in history are based upon an unfair professor. Jack is likely from a(n) ______ culture. A. collective C. competitive B. altruistic D. individualist CAN YOU IMPROVE THIS ITEM? It is easier to notice the difference in loudness between a two-piece and a three-piece string ensemble than between a 110-piece and a 120-piece orchestra. This is predicted by A. the noise constant. B. schema theory. C. perceptual constancy theory. D. Fechner's law. CAN YOU IMPROVE THIS ITEM? The witness claimed the robber had a gun, although later examination of the surveillance video showed that this was not true. The witness’s perception of the event was likely influenced by A. grouping principles. B. bottom-up processing. C. the simplicity principle. D. schemas. CAN YOU IMPROVE THIS ITEM? The Theory of Planned Behavior proposes that intentions are a result of the following belief(s) a. attitude and subjective norm b. perceived behavioral control c. a and b d. none of the above CAN YOU IMPROVE THIS ITEM? Factor analysis a. it is not designed to test hypotheses b. it is included in the SPSS package c. summarizes large sets of variables into smaller sets of factors d. all of the above CAN YOU IMPROVE THIS ITEM? In post traumatic stress disorder and according to the DSM IV, trauma is defined as: a. The level of damage to the individual after going through an intervention of any kind. b. An event or situation of overwhelming emotional intensity where the person feels or witnesses actual threat of death or serious injury. c. A natural catastrophe that does not affect the great majority of people but only some individuals to the extent of developing PTSD. d. None of the above. CAN YOU IMPROVE THIS ITEM? Which of these is NOT a sign of damage to descending motor pathways? a. Increased muscle tone b. Loss of ability to perform fine movements c. Babinski reflex d. None of the above CAN YOU IMPROVE THIS ITEM? Which is a skin pigment: a. Melanin b. Carotene c. Haemoglobin d. All of the above CAN YOU IMPROVE THIS ITEM? In social sciences, who is NOT a representative of the constructivist movement: a. Thomas Luckman b. Abraham Maslow c. Jean Piaget d. Gregory Bateson CAN YOU IMPROVE THIS ITEM? On which part of a neuron can the nodes of Ranvier be found? a. The dendritic spines b. The soma c. The axon d. The nucleus CAN YOU IMPROVE THIS ITEM? A researcher looking at an EEG of a participant who participated in a study looking at brain activity during sleep found that the prominent characteristics of the EEG were spindles and K-complexes. What stage of sleep is the EEG representative of ? a. Stage 1 b. Stage 2 c. Stage 3 d. Stage 4 CAN YOU IMPROVE THIS ITEM? The reticular formation has been found to play a part in arousal. Within what division of the brain can the reticular formation be found? a. Mesencephalon b. Metencephalon c. Myelencephalon CAN YOU IMPROVE THIS ITEM? In an experiment, the dependent variable is the a. causal variable b. affected variable c. experimental variable d. first variable mentioned in the title CAN YOU IMPROVE THIS ITEM? A test is reliable if it _____. a. provides accurate information b. provides adequate information c. produces similar scores on retests d. measures what it was designed to measure CAN YOU IMPROVE THIS ITEM? Which of the following would a person with prosopagnosia have difficulty with? a. recognizing a person’s face b. recognizing a person's voice c. finding a location on a map d. recognizing objects on the left-half of their visual field CAN YOU IMPROVE THIS ITEM? Who wrote the ‘cognitive-motivational analysis of anxiety’? a. Mogg & Bradley b. Ohman & Mineke c. Matthews & Macleod d. Fox & De Fockert CAN YOU IMPROVE THIS ITEM? What is the difference between state and trait anxiety? a. The former refers to general anxiety and the latter refers to situational anxiety b. The former refers to situational anxiety and the latter refers to general anxiety c. The former refers to anxiety measured via physiological stress responses and the latter refers to anxiety measured via a questionnaire. d. The former refers to anxiety measured via a questionnaire and the latter refers to anxiety measured via physiological stress responses. CAN YOU IMPROVE THIS ITEM? What are the three main components of attention as postulated by Posner and Petersen? a. Fixating, Tracking, & Scanning b. Orienting, Tracking, & Scanning c. Orienting, Shifting, & Disengaging d. Tracking, Shifting, & Disengaging CAN YOU IMPROVE THIS ITEM? Which of the following statements regarding the theories of learning is true a. Pavlov is responsible for the theory of classical conditioning b. In classical conditioning, an unconditioned stimulus is one that has not yet been paired with a response c. The amygdala is involved in learning d. The capacity of short term memory can be extended by chunking CAN YOU IMPROVE THIS ITEM? Which of the following statements about attachment are true a. Bowlby is significant in the development of attachment theory b. Rhesus monkeys given a choice between a wire surrogate mother providing milk preferred her to a toweling covered mother that did not provide milk c. According to Ainsworth there are 4 main types of attachment d. The first reaction to separation from an CAN YOU IMPROVE THIS ITEM? Recent neuroimaging studies show that amygdala hyper-responsivity and prefrontal hyporesponsivity could be characteristic of: a. Anxiety b. Depression c. Schizophrenia d. Anorexia Nervosa CAN YOU IMPROVE THIS ITEM? According to Rothbart, Ellis & Posner (2004), temperament can be seen as part of the broader domain of: a. Personality b. Aggression c. Negative Affect d. Character CAN YOU IMPROVE THIS ITEM? Cognitive tasks examining executive control have been shown to activate areas of the: a. Anterior Cingulate b. Broca's area c. Hippocampus d. Occipital lobe CAN YOU IMPROVE THIS ITEM? According to the HAPA, the motivation stage is made up of the following component(s) a. self-efficacy b. outcome expectancies c. none of the above d. a and b CAN YOU IMPROVE THIS ITEM? Which of these statements regarding ADHD are true a. ADHD as a disorder is more common in males b. ADHD is not hereditary c. According to the ICD10, Impulsivity is not one of the cardinal features of ADHD d. According to the ICD10, there are three main types of ADHD