MedEdWorld Updates highlight
useful and easily accessible sources
of information on key and important
topics in medical education
contributed by leading experts from
around the world.
MedEdWorld
A global online medical education community
U P D AT E O N
MULTIPLE CHOICE
QUESTIONS
Useful resources on MedEdWorld
Dr Catherine Kennedy, AMEE Projects Officer
GLOSSARY
A database of the expanding
vocabulary used in medical education
Multiple Choice Questions: ‘Multiple Choice Questions (MCQs) have been a common means
of written assessment in medical education for many years. They are suitable for use in areas where
objectively correct answers can be ascertained, such as the basic sciences and some clinical scenarios.
In general MCQs are designed with a stem – a question or scenario – and a set of responses. MCQs can
be implemented in a variety of ways including:
• Single best option - a student is expected to select the best response from a range of 4-5 possible options;
• Multiple true/false questions – a student has to assess a number of statements relating to the question
as either being true or false;
• Extended matching questions – students answer a series of questions relating to a scenario by selecting
responses from an extended list of approximately 20 options.
MCQs have a range of advantages, such as enabling examiners to cover a large degree of content, and
are increasingly easy to administer and score using computers. However, designing valid questions and
responses is a demanding skill that can be time consuming, and criticism has been made that they do
not assess higher-order learning and analytic skills. Further discussion of MCQs and their advantages and
disadvantages can be found in our Glossary.
The MedEdWorld Glossary can be accessed at www.mededworld.org/Glossary.aspx
WEBINARS
Hour-long interactive presentations
with experts in medical education
Developing High-quality Single- Best-Answer MCQs to Assess
Application of Knowledge Using Patient Vignettes
Presenters: Kathy Holtzman and Dave Swanson
Host Institution: National Board of Medical Examiners, USA
www.mededworld.org/Webinars/Webinar-Items/DevelopingHigh-quality-Single-Best-Answer-MCQs- t.aspx
This webinar focuses on writing high-quality MCQs using patient vignettes that go beyond testing recall of
isolated facts to assess application of basic science and clinical knowledge in decision making.
www.mededworld.org
www.amee.org
TEXTBOOKS
Tweed, M., Thompson-Fawcett, M., Schwartz, P., and Wilkinson, T.J. (2013) Determining measures of
insight and foresight from responses to multiple choice questions. Medical Teacher, 35:2, pp. 127-133.
www.mededworld.org/Publications/Articles/Determining-measures-of-insight-and-foresight-from.aspx
Clinical practitioners require insight, the ability to judge their personal capability, and foresight the ability to
predict the potential consequences of the actions they suggest.
A selection of texts recommended
on MedEdWorld
Case, S.M. and Swanson, D.B. (2002) Constructing Written Test Questions for the Basic and Clinical
Sciences (3rd Ed. Rev.) Philadelphia, PA: National Board of Medical Examiners.
www.mededworld.org/Publications/Textbooks/Constructing-Written-Test-Questions-For-the-Basic.aspx
The manual provides an overview of item formats, concentrating on the traditional one-best-answer and
matching formats. It reviews issues related to technical item flaws and item content; statistical indices of
item quality after test administration; and standard-setting.
Zoanetti, N., Beaves, M., Griffin, P., and Wallace, E.M. (2013) Fixed or mixed? a comparison of three,
four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program. BMC Medical
Education, March 2013.
www.mededworld.org/Publications/Articles/Fixed-or-mixed-a-comparison-of-three,-four-and-mix.aspx
This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms
within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal
Surveillance Education Program (FSEP).
Haladyna, T.M. and Rodriguez, M.C. (2013) Developing and Validating Test Items. London: Routledge.
www.mededworld.org/Publications/Textbooks/Developing-and-Validating-Test-Items.aspx
An update and development of ‘Developing and Developing Multiple-Choice Items (3rd Ed.)’ (Haladyna,
2004), this book provides a comprehensive resource for the development and validation of multiplechoice question items and constructed response question formats.
RESOURCES
Osterlind, S.J. (1997) Constructing Test Items: Multiple-Choice, Constructed Response, Performance
and Other Formats. Boston: Kluwer Academic Publishing.
www.mededworld.org/Publications/Textbooks/Constructing-Test-Items-Multiple-Choice,-Construct.aspx
This book addresses major issues in constructing test items by highlighting: the characteristics and
functions of test items; editorial guidelines for writing test items in all of the commonly used item formats;
methods for determining the quality of test items; procedures for ordering items in a test; item scoring
schemes; and computer-generated items.
JOURNAL ARTICLES
Guidelines for writing Multiple Choice Questions (MCQs) Single Best Answer (SBA) &
Extended Matching Items (EMI) formats
www.mededworld.org/Resources/Resources-Items/Guidelines-for-writing-Multiple-Choice
Questions-% 28.aspx
Date: 2012
Author: Intercollegiate Speciality Boards
This guide provides a range of useful information for the development of MCQs, including Single Best
Answer (SBA) and Extended Matching Item (EMI) formats.
A selection of key articles recommended
by the MedEdWorld Community
LEARNING TECHNOLOGY
Bandaranayake, R.C. (2008) Setting and maintaining standards in multiple choice examinations:
AMEE Guide No. 37. Medical Teacher, 30:9-10, pp. 836-845.
www.mededworld.org/Publications/Articles/Setting-and-maintaining-standards-in-multiple-choi.aspx
This guide examines the more commonly used methods of standard setting, illustrates the procedure
used in each with the help of an example, and discusses the advantages and disadvantages associated
with the use of each.
Recommended to support teaching,
learning and assessment
Questionbank
www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions.
aspx
This software offers a toolset for developing, managing and maintaining examination questions,
as well as generating detailed analysis on question performance.
Campbell, D.E. (2011) How to write good multiple-choice questions. Journal of Paediatrics and Child
Health. 47:6, pp.322-325.
www.mededworld.org/Publications/Articles/How-to-write-good-multiple-choice-questions.aspx
This paper provides a simple guideline that will assist with the production of high-quality multiple-choice
papers.
Questionmark
www.mededworld.org/Learning-Technology/Equipment-Items/Questionmark-Enables-measurement-ofknowledge,-ski.aspx
Questionmark’s technologies and services enable organizations to measure knowledge, skills, and
attitudes securely for certification, regulatory compliance and successful learning outcomes.
McCoubrie, P. (2004) Improving the fairness of multiple-choice questions: a literature review. Medical
Teacher, 26:8, pp. 709-712.
www.mededworld.org/Publications/Articles/Improving-the-fairness-of-multiple-choice-question.aspx
This article reviews the literature on MCQ-testing and the perceptions that they can be ‘unfair’. The article
explores these concerns before making a series of practice points for conducting fair and defensible MCQ
examinations.
Hot Potatoes
www.mededworld.org/Learning-Technology/Equipment-Items/Hot-Potatoes.aspx
The Hot Potatoes suite includes six applications, enabling you to create interactive multiple-choice, shortanswer, jumbled-sentence, crossword, matching/ordering and gap-fill exercises for the World Wide Web.
FORUMS
Markert, R.J., Goggans, D.S., and Richardson, W.S. (2012) Local Development of MCQ Tests for
Evidence-based Medicine and Clinical Decision Making Can Be Successful. Teaching and Learning in
Medicine, 24:4, pp. 341-347.
www.mededworld.org/Publications/Articles/Local-Development-of-MCQ-Tests-for-Evidence-based.aspx
This paper describes a strategy to develop an EBM/CDM MCQ test database guided by educational theory
and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy’s
success.
A space for debating a wide range of topics of
current interest to the health professions.
MCQs
http://www.mededworld.org/Forums/f129/t4388/MCQs.aspx
Post your queries, ideas and suggestions on MCQs here for discussion with colleagues.
You can also add your recommendations on topics to a forum for upload to the
MedEdWorld site.
Palmer, E.J. and Devitt, P.G. (2007) Assessment of higher order cognitive skills in undergraduate
education: modified essay or multiple choice questions? Research paper. BMC Medical Education,
49:7.
www.mededworld.org/Publications/Articles/Assessment-of-higher-order-cognitive-skills-in-und.aspx
This study was designed to assess the effectiveness of Modified Essay Questions (MEQs) to measure higherorder cognitive skills in an undergraduate institution.
www.mededworld.org
Including reports, websites, video clips,
images and diagrams.
www.amee.org
www.mededworld.org
www.amee.org
MedEdWorld was launched by AMEE to help all with an interest in health professions education
learn, connect and debate key issues in medical education. Join the MedEdWorld Community by
registering as a user for free or joining as a member at:
www.mededworld.org
AMEE members have full and free access to MedEdWorld. Simply login with your AMEE username
and password.
Copies of this and other MedEdWorld Updates are located in the Resources section of MedEdWorld.
This will give you direct access to all the hyperlinks listed in the summary.
www.mededworld.org
Scottish Charity SCO31618
www.amee.org