Writing Test Questions
- Adhere to evidence-based principles of item writing
- Focused on one topic
- Positively worded
- Options are short and of similar length
- All options are plausible
- No cues to the answer are present
- Wording is clear
- Link test questions to learning objectives
- Have a rationale for the content being tested
- Ask yourself whether the test measures what it is supposed to
- Validity refers to scores being meaningful and defensible.
- Reliability refers to reproducibility.
- Write test questions that follow evidence-based principles of item writing
- Avoid common test question flaws: unfocused stem, negative stem, window dressing (extraneous information), inappropriate content, non-functional flaws, cueing effects, partial K-type (choosing several of the given options), all/none of the above
- Be familiar with the term construct-irrelevant variance (CIV)
Writing Multiple Choice Test Questions. Nancy Sulla. 13 minutes.
Multiple Choice Question Writing Workshop. Devshikha Bose. 20 minutes.
How to Write a Better Question. A Guide to Construction of NBME-type Questions for the UASOM Pre-Clinical Curriculum. University of Alabama College of Medicine.
Question Writing Guide. American Board of Dermatology.
Guidelines for Constructing Multiple-Choice Items. American College of Surgeons.
McCoubrie P. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 2004 Dec;26(8):709-12. Review. PubMed PMID: 15763874.
Boland RJ, Lester NA, Williams E. Writing multiple-choice questions. Acad Psychiatry. 2010 Jul-Aug;34(4):310-6. doi: 10.1176/appi.ap.34.4.310. Review. PubMed PMID: 20576995.
♥Writing Multiple Choice Questions: An Introductory Tutorial. NBME. Self-paced interactive tutorial that provides a foundation for writing quality multiple-choice questions. This tutorial is designed to introduce some of the main principles of item writing to individuals who write questions for examinations in the field of medicine and across the range of health professions. The tutorial is organized into sections so that you can complete only a portion of it and easily return later; the entire tutorial takes about 45 minutes to complete.
Tao. Open source exam software. No usage fees but must be installed by IT, similar to Moodle.
Creating tests that measure critical thinking in health science professions. ExamSoft. 50 minutes.
Best practices in writing test items. Assessment Systems. 27 minutes.
Do This, Not That: A Survey of Multiple-Choice Question Types (Most of Which Should Be Avoided). ExamSoft. 43 minutes.
Teaching and Assessing Critical Thinking Using Assessments with Embedded Outcomes. ExamSoft. 48 minutes.
Embracing Multimedia in Exam Question Construction and Analysis. ExamSoft. 50 minutes.
Writing Great Clicker Questions: Faculty Workshop. Stephanie Chasteen. 48 minutes.
♥NMBE Item Writing Manual. Constructing Written Test Questions for the Basic and Clinical Sciences.
Writing Exam Questions Resources. University of Washington Center for Instructional Development and Research.
Item-Writing Seminar for Practitioners and Educators in the Health Professions. Objectives of the annual ½-day seminar include refining your item-writing techniques and enhancing the quality of items in your examinations and programs. The agenda includes a review of item-writing guidelines; examples of "good" and "bad" test items; a dynamic item-writing session; and the opportunity to interact with participants from a range of health professions. NBME Headquarters; 3750 Market Street; Philadelphia PA 19104. $375
♥NBME U is an online collection of lessons for individuals interested in learning a range of topics relevant to high quality assessment. The lessons are designed to provide brief overviews of assessment topics that are of interest to those in the health professions’ education community at no cost to the learner. http://www.nbme.org/about/nbmeu/
NBME Faculty Development Workshops are offered to groups of medical faculty, educators of healthcare disciplines, and bodies that assess or certify the competence of healthcare providers. Workshops are available globally on a year-round basis and can take place at the school. Workshops include Mastering the MCQ: Guidelines for Writing Multiple-Choice Questions, Critical Themes in Assessment, Case Development for Standardized Patient Based Examinations, Measurement Skills in Clinical Skills Assessment.