What is distractor generation?
Distractor generation is the writing of the wrong answers in a multiple choice item. A good distractor is plausible to a learner who has not understood the material and obviously wrong to one who has. It is the part of item authoring that carries the expertise, and the part automated generation finds hardest.
Also called: Distractor authoring
Why the wrong answers carry the measurement
Nearly every weak assessment fails in the same place, which is the three options nobody would ever choose. An item with one serious answer and three implausible ones measures reading rather than knowledge, and it produces the near universal pass rate that learning teams then report as evidence of competence. Good distractors come from knowing the mistakes people actually make, which is why they are difficult to generate from a document that describes only the correct procedure. The selection rate per option is the evidence of whether this was done well.
- Distractors determine whether an item measures understanding or reading speed.
- Plausible wrong answers come from the errors practitioners actually make.
- Option selection rates reveal which misconceptions a cohort holds.
- A source document describes the correct procedure and rarely the common errors.
How distractor generation works
- The correct answer is anchored to a located passage in the source.
- Known misconceptions for the topic are collected from incident reports and trainer experience.
- Each distractor is written to match one specific misunderstanding.
- Options are equalised for length and grammatical form so none stands out.
- Selection rates are reviewed after live use and dead options are replaced.
How distractor generation is evaluated
Distractors are evaluated on whether learners actually choose them and on who chooses them. Examine the following after a cohort has attempted the item:
- selection rate for each distractor in the item
- count of options selected by almost nobody
- discrimination of the item between strong and weak learners
- correlation between a chosen distractor and a known misconception
- difference in option length between correct and incorrect choices
Limitations, risks and common failure modes
- Three implausible options produce an item that everyone passes without reading.
- Generated distractors are frequently paraphrases of the correct answer.
- The longest option is the correct one often enough for learners to notice.
- Absolute wording such as always or never marks an option as wrong on sight.