Proctoring

Atlas

Supervision of an assessment, whether by a person or by software, so that the result means something.

What is proctoring?

Proctoring is the supervision of an assessment so that the result means something about the person who holds it. Supervision can come from a person in the room or from software observing the session. The purpose is narrow, which is to establish that the named candidate answered the questions without assistance.

Also called: Invigilation

Why an unsupervised pass is not evidence of anything

An unproctored assessment is cheap to deliver and worth very little as evidence. The cost appears later, when a regulator asks whether the person performing a controlled process was assessed on it, and the organisation produces a pass record that anyone in the office could have earned on that person's behalf. Proctoring is what separates a completion record from an attestation that will hold. The difficulty is that heavy supervision across a large network becomes the most expensive part of the programme.

  • An assessment nobody supervises evidences access to a device rather than competence.
  • Regulated qualifications are challenged individually, so each result has to stand alone.
  • Supervision cost grows with the population, which forces a design decision early.
  • Item design can carry part of the integrity load that supervision would otherwise carry.

How proctoring works

  • The integrity requirement is set per assessment according to what the result permits.
  • Identity is verified at the start of the session against a trusted record.
  • The session is observed, with the observation signal retained as part of the result.
  • Item banks are randomised per candidate so that answers cannot circulate.
  • Anomalies are flagged for human review rather than resolved automatically.
  • The result is recorded with the supervision method applied to it.

How proctoring is evaluated

Proctoring is assessed on the integrity of the results it produces and on what it costs per candidate. Examine the following across a full assessment cycle:

  • share of sessions supervised by each method in use
  • anomaly flag rate and the proportion upheld on review
  • pass rate difference between supervised and unsupervised sessions
  • item exposure across the candidate population
  • supervision cost per candidate against the population to be assessed

Limitations, risks and common failure modes

  • Software observation produces false flags that fall unevenly across candidates.
  • Candidates in a branch take an assessment together while one person answers.
  • Item banks are too small, so questions circulate within a week of release.
  • Privacy requirements in some jurisdictions prohibit the observation method chosen.
  • An anomaly is logged and nobody is accountable for reviewing the queue.

Related terms

Assessment integrityThe degree to which a score reflects the learner rather than the conditions the assessment was taken under.Assessment itemA single question with its options and its correct answer, traceable to the source it is drawn from.Automatic question generationProducing assessment items from a source document rather than authoring them separately from it.Competency frameworkThe structured description of what a role requires, against which an assessment score is interpreted.Item difficultyThe proportion of learners who answer an item correctly, which tells you whether it is measuring anything.Learner analyticsWhat completion and timing data can and cannot tell you about whether training worked.

All 315 terms