Data quality score

Loupe

A graded view of what arrived, so intake quality can be tracked rather than discussed anecdotally.

What is a data quality score?

A data quality score is a graded assessment of what a submission actually contained when it arrived. It combines completeness against the activated requirements with the legibility and internal consistency of what was supplied. The purpose of a single figure is comparison across schemes and across periods.

Why intake quality is argued about rather than measured

Every review of an intake backlog produces the same exchange, in which the operations lead says submissions are getting worse and the scheme owner says the forms are fine. Neither has a number, so the discussion resolves on seniority. A data quality score ends that by grading each submission on arrival, which turns a disagreement about impressions into a trend anyone can examine. It also makes a form redesign answerable for whether it improved anything.

  • A score converts intake quality from an opinion into a measured series.
  • Grading on arrival separates form design problems from applicant behaviour.
  • Scheme owners can be held to the quality of what their form collects.
  • A redesign can be judged against the cohort that preceded it.

How a data quality score works

  • The components of quality are defined and weighted before any submission is graded.
  • Completeness is measured against the requirements the applicant's answers activated.
  • Legibility of each attachment contributes according to how far assessment depends on it.
  • Internal consistency across related fields is assessed as a separate component.
  • The components combine into one figure with each component retained separately.
  • Scores are reported by scheme and by period against the preceding cohort.

How a data quality score is evaluated

A data quality score earns trust only if it predicts what assessors experience. Examine the following against assessed outcomes:

  • correlation between the score and assessor rework effort
  • spread of scores within a single scheme and period
  • stability of the score definition across reporting periods
  • contribution of each component to the overall figure
  • share of submissions whose score changed after a form revision

Limitations, risks and common failure modes

  • Component weights are chosen to produce a comfortable headline figure.
  • A score reported without its components hides which part degraded.
  • The definition changes mid-year, so the series compares nothing to nothing.
  • Scheme owners optimise the score rather than the experience it stands for.

Related terms

Learner analyticsWhat completion and timing data can and cannot tell you about whether training worked.First-pass yieldThe share of submissions accepted without rework, which is the cleanest measure of intake quality.Form abandonmentApplicants starting a form and stopping, which is usually a design problem rather than a motivation problem.ConcordanceThe share of cases where an automated score and a human reviewer arrive at the same conclusion.Document classificationIdentifying what kind of document has arrived before deciding how to treat it.Intelligent document processing (IDP)Turning submitted documents into structured, validated data rather than into images filed somewhere.

All 315 terms