What is red teaming?
Red teaming is adversarial testing performed before deployment by people whose objective is to make the system fail. Functional testing confirms that intended behaviour occurs, where a red team searches for the inputs and sequences nobody specified. What the exercise produces is a register of findings, each with the sequence that reproduced it.
Also called: Adversarial testing
Why the finding register outlives the exercise that produced it
Adversarial testing earns its place in the fortnight before a go live decision, when the system is complete enough to attack and the organisation can still change it. An exercise held earlier tests a prototype, and one held afterwards produces findings nobody has budget to address. What a board examines is the register itself, since it states what was discovered along with what happened to each item. An exercise with no register is an assurance activity with no output.
- Adversarial testing surfaces failures that no specification anticipated.
- An external team holds no stake in the design decisions already taken.
- Findings are the evidence of diligence a regulator or an insurer will examine.
- A finding closed before deployment costs a fraction of one closed afterwards.
How red teaming works
- Scope and rules of engagement are agreed with the system owner in writing.
- Threat scenarios are drawn from the consequence of failure for this deployment.
- Testers attack the complete system as deployed rather than a component in isolation.
- Each finding is recorded with the sequence that produced it and a severity.
- Remediations are implemented and the original sequence is replayed to confirm closure.
- Unresolved findings are accepted explicitly by a named owner before go live.
How red teaming is evaluated
An exercise is judged on coverage and on what happened to the findings. Examine the following once each exercise closes:
- share of agreed threat scenarios actually attempted
- severity distribution of findings and the time to close each band
- proportion of findings reproducible from the recorded sequence
- findings accepted as residual risk, with a named owner for each
- overlap between this exercise and the one preceding it
Limitations, risks and common failure modes
- Scope is negotiated down until the exercise avoids the uncomfortable surfaces.
- Testing occurs against a staging environment configured unlike production.
- Findings are logged as recommendations and never enter a tracked backlog.
- The same team repeats the exercise and converges on its own familiar techniques.
- A report dated before launch is presented years later as current assurance.