Rollback

Forge

Returning a system to its previous state after a change, which has to be planned rather than improvised.

What is rollback?

Rollback is the return of a system to the state it held before a change was applied. Inside a single database it is a transactional guarantee. Across several enterprise systems it is a sequence of compensating actions that somebody has to design in advance.

Why there is no undo across systems of record

Halfway through a multi-system workflow, the fourth step is rejected and the first three have already committed. Nothing in the estate offers a single transaction spanning a core system and a document store whose record has already been written. What exists instead is a reversal designed per step, carrying its own approval alongside its own record. Teams treating reversal as an exception path discover during the first incident that the path was never engineered.

  • A committed action in another system cannot be withdrawn by the caller.
  • Compensating actions have to be engineered per step, with their own permissions.
  • Some actions are irreversible, which changes where approval has to occur.
  • A partial reversal leaves the estate in a state no reconciliation expects.

How rollback works

  • Each step declares whether it can be reversed and by what action.
  • Irreversible steps are sequenced as late as the process permits.
  • A compensating action is engineered for every reversible step that writes data.
  • The workflow records which steps committed before the failure occurred.
  • Reversal proceeds in the opposite order, with each compensation recorded as an action.
  • Steps that cannot be reversed are escalated to a named person for manual remedy.

How rollback is evaluated

Rollback is only credible once it has been exercised against a real failure. Test the following on a copy of the production estate:

  • proportion of steps with a defined compensating action
  • share of workflows reversed completely without manual repair
  • elapsed time from a failure to a restored state
  • reconciliation breaks remaining after an attempted reversal
  • count of irreversible steps and their position in the sequence

Limitations, risks and common failure modes

  • Reversal is documented as a procedure that nobody has ever performed.
  • A compensating action requires an entitlement the agent was never granted.
  • Downstream systems have already consumed the event that the reversal removes.
  • A notification sent to a customer cannot be withdrawn by any reversal.
  • The reversal path itself fails, leaving no recorded account of what remains.

All 315 terms