The First Ninety Days After Go-Live: What Operating an Agent Involves

What changes on day one

Testing covers the cases a team can imagine. Production delivers the ones it could not. Customers phrase requests in ways no test set anticipated, and staff working alongside the agent change their own behaviour once they see what it handles. Some begin routing difficult cases to it; others stop trusting it after a single bad answer.

None of this signals that the launch has failed. It is information the agent could not receive before launch, and the first weeks exist to absorb it deliberately. The risk lies in treating go-live as the finish line, so that the information arrives with nobody assigned to act on it.

How we structure the first ninety days

We divide the period into four stages, each with its own purpose and its own exit test.

Hypercare, days one to fourteen. Every exception is reviewed daily by the people who engineered the agent. Thresholds are deliberately conservative, and more work escalates to people than the steady state will need.

Tuning on evidence, weeks three to six. Patterns in the exceptions become specific changes, each tested against a growing set of real cases before release. Escalation thresholds relax only where the evidence supports it.

Controlled expansion, weeks seven to ten. New variants or user groups are added one at a time, each with its own short hypercare window.

Handover of steady-state ownership, weeks eleven to thirteen. Review moves from daily to weekly, and the client's named owners take the chair.

The exit test for each stage is agreed before launch, in the same terms as the business case. A programme that cannot say what would make it ready for the next stage is not yet ready for this one.

Who owns what once the launch team steps back

Ownership after go-live has two halves, and both need a name attached. The business owner is accountable for the outcome: the resolution rate and the cost per resolved case. The operator is accountable for the system: its evaluation set and its change log. Where Es Magico operates the agent for a client, we hold the second role and report against the first.

The weekly review is where those two meet. It looks at a sample of failed and escalated cases and at each proposed change, with the evidence it was tested against. It is a working session with a short agenda, and its records become the history a regulator or an internal auditor will ask to see.

The metric that looked healthy in week six

On one engagement, the containment rate of a customer-service agent climbed steadily through its first six weeks. The dashboard was green and the client was pleased. The weekly review sample told a different story: a growing share of contained conversations ended with customers giving up, closing the chat without a resolution the agent could record.

The agent had not been designed to close conversations early. A tuning change intended to reduce unnecessary escalations had made it slower to offer a person, and customers who wanted one simply left. Containment rose because the customers who most needed help stopped asking for it.

We reversed the change and paired containment with two further measures, repeat contact within seven days and complaints referencing the channel. No single metric now moves the escalation threshold on its own. The lesson shaped every engagement since: in the first ninety days, a metric moving the right way deserves the same scrutiny as one moving the wrong way.

We engineer AI ambition into enterprise value, and most of that value is earned after the launch email. An executive sponsoring an agent should ask, before it goes live, who will review its failures in week three and who will own it in month four. A programme that can name both people, and show the evidence they act on, is the kind that produces outcomes that survive scrutiny.