Monitoring, costs, and incident response

Home Learn Monitoring, costs, and incident response

In short

An AI workflow needs an owner after it starts operating. Monitoring shows whether it is doing the job, using resources sensibly, and staying inside its boundaries. Incident planning gives people a way to pause, investigate, correct, and recover when it does not.

Why it matters

A useful pilot can drift as sources change, people ask new questions, or a connector fails. A workflow may repeat a tool call until time or cost grows. A wrong answer can keep appearing if nobody sees the pattern. Someone needs a view of the system’s actual work, not only a count of how many requests it handled.

How it works

Record a run’s meaningful steps: the task, source versions, tool calls, outcomes, errors, approvals, and final decision. Keep that record proportionate to the job and avoid unnecessary sensitive data. A trace explains the path; it does not prove the conclusion was correct. Set signals tied to consequences, such as wrong-source answers, missed escalations, duplicate actions, connection failures, unusually long loops, or a rising volume of human corrections.

NIST’s 2023 AI Risk Management Framework describes managing risk throughout operation and using measurements to guide action. Anthropic’s 2026 agent-evaluation guidance shows why intermediate steps matter when judging a system that uses tools. Together, they support a practical operating cycle: set thresholds, look at representative failures, assign an owner, and make changes when evidence shows the process is no longer serving its purpose.

Put time, step, and spend limits on the workflow. Define who receives an alert and what they can pause. An incident is a problem that needs a coordinated response, such as an unexpected external action or repeated disclosure of the wrong material. The response should preserve enough evidence to understand the cause, stop the affected path, assess what changed, correct the issue, and decide when to resume. Review the human experience as well: a flood of low-value alerts can hide the one that matters.

Example

Illustrative example: this scenario is invented.

Lena, 31, uses a draft-only assistant to organize descriptions of public art exhibits. One museum feed begins returning the same item repeatedly. The workflow’s run record shows duplicate retrievals and growing processing time. Lena pauses that feed, checks the source, and revises the duplicate check before using more drafts. No exhibit entry was published automatically. The useful signal was the repeated source item and time spent, not a vague declaration that the assistant seemed busy.

She records the feed version and the repeated item identifier in the issue note. Those details let the next reviewer distinguish a source problem from a model summary problem.

What it doesn’t mean

A dashboard full of numbers is not oversight if nobody knows which signal calls for action. More logs are not always better; access and retention matter. A passing evaluation before launch doesn’t cover later changes. Nor should an incident record end with a blame-free slogan while the same failure path remains open.

Try it

Choose a harmless public-list assistant. In ten minutes, write three signals you would monitor, one limit on repeated steps, and the name of a role that could pause it. Ask any assistant to suggest an incident path if a source stops responding. Check whether its proposal includes both stopping the run and preserving the information needed to diagnose it.

Keep going

Sources

Share this page