AI agent observability in 2026: traces, evaluations and production guardrails

Agents fail differently from ordinary services. A traditional outage looks like a 500, a timeout or a saturated queue. An agent failure often returns HTTP 200 with a confident, wrong answer, or quietly calls the right tool with the wrong argument, or loops through six retries and burns a rupee of tokens before giving up. None of that shows up in a request-rate dashboard.

So the question for 2026 is not whether your agent works in a demo. It is whether, when a customer complains about something the agent did last Tuesday, you can reconstruct exactly what happened, prove it was within policy, and show that your last release did not make it more likely.

What observability means for an agent

Borrow the discipline from distributed systems and extend it. An agent run is a distributed transaction that happens to include a non-deterministic component, so you need the usual three signals plus two that are specific to AI:

• Traces: the causal spine of a run, showing the order and nesting of model calls, retrievals, tool invocations and retries.
• Metrics: success rate, latency percentiles, token and cost per task, tool error rate, escalation and abandonment rate.
• Logs: redacted prompt and response payloads, tool arguments and results, linked to the trace that produced them.
• Evaluations: scores for correctness, groundedness, format compliance and safety, computed offline and on sampled live traffic.
• Policy decisions: the record of what a guardrail allowed, blocked, redacted or escalated, and why.

The last two are what most teams are missing. Without them you can see that a run was slow, but not that it was wrong.

Instrument end-to-end traces with OpenTelemetry

Resist the temptation to invent a bespoke logging format. OpenTelemetry traces already model exactly what an agent run is, and the project maintains semantic conventions for generative AI that standardise attribute names for model calls, token usage and tool execution. Using them means your agent telemetry lands in the same platform as your API and database telemetry, and stays portable if you change vendors.

A workable span layout for one user request:

• Root span per request, carrying tenant, user role, session, agent version and the prompt template version in use.
• Child span per model call with model name and version, input and output token counts, temperature, finish reason and computed cost.
• Child span per retrieval with the index or collection, query, number of chunks returned and the document identifiers cited.
• Child span per tool call with the tool name, a hash or reference to the arguments, the authorisation decision, latency, and whether the effect was a read or a write.
• Explicit spans for retries and fallbacks so a "successful" run that took four attempts does not look identical to one that took a single attempt.

Two practical rules save pain later. First, propagate one trace identifier from the user-facing request through every downstream service, and surface it in the UI or support tooling so a complaint maps to a trace in seconds. Second, keep payloads out of span attributes: store prompts and outputs in a redacted, access-controlled store and reference them by identifier, so your telemetry backend does not quietly become your largest repository of customer data.

Evaluations: offline before release, online after

Evaluation is the part teams skip because it feels like extra work, and it is the part that turns model updates from a gamble into a decision. The two modes are complementary.

Offline evaluations gate the release

Build a golden dataset of a few hundred representative cases: common requests, known hard cases, adversarial prompts and past production failures. Score each run for task success, factual groundedness against cited sources, correctness of tool selection and arguments, output format validity, and refusal behaviour on out-of-scope requests. Provider tooling makes this cheap to operate — see the guidance for OpenAI's evals and Anthropic's evaluation tooling — but the dataset is yours to own, and it is the real asset.

Wire the suite into CI so any change to a prompt, tool schema, retrieval index or model version must clear the gates before it ships. Use deterministic checks wherever you can (did the JSON validate, was the correct tool chosen, did the total stay within the tolerance) and reserve model-graded scoring for genuinely subjective qualities. Calibrate your graders against human labels on a sample; an unvalidated LLM judge is a confident number, not a measurement.

Online evaluations catch what the dataset missed

Production traffic is always stranger than your test set. Sample one to five percent of live runs for automated scoring, and score every interaction that ends in an escalation, a retry storm or negative feedback. Feed the failures back into the golden dataset monthly. Roll changes out behind flags with shadow or canary traffic, compare the new version against the current one on the same inputs, and keep a one-click rollback for prompts and tool definitions — they are configuration and deserve the same release rigour as code.

Tool-call safety is the real risk surface

A chatbot that hallucinates is embarrassing. An agent with write access that acts on a hallucination, or on instructions hidden inside a document it retrieved, is an incident. Prompt injection leading to unauthorised tool use remains the dominant class of failure, and the OWASP Top 10 for LLM applications is the most useful checklist to review your design against.

• Least privilege per role: scope tools to the acting user's permissions, not the agent's service account. If the human cannot delete the record, neither can the agent on their behalf.
• Separate read from write: automate reads, drafts and summaries freely; require typed confirmation for anything that moves money, sends external communication, or deletes data.
• Validate arguments, not just intent: enforce schemas, value ranges, allow-lists and monetary limits at the tool boundary, and reject calls that fail rather than letting the model retry into a valid-looking mistake.
• Treat retrieved content as untrusted input: content from documents, tickets, emails and web pages is data, never instructions. Keep it in a separate channel from your system policy.
• Idempotency and reversibility: give write tools idempotency keys and, where feasible, an undo path, so a retry does not create three purchase orders.
• Rate and blast-radius limits: cap tool calls per run and per tenant per hour, and trip a circuit breaker on abnormal patterns.

Cost and latency: measure per task, not per token

Token pricing keeps falling, and agent bills keep rising, because agents multiply calls: planning steps, tool round-trips, retries, self-critique passes. The metric that matters is cost per successfully completed task, with tokens as a diagnostic underneath it. A run that costs half as much but fails twice as often is more expensive once a human has to finish the job.

Attribute every span's cost to a tenant, feature and model so you can see which workflow is driving spend. Then apply the levers that usually pay off: route easy requests to smaller models and escalate only on low confidence; cache prompt prefixes and deterministic tool results; trim retrieved context to what is actually cited; cap plan depth and retry counts. On latency, budget end to end and track the percentiles users feel — p95 and p99 — because a slow tool call, not the model, is very often the culprit. Stream partial output so perceived latency stays acceptable while a long tool chain completes.

Auditability: reconstruct any decision months later

Regulated and enterprise buyers will ask you to explain a specific decision from a specific day. Design for that question up front. An audit record for a run should identify who initiated it, which agent and prompt version served it, which model and version produced each output, which documents grounded the answer, which tools ran with which arguments and results, which policies fired, and where a human approved or overrode the outcome.

Keep those records immutable and access-controlled, set retention deliberately by data class, and redact or tokenise personal data at ingestion rather than hoping to scrub it later. Note the data residency of your telemetry pipeline too — traces that leave your approved region are a compliance finding waiting to happen. Mapping these controls to a recognised framework such as the NIST AI Risk Management Framework makes security reviews considerably shorter, because you are answering with a structure the reviewer already knows.

Incident response when the failure is a bad answer

Your existing on-call process assumes a signal that fires. Add the AI-specific ones: a drop in eval scores on sampled traffic, a spike in tool-call rejections or retries, a jump in escalations to humans, a sudden shift in output length or refusal rate, and cost per task breaching its budget. Alert on the ones tied to user harm; put the rest on a dashboard.

Then rehearse the response. Containment for an agent usually means degrading capability rather than taking the service down: switch the affected tool to read-only, revert the prompt or model version, force human approval on a category of action, or fall back to a deterministic workflow. Because prompts, tool schemas and model choices are versioned configuration, rollback should take a minute, not a deploy cycle.

Post-incident, the trace is your evidence. Reconstruct the run, add the failing case to the golden dataset with an assertion that would have caught it, and record whether the gap was in the prompt, the retrieval, the tool contract or the guardrail. An eval suite that grows with every incident is the clearest sign an AI programme is maturing.

A production readiness scorecard

Before an agent gets write access to a real system, we expect a yes to every line below. Anything less, and it stays in pilot with a human in the loop.

Frequently asked questions

What should a single agent trace capture?

One trace per user request, with a child span for every model call, retrieval query, tool invocation and retry. Each span should record the model and version, prompt and tool identifiers, token counts, latency, cost, outcome status and the policy decision that allowed or blocked the step. Store prompt and response payloads by reference with redaction so the trace stays useful without becoming a copy of your sensitive data.

How often should we re-run agent evaluations?

Run the offline suite on every change to a prompt, tool, retrieval index or model version, and block the release if a scored gate regresses. Run online evaluations continuously on a sample of live traffic, typically one to five percent, plus every escalated or thumbs-down interaction. Refresh the golden dataset monthly with new failures found in production so the suite keeps pace with real usage.

Do we need a dedicated LLM observability tool?

Not at the start. Emit OpenTelemetry spans that follow the GenAI semantic conventions and you can view agent traces in the observability platform you already run, alongside your application and database telemetry. Add a specialised evaluation or prompt-management product when you need dataset versioning, annotation queues and side-by-side comparison at scale, and keep OpenTelemetry as the wire format so the data stays portable.

Where we start with clients

We instrument first. Before adding capability to an agent, we make the current one legible: OpenTelemetry traces on every run, a small golden dataset drawn from real transcripts, and a dashboard showing success rate, cost per task and tool error rate side by side. That baseline usually reveals the cheapest wins — a tool contract that was ambiguous, a retrieval step returning noise, a retry loop nobody had noticed.

From there, guardrails and scope expand together: each new write capability arrives with its own eval cases, its own approval rule and its own alert. If you are moving an agent from pilot to production and want that groundwork done properly, get in touch — it is a short engagement that makes every later decision about your AI stack an evidence-based one.