Stockertim Dictionary · Artificial intelligence · Technology
Observability
Also called AI observability, agent tracing
In short
Observability is being able to see what an automated system did and why, from its logs, traces and metrics, so that problems can be found, explained and fixed.
- Advanced
- Concept
- Checked
In detail
Observability rests on three kinds of record. Logs capture individual events, such as a request received or an error raised. Traces follow one task end to end through every step, model call and tool call, with inputs, outputs and timings. Metrics summarise behaviour over time: success rates, error rates, response times, costs and how often a person had to step in.
For AI agents, traces matter most. Because the model chooses its own steps, the only way to understand a surprising outcome is to see the path it took: which documents it retrieved, which tools it called with which inputs, and what each returned. Without that record a failure can only be guessed at and cannot be reliably reproduced or fixed.
Observability needs care with data. Traces may contain personal or confidential information from prompts and documents, so they are protected, kept only as long as needed and, where possible, stripped of sensitive fields. Alerts on the key metrics turn the records from a forensic tool into an early warning.
Worked example
Northwind Bikes reviews a week of traces from its order-status agent.
- Runs in the week: 2,000, of which 46 failed.
- Failure rate: 46 / 2,000 = 2.3%
- Traces show 30 of the 46 failures share one cause, a timeout in the tracking tool: 30 / 46 = 65.2%
Fixing one tool timeout addresses about two thirds of the failures, a finding only the traces could show.
In practice
Teams record a trace for every agent run, track success, error and intervention rates on a dashboard, set alerts on sudden changes and protect logs that contain customer data.
Watch out
- Logs that record only final answers are not enough to explain an agent's behaviour.
- Traces can hold sensitive data; protect and expire them like any other record.
Learn it properly
Official sources
- Guidelines for secure AI system development · National Cyber Security Centre
- NIST AI RMF Playbook · National Institute of Standards and Technology
Facts last checked on 3 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.