Evals and Information Provenance

Evals catch regressions systematically, and provenance tracking prevents misplaced trust in conflicting context.

Context & Reliability·Lesson 6 of 6·7 min

This closing topic covers two practices that make an agentic system trustworthy over time rather than just impressive in a single demo: evals, which catch regressions before they ship, and information provenance, which tracks where each piece of context came from so it gets trusted appropriately.

Evals: a structured test suite for behavior, not just output format

An eval, in this context, is a structured test suite that scores an agent's or prompt's outputs against expected behavior across a representative set of inputs. It is the equivalent of a unit test suite for a component whose behavior is defined by a natural-language prompt and a model rather than by deterministic code. Without evals, changes to a system prompt, a tool definition, or a model version are validated by spot-checking a handful of examples, which reliably misses regressions that only show up on inputs outside whatever the developer happened to try.

A good eval suite has a few properties worth naming precisely. It covers a representative range of inputs, including edge cases and known-hard cases, not just the happy path. It scores against defined expected behavior — which for agentic tasks often can't be exact-string matching, and instead uses techniques like checking that a tool call sequence matches an expected set, that specific facts appear in the output, or grading by a separate model-as-judge call against a rubric. Crucially, it runs automatically and repeatedly, on every meaningful change, not just once at initial launch. The exam scenario to recognize is any situation where a team changes a prompt, ships it, and only discovers a regression from user complaints — that's precisely the gap evals are meant to close, and the correct answer to "how could this have been caught earlier" is almost always "with an eval suite run before shipping," not "with more careful manual review."

Information provenance: know where a fact came from

Information provenance is tracking the origin of a piece of context: which tool call produced it, which file it was read from, which turn of the conversation introduced it, or which external source supplied it. This matters most sharply when an agent is reconciling two pieces of information that conflict. Without provenance, the agent (or a human reviewing its output) has no principled way to decide which source to trust — it can only guess, or default to whichever appeared most recently, which is not a reliable trust signal on its own.

With provenance tracked, the agent can reason about trust properly: a fact pulled directly from a verified database record generally deserves more trust than a fact pulled from a user's casual restatement of something they half-remember, and a fact from a tool call made three turns ago against live data deserves more trust than a stale fact carried in a compacted summary from much earlier in the session. Provenance also supports auditability after the fact — when an agent's output turns out to be wrong, being able to trace the specific tool call or file that supplied the faulty premise is what makes the failure fixable rather than mysterious.

Exam trap A question describes an agent that receives conflicting information — a file comment saying a function is deprecated, and a live API response showing the function is still actively called by other services — and picks the file comment because it's "more specific." The trap is treating specificity of wording as a proxy for trustworthiness. The correct reasoning is provenance-based: a live API response reflecting current system state is a more reliable signal of present reality than a code comment, which can go stale the moment the code around it changes and nobody updates the comment. The agent should weight the live data higher and flag the conflict rather than confidently picking the more specific-sounding source.

Scenario: An agent tasked with answering "is feature X enabled for customer Y" finds two pieces of context: a support ticket from two weeks ago where an engineer wrote "we turned this on for them," and a live query against the feature-flag service returning enabled: false for that customer. Which does it trust, and how should it respond? The live feature-flag query has stronger provenance for this specific question — it's a direct, current read of the actual system of record, whereas the support ticket is a secondhand, dated human statement that could be inaccurate even at the time or since reverted. The agent should answer based on the live flag state, but because the two sources genuinely conflict, it should surface both in its response rather than silently discarding the ticket, since the discrepancy itself (was it turned off again? was the ticket wrong?) may be exactly what the requester needs to know.

Try it

Take a prompt or agent behavior you maintain, and write three eval cases for it: one clear happy-path input with a specific expected output or tool-call sequence, one edge case you know is tricky, and one adversarial or malformed input. Then write down, for each, how you'd score pass/fail programmatically rather than by eyeballing it — that scoring rule is the part most people skip and the part that actually makes an eval suite catch regressions automatically.