This is the second of my posts on the talks at WeAreDevelopers World Congress in San Jose in September, and it covers the one on evaluating agents. The speakers built a refund agent for a customer-service demo in LangGraph and used it to show why grading an agent only on its final answer is dangerous. The framework they used to grade it instead is Agent GPA: Goal, Plan, Action. They cited it as coming from Snowflake AI Research.
The whole argument fits in one line from the talk: "A polished answer can hide an unsafe execution."
The Refund That Passed
Here is the example. The agent produced a refund that looked correct, and an evaluation that checked only the outcome passed it.
But the agent never called the policy check. The step that was supposed to decide whether the refund was allowed did not run. The answer was right anyway. Nothing about the process guaranteed it would be right next time, and nothing in a goal-only evaluation would ever have shown the gap.
That is the failure I care about most in any artificial intelligence (AI) system: it does not look like a failure. I wrote about the single-output version in It Fails Plausibly, Not Loudly, a handwritten date read with high confidence and the wrong year. An agent makes the problem worse, because there are more steps for it to hide in, and the final answer is the one place guaranteed to look tidy.
Three Grades, Not One
Agent GPA splits the evaluation into three questions, each graded on its own.
| Grade | The question | What failing it looks like |
|---|---|---|
| Goal | Did it produce the correct outcome, without harmful side effects? | A wrong result, or a right result that changed something it should not have |
| Plan | Was the path efficient? | Detours, redundant lookups, the same step taken twice |
| Action | Were the required tools called, in the right order, with valid arguments and permissions? | A skipped check, a call out of order, an argument the tool should have rejected |
The refund passes Goal and fails Action. That is the case for grading separately. One combined score would have averaged a skipped policy check into a respectable number, and a respectable number is exactly what nobody looks into.
Plan is the one people underrate. An agent that reaches the right answer by wandering through twice the necessary steps is not just slow. Every step costs something, and every extra step is another place for something to go wrong. An inefficient path is often the first visible sign of an agent that does not understand the task.
You Cannot Grade What You Did Not Record
None of this works without traces. A trace is the record of every step the agent took: each model call, each tool call, the arguments it passed, and what came back. In the talk, traces flowed out of the agent through OpenTelemetry and OpenInference, open standards for instrumenting software and AI applications, into Arize Phoenix, where the evaluation ran against them.
The specific tools matter less than the principle. If your agent does not leave a record of its steps, you can only grade its answer, which means you can only grade the part least likely to show you a problem. Instrumentation is not a nice-to-have for later. It is the precondition for the Plan and Action grades existing at all.
Bound the Damage, Then Fix the Behavior
The talk drew a distinction I found useful: iteration limits bound the damage, and progress detection fixes the behavior.
An iteration limit is a ceiling. The agent gets a fixed number of steps, then it stops. That guarantees a runaway agent cannot run forever. It does not make the agent any better. It only caps how bad a bad run can get.
Progress detection goes after the cause. It watches for repeated calls that bring back no new information, the agent asking the same question and getting the same answer, and treats that as the signal that the agent is stuck. A stuck agent does not know it is stuck. Something outside it has to notice.
You want both. The limit is the guardrail. Progress detection is the steering.
What I Would Do
My take: the Action grade is where a specification earns its keep. You cannot check whether the required tools were called in the right order unless someone wrote down, before the build, which tools are required and in what order. That is a specification question before it is an evaluation question.
So before building an agent, I would add three things to the specification:
- The required sequence. Which tool calls must happen, and which must happen before which. "The policy check runs before any refund is issued" is a requirement, and it is a testable one: the trace either shows it or it does not.
- The never list. Actions the agent must never take, or never take without a prior step. This is the third question I ask before building anything: what must it never do? For an agent, most of the answers turn out to be about actions, not outputs.
- The step budget. A rough count of how many steps a good run takes, so the Plan grade has a reference point and the iteration limit has a number that came from somewhere.
Then I would grade with a rubric, the same way I grade specifications: anchored scores, and for every score, the line from the trace that justifies it. A judge that has to quote the tool call cannot award an Action grade for a call that never happened. I wrote up how I build and calibrate that kind of judge in The Judge Needs a Rubric, and the same structure carries over to traces with almost no change.
The six questions I answer before building anything, including "what must it never do?", are on one page, free. For an agent, answer them about the steps, not just the answer.