Trajectory evaluation shows how a final answer can look perfect while the agent behind it has already failed.

Imagine an agent that tells a support team: “The customer record has been updated.” The sentence is clear and reassuring. However, the trace may show a different story. It may select a search tool instead of an update tool. It may pass "50" where the API requires the integer 50. Or it may never make the update call at all.

If your evaluation checks only the final text, each run can appear successful. This is like a software test that checks only whether a program exited with status code 0. The endpoint looked acceptable, but the route may have been wrong.

Key takeaways

• A correct final response does not prove that an autonomous agent used the right tools or completed the requested action.
• Trajectory evaluation scores observable execution steps, including tool calls, outputs, retries, and state transitions.
• Agent task success and tool use quality expose different failure modes, so evaluate them separately.
• Start with one important workflow and a small set of trace-level checks before building a larger evaluation system.

Why trajectory evaluation matters beyond prompt-to-output testing

For a simple, single-turn assistant, input and output testing is often a useful starting point. Give the model a prompt, compare its answer with the expected result, and inspect the response.

Autonomous agents expand the surface area. A run may retrieve information and select from several tools. It may build structured arguments, interpret a return payload, retry a failed operation, and change application state before it writes a reply. Therefore, the final answer is only the visible endpoint of that sequence.

Final-answer evaluation can catch a bad answer. However, it cannot reliably show whether a good answer came from a valid execution path. LangChain’s trajectory evaluation documentation describes evaluating an agent’s sequence of actions and intermediate tool use, not only the response at the end.

Trajectory evaluation scores the path to the answer

A trajectory is the observable record of how an agent handled a task. Depending on the system, that record can include the task input, active constraints, available tool schemas, tool selections, structured arguments, tool outputs, retries, state transitions, and final response.

This does not require capturing or exposing private chain-of-thought. Instead, retain operational evidence: which tool was selected, which arguments were sent, what the tool returned, and what the agent did next.

That evidence turns a vague failure into a diagnosis. An evaluator can identify a parameter type mismatch, an unsupported tool name, a missed negative constraint, or a response that claimed an action never happened. As a result, remediation becomes more precise. Work on systematic agent debugging, such as Microsoft Research’s AgentRx framework, reinforces the need to investigate the execution process rather than relying solely on the final reply.

Pillar one: Agent Task Success

Agent task success asks whether the agent behaved in a way that aligned with the requested goal and its constraints. A fluent answer is not enough.

• Hallucinations: Did the agent claim it lacked a tool that was available? Did it state that it sent an email, updated a record, or completed a search when no such action occurred?
• Instruction following: Did it obey negative constraints? Did it ask for permission that the task already granted? Did it stop halfway through a task that it was expected to complete?
• Tool-output handling: Did it correctly interpret a valid API response before choosing the next step?

Consider a task: update a customer’s address, but do not send a confirmation email. An expected trace selects the customer-update tool and sends a valid address payload. It receives a success response, confirms the update, and does not call any email tool.

A failing trace might send the update successfully and then trigger an email despite the negative constraint. Another might receive a valid success payload but fail to parse it. Consequently, it may ask for unnecessary permission and stop. Both failures can be hidden if the evaluator sees only “The address has been updated.”

Pillar two: Tool Use Quality

Tool use quality asks whether the agent interacted with its available interfaces precisely. This is not the same as task success. An agent may have the right intention and still produce invalid or unsafe tool calls.

• Parameter errors: A schema expects the integer 50, but the agent sends the string "50". It might also swap two fields that have similar names.
• Selection and syntax: The agent chooses a search function when it needs an update function, or sends malformed JSON that a tool cannot accept.
• Tool hallucinations: The agent invents a function name or parameter key that does not exist in the available schema.

These checks are valuable because the final text can remain plausible after a tool failure. A model can write a convincing completion message even when the API rejected its payload or the tool call was never possible.

Build an evaluation record that engineers can inspect

The goal is not to log everything indiscriminately. Instead, retain enough evidence to answer a practical question: where did this run deviate from the intended path?

At minimum, record the task input, applicable constraints, available tools and schemas, each invocation, returned payload, state transition, final answer, and evaluator verdict. This record gives developers and reviewers a shared artifact for debugging.

What trajectory evaluation does not solve

Trajectory evaluation does not prove that an external tool was correct in the real world. It does not replace authorization controls, validation at system boundaries, or final-answer evaluation. It does not, by itself, establish that an agent is reliable or safe.

Its value is diagnostic coverage. Outcome checks tell you whether the destination was acceptable. In contrast, trajectory checks tell you whether the agent took an acceptable path to get there. Production systems that retrieve data, call tools, or mutate state need both.

Start with one high-value workflow

Teams deploying agents that act, retrieve, or mutate state should use trajectory evaluation. Teams still experimenting with a simple single-turn assistant can begin with output evaluation. Before introducing meaningful autonomy, however, they should instrument execution traces.

The safest useful first step is narrow. Choose one high-value workflow and define a handful of trace-level checks for constraints, tool selection, parameter validity, tool-output handling, and action completion. Then review the failures and expand the suite around the patterns you find.

In Part 2, I will turn that foundation into a five-stage evaluation flywheel and show how evolutionary prompt engineering can use the resulting evidence.

Categories: AI Agents, Artificial Intelligence, Software Architecture

Tags: AI Agents, AI automation, LLM, System Architecture, Workflow Automation

The post Why Final-Answer Evals Leave AI Agent Failures Invisible appeared first on Alpesh Kumar.