TechByteByByte

Agent Evaluation

Level 8 begins here: why evaluating an agent is harder than evaluating a single LLM response, and the multiple, distinct dimensions — completion, tool accuracy, planning quality, cost, safety — that actually matter.

#AI Agents#AI#Evaluation#Level 8

Begin with the problem

A fluent final answer does not prove the agent succeeded. Evaluation checks goal completion, tool choices, safety, cost, latency, and behavior across many cases.

evaluation case → run agent trajectory → score outcome + path + safety + cost → compare versions

What you will learn

  • Evaluate an entire trajectory instead of judging only the final sentence.
  • Measure task success, tool accuracy, safety, latency, cost, and recovery behavior.
  • Build datasets containing normal, edge, adversarial, and tool-failure cases.
  • Combine automated checks, model graders, and human review appropriately.

Current real-system grounding: OpenAI’s evaluation guidance supports dataset-based testing, and Google’s tools guide makes the application/tool execution boundary explicit.

These official links document available product features. They do not reveal every provider’s private implementation, hidden reasoning, default setting, or internal limit.

1. The problem this module solves

Modules 1-19 built and secured a real agent. Level 8 addresses production readiness — starting with a question worth asking honestly: how do you actually know whether an agent is good? This is harder than evaluating a single LLM response, and this module explains precisely why.

2. Why Agent Evaluation Is Harder Than LLM Evaluation

Evaluating a SINGLE LLM response: was THIS specific output good?
                                  (your Prompt Engineering course's
                                  evaluation coverage)

Evaluating an AGENT: was the ENTIRE multi-step TRAJECTORY good --
                     not just the final answer, but every decision,
                     tool call, and intermediate step along the way?

A important, honest point: an agent can reach the RIGHT final answer through a bad process (wasted steps, wrong tool calls that happened to not matter, excessive cost) — or reach a WRONG final answer despite good reasoning at every individual step (Module 19’s tool failure, for instance). Evaluating only the final answer misses BOTH of these important cases.

3. The Distinct Dimensions to Evaluate

DimensionWhat It Measures
Task completionDid the agent actually achieve the stated goal?
Tool selection accuracyDid it choose the correct tool at each step? (Module 6, 19)
Tool argument accuracyWere the generated parameters correct? (Module 7)
Planning qualityWas the real decomposition (Module 8) sensible and complete?
GroundednessWere claims supported by actual observations? (Module 19’s hallucination)
SafetyDid the agent respect guardrails and permission boundaries? (Module 17-18)
ReliabilityDoes the SAME task produce consistent results across runs? (Module 5’s non-determinism)
CostHow much did the ENTIRE trajectory consume? (Module 19’s cost explosion)
LatencyHow long did the real end-to-end task take?
Number of stepsWas the trajectory efficient, or did it take unnecessary detours?
Failure rateAcross MANY runs, how often does the agent fail?

4. Evaluating the Trajectory, Not Just the Destination

flowchart TD
    T[Full Agent Trajectory] --> S1[Step 1: was the DECISION good?]
    T --> S2[Step 2: was the TOOL CALL correct?]
    T --> S3[Step 3: was the OBSERVATION used correctly?]
    T --> F[Final Answer: was it CORRECT?]
    S1 --> Score[Combined Evaluation]
    S2 --> Score
    S3 --> Score
    F --> Score

This directly connects to Module 9’s ReAct trace — a > logged, step-by-step trajectory is precisely what makes trajectory- level evaluation possible at all. Without a visible, structured trace, you can only evaluate the final answer, missing everything Section 2 described.

5. A Worked Example Evaluation Flow

TechCorp evaluates 50 real agent runs against a GOLDEN dataset
(directly your RAG course's Module 32 evaluation practice, applied
to agent trajectories):

For EACH run, score:
1. Task completion (did it resolve the ticket?)
2. Tool accuracy (were the RIGHT tools called, with RIGHT arguments?)
3. Number of steps (was it EFFICIENT, or did it wander?)
4. Cost (total tokens/API calls consumed)
5. Safety (did it respect guardrails? Module 17)

AGGREGATE across all 50 runs:
- Average task completion rate
- Average tool accuracy
- Average steps per task
- Average cost per task
- Number of real guardrail violations (should be ZERO)

6. A Real Developer Example

TechCorp compares two versions of their support agent:

MetricVersion AVersion Breal Winner
Task completion rate87%91%B
Average tool accuracy94%89%A
Average steps per task3.25.1A (more efficient)
Average cost per task$0.04$0.09A (cheaper)
Safety violations00Tie

Notice: NEITHER version is, unambiguously “better” — B completes more tasks but at higher cost and lower per-step accuracy. This is EXACTLY why evaluating multiple, distinct dimensions matters — a single “task completion” number alone would have missed B’s real efficiency and accuracy regression.

7. A Simple Agentic AI Connection

Trajectory-level evaluation directly connects to Module 15’s multi-agent systems — a multi-agent system’s evaluation needs to assess not just the overall task outcome, but which specific agent’s reasoning contributed to a failure, exactly extending this module’s per-step evaluation principle to per-agent evaluation in a coordinated system.

8. How Is This Used in AI?

🤖 How Is This Used in AI?

Mature agent engineering teams maintain real, ongoing evaluation infrastructure — golden trajectories, multi-dimensional scoring, and systematic before/after comparison before deploying any change — directly mirroring your RAG course’s Module 32 evaluation discipline, now applied across an agent’s entire multi-step trajectory rather than a single retrieval-and-generation pass.

9. Real-World Applications

  • Comparing candidate models, prompts, or tool configurations before deployment
  • Regression testing when changing any agent component (Module 22’s frameworks, tool definitions, prompts)
  • Ongoing production quality monitoring across distinct metrics

10. Common Mistakes

Incorrect idea: Evaluating only final task completion.

Why it is incorrect: As shown directly in Section 2 and 6, this misses important process quality and efficiency signals.

Incorrect idea: Treating one metric as unambiguously “the” measure of agent quality.

Why it is incorrect: As shown directly in Section 6, real comparisons often involve trade-offs across multiple dimensions.

Incorrect idea: Not accounting for non-determinism when evaluating.

Why it is incorrect: Directly connecting to Module 5, Section 6 — a single run’s result may not be representative; multiple trials matter.

11. Limitations

  • Building a representative golden dataset of agent tasks requires real, ongoing effort — a narrow or stale dataset provides false confidence
  • Some dimensions (like “was the reasoning sound?”) require real human or LLM-judge review, which has its own real limitations (directly your RAG course’s Module 32 caveat)

12. Quick Reference

flowchart LR
    Trace[Full Trajectory] --> D1[Task Completion]
    Trace --> D2[Tool Accuracy]
    Trace --> D3[Planning Quality]
    Trace --> D4[Cost/Latency]
    Trace --> D5[Safety]
    D1 --> Agg[Aggregate Across Many Runs]
    D2 --> Agg
    D3 --> Agg
    D4 --> Agg
    D5 --> Agg

13. Code — Implementing Multi-Dimensional Agent Evaluation

🎯 Target of this example: implement Section 3 and 5’s real evaluation flow directly — scoring a single agent run across distinct dimensions (task completion, tool accuracy, steps, cost), exactly Section 6’s comparison table made into working, computable logic.

Example 1 — Simple

from dataclasses import dataclass

@dataclass
class AgentEvalResult:
    task_completed: bool
    correct_tool_selections: int
    total_tool_calls: int
    num_steps: int
    total_cost_estimate: float

def evaluate_agent_run(trace: list, expected_final_state: dict, actual_final_state: dict) -> AgentEvalResult:
    """Evaluates MULTIPLE distinct dimensions (Section 3)
    of an agent run, not just whether the final answer looks right."""
    task_completed = actual_final_state == expected_final_state
    correct_tools = sum(1 for step in trace if step.get("tool_was_correct", False))
    total_tools = sum(1 for step in trace if "tool" in step)
    num_steps = len(trace)
    cost = sum(step.get("cost", 0.0) for step in trace)

    return AgentEvalResult(
        task_completed=task_completed,
        correct_tool_selections=correct_tools,
        total_tool_calls=total_tools,
        num_steps=num_steps,
        total_cost_estimate=cost,
    )

trace = [
    {"tool": "check_order_status", "tool_was_correct": True, "cost": 0.002},
    {"tool": "check_shipping_carrier", "tool_was_correct": True, "cost": 0.003},
    {"decision": "escalate", "cost": 0.001},
]

expected = {"order_status": "late", "carrier_status": "delivered", "decision": "escalate"}
actual = {"order_status": "late", "carrier_status": "delivered", "decision": "escalate"}

result = evaluate_agent_run(trace, expected, actual)
print(f"Task completed: {result.task_completed}")
print(f"Tool accuracy: {result.correct_tool_selections}/{result.total_tool_calls}")
print(f"Steps: {result.num_steps}")
print(f"Estimated cost: ${result.total_cost_estimate}")

Expected Output:

Task completed: True
Tool accuracy: 2/2
Steps: 3
Estimated cost: $0.006

What we conclude from this example: the evaluation captures FOUR distinct dimensions from a single trace — not just whether the final state matched, but tool accuracy, step count, and cost too — exactly Section 3’s table, made into real, computable metrics from a real trajectory.

Example 2 — Intermediate

def aggregate_evaluation(results: list) -> dict:
    """Directly implements Section 5's aggregation step -- averaging
    metrics across MULTIPLE runs, exactly your RAG course's Module
    32 golden-dataset evaluation practice."""
    completion_rate = sum(1 for r in results if r["task_completed"]) / len(results)
    avg_tool_accuracy = sum(r["tool_accuracy"] for r in results) / len(results)
    avg_steps = sum(r["num_steps"] for r in results) / len(results)
    avg_cost = sum(r["cost"] for r in results) / len(results)

    return {
        "completion_rate": round(completion_rate, 3),
        "avg_tool_accuracy": round(avg_tool_accuracy, 3),
        "avg_steps": round(avg_steps, 2),
        "avg_cost": round(avg_cost, 4),
    }

# Simulates 5 real agent runs with VARYING outcomes
runs = [
    {"task_completed": True, "tool_accuracy": 1.0, "num_steps": 3, "cost": 0.006},
    {"task_completed": True, "tool_accuracy": 0.5, "num_steps": 5, "cost": 0.011},
    {"task_completed": False, "tool_accuracy": 1.0, "num_steps": 4, "cost": 0.008},
    {"task_completed": True, "tool_accuracy": 1.0, "num_steps": 2, "cost": 0.004},
    {"task_completed": True, "tool_accuracy": 0.8, "num_steps": 3, "cost": 0.007},
]

aggregated = aggregate_evaluation(runs)
for metric, value in aggregated.items():
    print(f"{metric}: {value}")

Expected Output:

completion_rate: 0.8
avg_tool_accuracy: 0.86
avg_steps: 3.4
avg_cost: 0.0072

What we conclude from this example: aggregating across 5 real runs (one of which failed to complete) produces representative average metrics — an 80% completion rate immediately reveals real room for improvement that a single successful run wouldn’t have shown, exactly Section 11’s warning about single-run evaluation providing false confidence.

Example 3 — Production Grade

from dataclasses import dataclass, field

@dataclass
class VersionComparison:
    metric: str
    version_a: float
    version_b: float
    genuine_winner: str

class AgentVersionComparator:
    """A production-style comparator implementing Section 6's REAL
    developer example -- comparing two agent versions across MULTIPLE
    dimensions, explicitly flagging which version wins each
    metric, rather than reducing everything to one ambiguous number."""

    HIGHER_IS_BETTER = {"completion_rate", "tool_accuracy"}
    LOWER_IS_BETTER = {"avg_steps", "avg_cost"}

    def compare(self, metrics_a: dict, metrics_b: dict) -> list:
        comparisons = []
        for metric in metrics_a:
            a_val, b_val = metrics_a[metric], metrics_b[metric]
            if metric in self.HIGHER_IS_BETTER:
                winner = "A" if a_val > b_val else ("B" if b_val > a_val else "Tie")
            elif metric in self.LOWER_IS_BETTER:
                winner = "A" if a_val < b_val else ("B" if b_val < a_val else "Tie")
            else:
                winner = "Tie" if a_val == b_val else "Unclear"
            comparisons.append(VersionComparison(metric, a_val, b_val, winner))
        return comparisons

version_a = {"completion_rate": 0.87, "tool_accuracy": 0.94, "avg_steps": 3.2, "avg_cost": 0.04}
version_b = {"completion_rate": 0.91, "tool_accuracy": 0.89, "avg_steps": 5.1, "avg_cost": 0.09}

comparator = AgentVersionComparator()
results = comparator.compare(version_a, version_b)

for r in results:
    print(f"{r.metric}: A={r.version_a}, B={r.version_b} -> Winner: {r.genuine_winner}")

a_wins = sum(1 for r in results if r.genuine_winner == "A")
b_wins = sum(1 for r in results if r.genuine_winner == "B")
print(f"\nA wins {a_wins} metric(s), B wins {b_wins} metric(s) -- no clear overall winner without weighing priorities.")

Expected Output:

completion_rate: A=0.87, B=0.91 -> Winner: B
tool_accuracy: A=0.94, B=0.89 -> Winner: A
avg_steps: A=3.2, B=5.1 -> Winner: A
avg_cost: A=0.04, B=0.09 -> Winner: A

A wins 3 metric(s), B wins 1 metric(s) -- no clear overall
winner without weighing priorities.

What we conclude from this example: exactly Section 6’s table, reproduced as working, automated comparison logic — Version A wins three of four metrics (accuracy, efficiency, cost) while Version B wins only completion rate, directly demonstrating Section 6’s real point: real agent comparisons involve trade-offs that a single aggregate score would obscure, requiring an explicit decision about which dimensions matter most for the specific application.

14. Interview Questions

Q: Why is evaluating an agent harder than evaluating a single LLM response?

Ans: A single LLM response evaluation asks whether one specific output was good. Agent evaluation must assess an entire multi-step trajectory — every decision, tool call, and intermediate step, not just the final answer. An agent can reach the right final answer through a inefficient or costly process, or reach a wrong answer despite sound reasoning at individual steps if something like a tool failure occurred along the way. Evaluating only the final output misses both of these important cases.

Q: List at least five distinct dimensions worth measuring when evaluating an agent, beyond simple task completion.

Ans: Tool selection accuracy (was the correct tool chosen at each step), tool argument accuracy (were the generated parameters correct), planning quality (was the task decomposition sensible), groundedness (were claims actually supported by observations), safety (did the agent respect guardrails and permission boundaries), cost (total resources consumed across the trajectory), latency (total time taken), and reliability (does the same task produce consistent results across multiple runs).

Q: Using the two-version comparison example, explain why neither version can be called unambiguously “better” without further context.

Ans: Version A had higher tool accuracy, fewer steps, and lower cost, while Version B had a higher task completion rate. Neither dominates across every metric — improving completion rate came at the cost of efficiency and accuracy in this comparison. Determining which version is better requires explicitly weighing which dimensions matter most for the specific application — a cost-sensitive deployment might prefer Version A, while an application where completing more tasks matters most, even at higher cost, might prefer Version B.

Q: Why is single-run evaluation potentially misleading for an agent, and what should be done instead?

Ans: Because LLM generation involves non-determinism, a single run’s result may not be representative of the agent’s typical behavior — running the same task multiple times could produce different outcomes. Aggregating results across many runs against a representative golden dataset provides a more reliable picture of average performance, revealing issues like an actual completion rate below 100% that a single successful run would have completely missed.

15. What You Should Remember

  • Agent evaluation must assess the entire trajectory, not just the final answer — a good final answer can come from a bad process, and vice versa.
  • Multiple, distinct dimensions — task completion, tool accuracy, planning quality, cost, safety — each capture something different, verified directly through a single evaluation function computing four separate metrics from one trace.
  • Real comparisons involve trade-offs across dimensions, not a single unambiguous winner — verified directly through a comparator correctly identifying that neither of two agent versions dominates every metric.

16. Quick Practice

Design a golden evaluation dataset (3-5 test tasks) for an agent in your own domain, specifying for each task: the expected final state, the correct tool sequence, and a reasonable cost/step budget — exactly the kind of dataset Section 5’s evaluation flow requires.

17. Next Step

Next: Module 21 — Observability — why ordinary application logging isn’t enough for agents, and what a real agent trace needs to capture to make Module 19’s diagnostic process actually possible in production.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed