TechByteByByte

Reflection and Self-Correction

Closing Level 4: how an agent can evaluate its own output and revise it before finishing — the generate-evaluate-revise-retry loop, and the real risks of relying on self-evaluation.

#AI Agents#AI#Reflection#Level 4

Begin with the problem

An agent can inspect and revise an output, but self-review is not an independent guarantee of correctness because the same model may repeat the same mistake.

draft → evaluate against criteria → revise → accept or stop at revision limit

What you will learn

  • Explain reflection as a generate–evaluate–revise process.
  • Separate self-critique from independent verification and human review.
  • Prevent endless revision with limits and acceptance rules.
  • Choose reflection only when its extra cost produces measurable improvement.

Current real-system grounding: Google’s tool documentation shows the critical difference between provider-executed built-in tools and custom functions executed by your application.

These official links document available product features. They do not reveal every provider’s private implementation, hidden reasoning, default setting, or internal limit.

1. The problem this module solves

Module 9’s ReAct recorded a generated action rationale before each action. This module closes Level 4 with a related but distinct capability: having an agent evaluate its own output — not just its next action — and revise it before considering the task complete.

2. Why an Agent Might Need to Evaluate Its Own Output

An agent GENERATES a draft response, a piece of code, or a plan.

Should it just RETURN this immediately? Or should it FIRST check:
"does this meet the requirements of the task?"

Reflection adds exactly this check — a real, explicit step where the agent (or a separate evaluation call) assesses its own output against the task’s actual requirements, BEFORE considering the task finished.

3. The Reflection Loop

flowchart TD
    G[Generate Output] --> E[Evaluate]
    E --> Q{Is result<br/>acceptable?}
    Q -->|Yes| F[Finish]
    Q -->|No| R[Revise]
    R --> G
Generate:      produce an initial output
Evaluate:          check it against real requirements or quality
                 criteria
Is it acceptable?:      a real YES/NO judgment
   YES -> Finish
   NO -> Revise -> back to Generate (with the critique as new
        context)

4. What “Evaluate” Means

Evaluation can be implemented several ways:

- Checking against EXPLICIT requirements (does the output mention
  required elements? Section 12's code example does exactly this)
- A SEPARATE LLM call specifically asked to critique the output
  (directly connecting to your Generative AI course's LLM-as-judge
  concept)
- Automated checks (does generated code actually RUN without errors?)

The important point: evaluation should be a real, distinct step — not simply assuming the first generated output is automatically correct.

5. A Real Developer Example

TechCorp’s agent drafts a reply to a customer complaint:

AttemptDraftRequirements CheckResult
1“Sorry for the trouble.”Missing: mentions the specific order, mentions the resolution (refund)❌ Not acceptable
2“Sorry for the delay with your order. We apologize.”Missing: mentions the resolution (refund)❌ Not acceptable
3“Sorry for the delay with your order #4471. We apologize and are offering a refund.”All required elements present✅ Acceptable

Each unacceptable attempt informs the next draft — the critique (“missing: mentions the resolution”) becomes context for generating a better attempt, exactly Module 4’s loop, now applied to refining a single output.

6. The Risks of Self-Evaluation

This is worth being direct about, since reflection is not a perfect solution:

- The SAME model doing the evaluating may share the SAME blind spots
  as the model that generated the output -- it might fail
  to notice a flaw it was already prone to making

- An agent can be CONFIDENTLY WRONG in its self-evaluation, judging
  a flawed output as "acceptable" (directly connecting to
  your Generative AI course's hallucination and overconfidence
  discussion)

- Reflection adds real ITERATIONS, cost, and latency -- not free

Incorrect idea: Reflection improves output quality on average, but it is not a guarantee of correctness. A separate, independent evaluation (a different model, or explicit programmatic checks, Section 4) tends to catch more real issues than self-evaluation by the same model that generated the output.

Why it is incorrect:

7. A Simple Agentic AI Connection

Reflection connects directly to Module 8’s dynamic planning — just as a plan might be revised when new information invalidates it, reflection revises a specific output when self-evaluation reveals it doesn’t meet requirements. Both are instances of the same broader idea: an agent’s initial attempt at something is treated as provisional, not final, until it’s actually verified.

8. How Is This Used in AI?

🤖 How Is This Used in AI?

Reflection is a standard technique in production agent systems generating important outputs — code, customer-facing content, structured data — where a quick, low-cost self-check before finalizing measurably reduces the rate of obviously flawed or incomplete results reaching the end user.

9. Real-World Applications

  • Code-generation agents checking whether generated code actually compiles or passes basic tests before presenting it
  • Customer support agents verifying a drafted response covers all required elements
  • Content generation agents checking outputs against explicit style or policy requirements

10. Common Mistakes

Incorrect idea: Treating self-evaluation as a guaranteed correctness check.

Why it is incorrect: As shown directly in Section 6, the same model can share the same blind spots as the generation step.

Incorrect idea: Implementing reflection without a real limit on retries.

Why it is incorrect: Directly connecting to Module 4’s max-iterations principle — an unbounded generate-evaluate-revise loop risks the exact same runaway-execution problem.

Incorrect idea: Using vague evaluation criteria.

Why it is incorrect: As shown directly in Section 4-5, explicit, checkable requirements produce more reliable evaluation than a vague “is this good?” judgment.

11. Limitations

  • Reflection adds real cost and latency for every additional generate-evaluate-revise cycle — a real trade-off against simply accepting the first output
  • Self-evaluation’s real reliability limit (Section 6) means reflection reduces, but does not eliminate, the risk of a flawed final output reaching the user

12. Quick Reference

flowchart LR
    D1[Draft 1] --> Ev1{Meets<br/>requirements?}
    Ev1 -->|No| D2[Draft 2<br/>informed by critique]
    D2 --> Ev2{Meets<br/>requirements?}
    Ev2 -->|No| D3[Draft 3]
    Ev2 -->|Yes| Done1[Finish]
    D3 --> Done2[Finish<br/>or max retries]

13. Code — Implementing a Generate-Evaluate-Revise Loop

🎯 Target of this example: implement Section 5’s real developer example directly — a loop that evaluates each draft against explicit requirements, revising until the output is acceptable or a retry limit is reached, exactly Module 4’s safety-limit principle applied to reflection.

Example 1 — Simple

from dataclasses import dataclass

@dataclass
class ReflectionResult:
    output: str
    is_acceptable: bool
    critique: str = None

def evaluate_output(output: str, requirements: list) -> ReflectionResult:
    """A a self-evaluation step -- checking the output against
    EXPLICIT requirements (Section 4), rather than just accepting it
    as-is."""
    missing = [req for req in requirements if req.lower() not in output.lower()]
    if missing:
        return ReflectionResult(
            output=output, is_acceptable=False,
            critique=f"Missing required elements: {missing}",
        )
    return ReflectionResult(output=output, is_acceptable=True)

def generate_and_reflect(draft_fn, requirements: list, max_retries: int = 3) -> ReflectionResult:
    """Generate -> Evaluate -> Revise -> Retry, exactly Section 3's
    reflection loop, WITH a real retry limit (Section 10-11)."""
    attempt = 0
    output = draft_fn(attempt)
    result = evaluate_output(output, requirements)

    while not result.is_acceptable and attempt < max_retries:
        attempt += 1
        output = draft_fn(attempt)
        result = evaluate_output(output, requirements)

    return result

def draft_email(attempt: int) -> str:
    """Simulates progressively IMPROVING drafts across retries --
    exactly Section 5's real developer example."""
    drafts = [
        "Sorry for the trouble.",
        "Sorry for the delay with your order. We apologize.",
        "Sorry for the delay with your order #4471. We apologize and are offering a refund.",
    ]
    return drafts[min(attempt, len(drafts) - 1)]

requirements = ["order", "refund"]
result = generate_and_reflect(draft_email, requirements)

print(f"Final output: {result.output}")
print(f"Acceptable: {result.is_acceptable}")

Expected Output:

Final output: Sorry for the delay with your order #4471. We
apologize and are offering a refund.
Acceptable: True

What we conclude from this example: the loop correctly rejects the first two drafts (missing “order” and “refund” respectively) and only accepts the third, complete draft — exactly Section 5’s table, made into working, observable behavior.

Example 2 — Intermediate

from dataclasses import dataclass

@dataclass
class ReflectionResult:
    output: str
    is_acceptable: bool
    critique: str = None
    attempt_number: int = 0

def evaluate_output(output: str, requirements: list) -> tuple:
    missing = [req for req in requirements if req.lower() not in output.lower()]
    return (len(missing) == 0, f"Missing: {missing}" if missing else None)

def generate_and_reflect_with_trace(draft_fn, requirements: list, max_retries: int = 3) -> list:
    """Extends Example 1 to return the FULL trace of every attempt
    and its critique -- directly demonstrating Section 5's table as
    an inspectable trajectory, not just a final result."""
    trace = []
    for attempt in range(max_retries + 1):
        output = draft_fn(attempt)
        is_acceptable, critique = evaluate_output(output, requirements)
        trace.append(ReflectionResult(output, is_acceptable, critique, attempt))
        if is_acceptable:
            break
    return trace

def draft_email(attempt: int) -> str:
    drafts = [
        "Sorry for the trouble.",
        "Sorry for the delay with your order. We apologize.",
        "Sorry for the delay with your order #4471. We apologize and are offering a refund.",
    ]
    return drafts[min(attempt, len(drafts) - 1)]

trace = generate_and_reflect_with_trace(draft_email, ["order", "refund"])

for entry in trace:
    status = "ACCEPTED" if entry.is_acceptable else f"REJECTED ({entry.critique})"
    print(f"Attempt {entry.attempt_number + 1}: {status}")
    print(f"  Draft: {entry.output}")

Expected Output:

Attempt 1: REJECTED (Missing: ['order', 'refund'])
Attempt 2: REJECTED (Missing: ['refund'])
Attempt 3: ACCEPTED
  Draft: Sorry for the delay with your order #4471. We apologize
and are offering a refund.

What we conclude from this example: the full trace shows EXACTLY which requirements were missing at each rejected attempt — the first draft was missing both “order” and “refund,” the second only “refund” — directly demonstrating real, incremental improvement across attempts, informed by specific, explicit critiques rather than vague rejection.

Example 3 — Production Grade

from dataclasses import dataclass, field
from enum import Enum

class ReflectionOutcome(Enum):
    ACCEPTED = "accepted"
    MAX_RETRIES_EXCEEDED = "max_retries_exceeded_unresolved"

@dataclass
class ReflectionSession:
    final_output: str
    outcome: ReflectionOutcome
    total_attempts: int
    trace: list = field(default_factory=list)

class ReflectionAgent:
    """A production-style reflection agent implementing Section 10's
    real safety requirement -- a hard retry limit, directly
    mirroring Module 4's max_iterations principle, so an agent that
    CANNOT satisfy its own requirements doesn't loop
    forever."""

    def __init__(self, requirements: list, max_retries: int = 3):
        self.requirements = requirements
        self.max_retries = max_retries

    def _evaluate(self, output: str) -> tuple:
        missing = [req for req in self.requirements if req.lower() not in output.lower()]
        return (len(missing) == 0, missing)

    def run(self, draft_fn) -> ReflectionSession:
        trace = []
        for attempt in range(self.max_retries + 1):
            output = draft_fn(attempt)
            is_acceptable, missing = self._evaluate(output)
            trace.append({"attempt": attempt + 1, "output": output, "accepted": is_acceptable, "missing": missing})

            if is_acceptable:
                return ReflectionSession(output, ReflectionOutcome.ACCEPTED, attempt + 1, trace)

        # could NOT produce an acceptable output within the limit
        return ReflectionSession(trace[-1]["output"], ReflectionOutcome.MAX_RETRIES_EXCEEDED,
                                  self.max_retries + 1, trace)

def always_incomplete_draft(attempt: int) -> str:
    """Simulates a stuck generator -- never mentions
    'refund', no matter how many times it retries."""
    return f"Attempt {attempt}: Sorry for the delay with your order #4471."

agent = ReflectionAgent(requirements=["order", "refund"], max_retries=2)
session = agent.run(always_incomplete_draft)

print(f"Outcome: {session.outcome.value}")
print(f"Total attempts: {session.total_attempts}")
print(f"Final output (possibly still imperfect): {session.final_output}")

Expected Output:

Outcome: max_retries_exceeded_unresolved
Total attempts: 3
Final output (possibly still imperfect): Attempt 2: Sorry for the
delay with your order #4471.

What we conclude from this example: when the draft generator is , permanently unable to satisfy the “refund” requirement, the agent correctly stops after exactly max_retries + 1 attempts rather than looping forever — exactly Section 6 and 10’s warning made into enforced, structural safety, directly mirroring Module 4’s max_iterations principle applied specifically to the reflection loop.

14. Interview Questions

Q: Describe the reflection pattern and explain what real problem it solves.

Ans: Reflection has an agent generate an output, then explicitly evaluate that output against actual requirements or quality criteria before considering the task complete — if the evaluation finds the output unacceptable, the agent revises and tries again. This solves the problem of an agent’s first generated attempt not being automatically assumed correct — reflection adds a real, explicit verification step rather than blindly returning whatever was generated first.

Q: What are the real risks of relying on self-evaluation, and why doesn’t reflection guarantee a correct final output?

Ans: The same model performing the evaluation may share the same blind spots as the model that generated the output, potentially failing to notice a flaw it was already prone to making. An agent can also be confidently wrong in its self-assessment, judging a flawed output as acceptable. Reflection improves output quality on average, but a separate, independent evaluation — a different model or explicit programmatic checks — tends to catch more real issues than self-evaluation by the same model that produced the output.

Q: Why is a maximum retry limit necessary for a reflection loop, similar to the max-iterations requirement for the standard agent loop?

Ans: Without a hard limit, an agent that’s unable to satisfy its own evaluation criteria — whether due to a fundamental misunderstanding of the requirements or an impossible constraint — could loop through generate-evaluate-revise indefinitely, consuming unbounded time and cost with no guarantee of eventually succeeding. Exactly mirroring Module 4’s max-iterations safety requirement, a retry limit ensures the reflection loop terminates and reports failure gracefully rather than running forever.

Q: Design an evaluation approach for a reflection loop that would be more reliable than having the same model that generated the output also judge whether it’s acceptable.

Ans: I’d use explicit, checkable requirements wherever possible — verifying specific elements are present, or running automated checks like whether generated code actually executes without errors — rather than a vague, subjective “is this good?” judgment made by the same model. Where a nuanced quality judgment is needed, I’d prefer a separate evaluation call, potentially using a different model or a more constrained, specifically-prompted evaluation task, since this reduces the risk of both the generation and evaluation steps sharing the same underlying blind spots.

15. What You Should Remember

  • Reflection adds a real generate-evaluate-revise-retry loop, checking an agent’s output against real requirements before finishing — verified directly by observing progressively improving drafts across attempts, each informed by a specific critique.
  • Self-evaluation has real reliability limits — the same model can share blind spots with itself — reflection improves quality on average but doesn’t guarantee correctness.
  • A maximum retry limit is required, directly mirroring Module 4’s max-iterations principle — verified directly through an agent that correctly stops rather than looping forever when it cannot satisfy its own requirements.

16. Quick Practice

Design explicit, checkable requirements (following Section 4’s approach) for evaluating an output type relevant to your own work — what specific, concrete elements would you check for, rather than relying on a vague “is this good?” judgment?

17. Next Step

Next: Module 11 — Agent Memory — Level 5 begins here: why memory is needed, and the critical distinction between state, context, conversation history, and memory — four different concepts this course will keep carefully separate.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed