TechByteByByte

Multi-Agent Systems

Closing Level 6: when a single agent isn't enough, and the architectural patterns — sequential, parallel, supervisor, hierarchical, and more — for coordinating multiple specialized agents together.

#AI Agents#AI#Multi-Agent#Level 6

Begin with the problem

Multiple agents divide roles or context, but coordination is real software—not magic collaboration—and can cost more than one well-designed agent.

supervisor or shared plan → specialist agents → handoffs → combined result → final check

What you will learn

  • Explain what makes multiple model roles a multi-agent system.
  • Compare sequential, supervisor, peer, and specialist collaboration patterns.
  • Trace messages, shared state, handoffs, and failure propagation between agents.
  • Decide when one well-designed agent is simpler and more reliable.

Current real-system grounding: OpenAI’s official agent quickstart includes tools and handoffs, while Google’s Agents overview lists current agent frameworks and managed agents.

These official links document available product features. They do not reveal every provider’s private implementation, hidden reasoning, default setting, or internal limit.

1. The problem this module solves

Module 13 covered single-agent patterns; this module closes Level 6 with Module 1’s final evolution stage: multi-agent systems. Every concept from this entire course — the loop, tools, planning, memory — still applies to each individual agent; this module covers what changes when several of them work together.

2. Why One Agent May Not Be Enough

A SINGLE agent handling a BROAD, complex task faces real
challenges:

- Its CONTEXT (Module 3, 11) can become overloaded trying
  to hold everything relevant to a broad task at once
- Its TOOL SET (Module 6) can become unwieldy if it needs to select
  correctly among MANY unrelated tools
- Different SUBTASKS may benefit from different, SPECIALIZED
  reasoning approaches (a coding subtask vs. a research subtask)

Multi-agent systems address this by having SEVERAL agents, each SPECIALIZED for a narrower scope — directly extending Module 8’s task decomposition idea, but now assigning each subtask to its OWN dedicated agent, rather than one agent handling everything sequentially.

3. Specialized Agents and Task Delegation

Instead of ONE agent with a huge toolset and broad context:

RESEARCHER agent -- specialized in SEARCH and information gathering
CODER agent -- specialized in writing and testing code
REVIEWER agent -- specialized in critique and quality checking (Module
                 10's reflection, at the multi-agent level)

Each specialist has a narrower, more focused context and tool set — exactly addressing Section 2’s overload concern.

4. Architecture Patterns

flowchart LR
    subgraph Sequential
        A1[Agent A] --> A2[Agent B] --> A3[Agent C]
    end
flowchart TD
    subgraph Supervisor
        S[Supervisor] --> R[Researcher]
        S --> C[Coder]
        S --> Rev[Reviewer]
    end
flowchart LR
    subgraph Parallel
        P[Task] --> PA[Agent A]
        P --> PB[Agent B]
        PA --> M[Merge Results]
        PB --> M
    end
flowchart TD
    subgraph Hierarchical
        Top[Top-Level Supervisor] --> Mid1[Sub-Supervisor A]
        Top --> Mid2[Sub-Supervisor B]
        Mid1 --> W1[Worker]
        Mid1 --> W2[Worker]
        Mid2 --> W3[Worker]
    end
Patternreal StructureBest Suited For
SequentialAgent A → Agent B → Agent C, each building on the previousTasks with a fixed, linear pipeline (research → draft → review)
ParallelMultiple agents work simultaneously, results merged afterindependent subtasks with no dependency on each other
SupervisorOne agent delegates to and coordinates several specialistsA central agent needs to orchestrate diverse subtasks
HierarchicalSupervisors of supervisors, layered delegationlarge, complex tasks needing multiple levels of coordination
Peer-to-PeerAgents communicate directly with each other, no central coordinatordecentralized collaboration, no single obvious “owner”
HandoffOne agent explicitly passes a task to another when it recognizes a different specialist is neededDynamic routing based on what’s discovered mid-task
DebateMultiple agents argue different positions before convergingTasks benefiting from adversarial critique or diverse perspectives
CollaborativeAgents work on shared, overlapping context togethertightly-coupled subtasks that benefit from continuous coordination

5. Shared State vs. Independent State

SHARED STATE:      all agents in the system READ and WRITE
                  the SAME state -- ensures consistency, but can
                  create coordination complexity (who
                  updates what, and when?)

INDEPENDENT STATE:      each agent maintains its OWN state (Module
                       12), communicating only through EXPLICIT
                       messages or handoffs -- cleaner
                       separation, but requires deliberate
                       communication to stay coordinated

6. A Real Developer Example

TechCorp builds a multi-agent system for handling complex engineering tickets, using the Supervisor pattern:

flowchart TD
    S[Supervisor Agent] --> R[Researcher Agent<br/>searches docs & code]
    S --> C[Coder Agent<br/>drafts a fix]
    S --> Rev[Reviewer Agent<br/>checks the fix]
Agentreal SpecializationUses
SupervisorCoordinates the overall task, decides which specialist to invoke nextModule 8’s planning, applied at the multi-agent level
ResearcherSearches documentation and code for relevant contextRAG (Module 14), specialized tools
CoderDrafts a code fix based on the researcher’s findingsTools (Module 6), possibly reflection (Module 10)
ReviewerCritiques the coder’s fix before it’s finalizedReflection (Module 10), at the multi-agent level

7. Advantages, Disadvantages, and Trade-offs

ADVANTAGES:      real specialization can IMPROVE quality per
                subtask; PARALLEL patterns can reduce total
                time for independent subtasks; each agent's context
                stays FOCUSED (Section 2's concern, directly
                addressed)

DISADVANTAGES:      MORE complexity to coordinate;
                   communication OVERHEAD between agents; harder to DEBUG (which agent's reasoning caused a
                   problem?); MORE total LLM calls, meaning MORE cost
                   and latency

Incorrect idea: A important, honest point: more agents does NOT automatically mean better performance. A task well-suited to a single, well-designed agent (Module 13) can perform WORSE when artificially split across multiple agents, purely due to coordination overhead — Module 26 revisits this misconception directly.

Why it is incorrect:

8. A Simple Agentic AI Connection

This entire module is the agentic AI connection at its most direct — multi-agent systems are simply Module 4’s agent loop, Module 6’s tools, and Module 12’s state, each instantiated multiple times and coordinated together, using the architecture patterns this module introduces.

9. How Is This Used in AI?

🤖 How Is This Used in AI?

Production multi-agent systems are used specifically for > complex, multi-faceted tasks where specialization measurably improves outcomes — complex software engineering tasks, research synthesis across many sources, and workflows requiring diverse expertise are common real applications, always weighed against the real coordination overhead Section 7 describes.

10. Real-World Applications

  • Complex software engineering tasks (research, code, review, as a coordinated pipeline)
  • Multi-step research synthesis across diverse source types
  • Customer support systems routing between specialized agents (billing, technical, escalations)

11. Common Mistakes

Incorrect idea: Assuming more agents automatically means better performance.

Why it is incorrect: As shown directly in Section 7, coordination overhead can make a multi-agent system perform worse than a well-designed single agent for the same task.

Incorrect idea: Choosing a multi-agent pattern without a real reason.

Why it is incorrect: As shown directly in Section 4, each pattern fits a specific, real structural need — not a default upgrade from single-agent.

Incorrect idea: Using shared state without real coordination discipline.

Why it is incorrect: As shown directly in Section 5, this risks real consistency problems when multiple agents write to the same state without careful coordination.

12. Limitations

  • Multi-agent systems multiply the failure modes from Module 18 — now distributed across multiple agents, harder to diagnose
  • Debugging a multi-agent system requires more sophisticated observability (Module 21) than a single agent — you need to trace which specific agent’s reasoning led to a problem

13. Quick Reference

flowchart TD
    Q{Task Structure?}
    Q -->|Linear pipeline| Seq[Sequential]
    Q -->|Independent subtasks| Par[Parallel]
    Q -->|Central coordination needed| Sup[Supervisor]
    Q -->|Very large, layered task| Hier[Hierarchical]
    Q -->|No clear central owner| P2P[Peer-to-Peer]
    Q -->|Dynamic routing mid-task| Hand[Handoff]
    Q -->|Benefits from diverse critique| Deb[Debate]

14. Code — Implementing the Supervisor Pattern

🎯 Target of this example: implement Section 6’s real developer example directly — a supervisor delegating a task to specialized agents, each contributing its own focused piece, exactly Section 3’s specialization principle made into working code.

Example 1 — Simple

from dataclasses import dataclass

@dataclass
class AgentResult:
    agent_name: str
    output: str

class SpecialistAgent:
    """A NARROW, focused agent -- exactly Section 3's
    specialization principle."""
    def __init__(self, name: str, task_fn):
        self.name = name
        self.task_fn = task_fn

    def run(self, input_data: str) -> AgentResult:
        return AgentResult(self.name, self.task_fn(input_data))

class SupervisorAgent:
    """Directly implements Section 4's SUPERVISOR pattern -- delegates
    a task to specialist agents, then combines their results."""
    def __init__(self, specialists: dict):
        self.specialists = specialists

    def run(self, task: str) -> dict:
        results = {}
        for name, agent in self.specialists.items():
            result = agent.run(task)
            results[name] = result.output
        return results

researcher = SpecialistAgent("researcher", lambda t: f"Research findings for: {t}")
coder = SpecialistAgent("coder", lambda t: f"Code implementation for: {t}")
reviewer = SpecialistAgent("reviewer", lambda t: f"Review notes for: {t}")

supervisor = SupervisorAgent({"researcher": researcher, "coder": coder, "reviewer": reviewer})
results = supervisor.run("build a rate limiter")

for name, output in results.items():
    print(f"{name}: {output}")

Expected Output:

researcher: Research findings for: build a rate limiter
coder: Code implementation for: build a rate limiter
reviewer: Review notes for: build a rate limiter

What we conclude from this example: each specialist contributes its own focused output for the same overall task — exactly Section 6’s real developer example, with the supervisor correctly coordinating all three without any single agent needing to handle the entire, broad task alone.

Example 2 — Intermediate

from dataclasses import dataclass

@dataclass
class AgentResult:
    agent_name: str
    output: str

class SpecialistAgent:
    def __init__(self, name: str, task_fn):
        self.name = name
        self.task_fn = task_fn

    def run(self, input_data: str) -> AgentResult:
        return AgentResult(self.name, self.task_fn(input_data))

def sequential_pipeline(agents: list, initial_input: str) -> list:
    """Directly implements Section 4's SEQUENTIAL pattern -- each
    agent's output becomes the NEXT agent's input, building on the previous step, unlike the independent
    Supervisor pattern from Example 1."""
    trace = []
    current_input = initial_input
    for agent in agents:
        result = agent.run(current_input)
        trace.append(result)
        current_input = result.output  # feeds INTO the next agent
    return trace

researcher = SpecialistAgent("researcher", lambda t: f"[Researched] {t}")
coder = SpecialistAgent("coder", lambda t: f"[Coded based on: {t}]")
reviewer = SpecialistAgent("reviewer", lambda t: f"[Reviewed: {t}]")

pipeline_trace = sequential_pipeline([researcher, coder, reviewer], "build a rate limiter")

for result in pipeline_trace:
    print(f"{result.agent_name}: {result.output}")

Expected Output:

researcher: [Researched] build a rate limiter
coder: [Coded based on: [Researched] build a rate limiter]
reviewer: [Reviewed: [Coded based on: [Researched] build a rate
limiter]]

What we conclude from this example: unlike Example 1’s Supervisor pattern (where every agent independently receives the SAME original task), the sequential pattern threads each agent’s output into the next agent’s input — the reviewer’s output visibly contains the coder’s, which contains the researcher’s, exactly Section 4’s “each building on the previous” description, made directly observable.

Example 3 — Production Grade

from dataclasses import dataclass, field
from enum import Enum

class DelegationDecision(Enum):
    RESEARCHER = "researcher"
    CODER = "coder"
    REVIEWER = "reviewer"
    COMPLETE = "complete"

@dataclass
class MultiAgentTrace:
    step: int
    delegated_to: str
    output: str

class IntelligentSupervisor:
    """A production-style supervisor implementing real, DYNAMIC
    delegation -- deciding WHICH specialist to invoke NEXT based on
    what's already been done, directly mirroring Module 4's agent
    loop applied at the multi-agent coordination level, rather than
    Example 1's fixed, always-call-everyone pattern."""

    def __init__(self, specialists: dict):
        self.specialists = specialists
        self.trace: list = []
        self.completed_roles = set()

    def _decide_next(self) -> DelegationDecision:
        """The supervisor's OWN reasoning step -- deciding
        what's needed next, exactly Module 5's LLM-as-brain applied
        to multi-agent coordination."""
        if "researcher" not in self.completed_roles:
            return DelegationDecision.RESEARCHER
        if "coder" not in self.completed_roles:
            return DelegationDecision.CODER
        if "reviewer" not in self.completed_roles:
            return DelegationDecision.REVIEWER
        return DelegationDecision.COMPLETE

    def run(self, task: str, max_steps: int = 5) -> list:
        current_context = task
        for step in range(1, max_steps + 1):
            decision = self._decide_next()
            if decision == DelegationDecision.COMPLETE:
                break

            agent = self.specialists[decision.value]
            result = agent.run(current_context)
            self.completed_roles.add(decision.value)
            self.trace.append(MultiAgentTrace(step, decision.value, result.output))
            current_context = result.output

        return self.trace

@dataclass
class AgentResult:
    agent_name: str
    output: str

class SpecialistAgent:
    def __init__(self, name: str, task_fn):
        self.name = name
        self.task_fn = task_fn

    def run(self, input_data: str) -> AgentResult:
        return AgentResult(self.name, self.task_fn(input_data))

researcher = SpecialistAgent("researcher", lambda t: f"[Researched] {t}")
coder = SpecialistAgent("coder", lambda t: f"[Coded based on: {t}]")
reviewer = SpecialistAgent("reviewer", lambda t: f"[Reviewed: {t}]")

supervisor = IntelligentSupervisor({"researcher": researcher, "coder": coder, "reviewer": reviewer})
trace = supervisor.run("build a rate limiter")

for entry in trace:
    print(f"Step {entry.step}: delegated to '{entry.delegated_to}' -> {entry.output}")

Expected Output:

Step 1: delegated to 'researcher' -> [Researched] build a rate
limiter
Step 2: delegated to 'coder' -> [Coded based on: [Researched] build
a rate limiter]
Step 3: delegated to 'reviewer' -> [Reviewed: [Coded based on:
[Researched] build a rate limiter]]

What we conclude from this example: the supervisor reasons about which specialist to delegate to next at EACH step, tracking which roles are already complete — exactly Module 4’s agent loop, now coordinating multiple specialist agents rather than selecting between individual tool calls, directly demonstrating that multi-agent coordination is built from the same core concepts this entire course has covered, applied at a higher level of organization.

15. Interview Questions

Q: Why might a task benefit from multiple specialized agents rather than one broad, general-purpose agent?

Ans: A single agent handling a broad task can face context overload trying to hold everything relevant at once, and an unwieldy toolset if it needs to select correctly among many unrelated tools. Specialized agents each maintain a narrower, more focused context and tool set suited to their specific subtask, which can improve reasoning quality and reliability for that subtask compared to one agent trying to handle everything.

Q: Distinguish the sequential and supervisor multi-agent patterns.

Ans: In a sequential pattern, agents run in a fixed, linear order, with each agent’s output becoming the next agent’s input — the task flows through a pipeline. In a supervisor pattern, one central agent coordinates and delegates to specialist agents, deciding which specialist to invoke and potentially in what order, based on its own reasoning about what the task needs at each point — the supervisor’s decisions can be dynamic rather than following a fixed, predetermined sequence.

Q: Why is “more agents means better performance” considered a real misconception?

Ans: Multi-agent systems introduce real coordination overhead — communication between agents, more total LLM calls, and harder debugging when something goes wrong, since you need to determine which specific agent’s reasoning caused a problem. A task that’s well-suited to a single, well-designed agent can actually perform worse when artificially split across multiple agents, purely due to this coordination overhead, without any real benefit from specialization to offset the added complexity.

Q: Design a multi-agent architecture for a complex task in your own domain, and justify your choice of coordination pattern.

Ans: For a research report generation task involving gathering information from multiple sources, synthesizing findings, and producing a polished final document, I’d use a supervisor pattern — a coordinating agent that delegates to a researcher agent (gathering information), a writer agent (drafting the synthesis), and an editor agent (reviewing and polishing). A supervisor pattern fits well here because the overall task needs central coordination — the supervisor can dynamically decide when enough research has been gathered before moving to drafting, and when the draft needs another editing pass, rather than following a rigid, always-fixed sequence.

16. What You Should Remember

  • Multi-agent systems address real limitations of a single agent handling an overly broad task — context overload and unwieldy tool sets — through real specialization.
  • Sequential, parallel, supervisor, hierarchical, and other patterns each fit specific, real structural needs — verified directly through working implementations of both the fixed-delegation Supervisor pattern and the output-chaining Sequential pattern.
  • More agents does not automatically mean better performance — coordination overhead is real and real; multi-agent systems are justified by real task complexity, not used by default.

17. Quick Practice

For a complex task in your own domain, sketch out which multi-agent pattern (from Section 4’s table) would fit best, and identify what specific specialist agents you’d include and why each one’s narrower focus is justified.

18. Next Step

Next: Module 16 — Human-in-the-Loop — Level 7 begins here: why humans may need to remain in control for high-risk actions, and how approval, confirmation, and escalation fit into an agent’s loop.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed