Middleware: Customizing the Agent Loop

Recall the token-cost concern raised at the end of the last module — a long conversation’s history keeps growing, and resending. Recall also the Module 2 architecture map, where “Middleware” sat as its own box, promised for later. This module is where both of those threads finally get resolved: middleware, the current, primary way to customize an agent’s behavior without touching its core loop.

The problem middleware solves

Recall Module 14’s loop: model call, check for tools, execute, repeat. Suppose you want every single tool call logged, or every request checked for sensitive personal information, or the model swapped for a cheaper one on simple questions. You could rebuild the entire loop yourself to add this — but that means abandoning create_agent’s real, tested loop from Module 17, including its automatic error handling. Middleware exists specifically so you don’t have to make that trade-off.

The core idea: hooks around the loop, not inside it

flowchart TD
    A[before_agent: runs once, before anything starts] --> B[before_model: runs before every model call]
    B --> C[Model runs]
    C --> D[after_model: runs after every model call]
    D --> E{Tool requested?}
    E -->|Yes| F[Tool runs] --> B
    E -->|No| G[Loop ends]

Middleware lets you attach your own function to specific points in this loop — before the agent starts, before or after each model call — without modifying the loop’s own internal code at all.

Example 1: a logging middleware

from langchain.chat_models import init_chat_model
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware

class LoggingMiddleware(AgentMiddleware):
    def before_model(self, state):
        print(f"→ About to call the model. Messages so far: {len(state['messages'])}")

agent = create_agent(
    model=init_chat_model("openai:gpt-4o-mini"),
    tools=[],
    middleware=[LoggingMiddleware()],
)

agent.invoke({"messages": [{"role": "user", "content": "Say hello."}]})

before_model runs automatically, every single time, right before the agent’s loop calls the model — exactly the visibility astream_events gave you in Module 17, now built directly into the agent’s own configuration rather than something you observe from outside.

Example 2: automatically summarizing long conversation history

Recall the real cost concern from Module 19 — a growing conversation resends its entire history on every call. LangChain ships a built-in middleware specifically for this.

from langchain.chat_models import init_chat_model
from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model=init_chat_model("openai:gpt-4o-mini"),
    tools=[],
    middleware=[
        SummarizationMiddleware(
            model=init_chat_model("openai:gpt-4o-mini"),
            max_tokens_before_summary=2000,
        )
    ],
)

Once a conversation’s history grows past max_tokens_before_summary, this middleware automatically condenses older messages into a shorter summary before the next model call, keeping the conversation’s real token cost — and Module 19’s own concern — under control, without you writing any summarization logic yourself.

Example 3: redacting sensitive information automatically

from langchain.chat_models import init_chat_model
from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware

agent = create_agent(
    model=init_chat_model("openai:gpt-4o-mini"),
    tools=[],
    middleware=[PIIMiddleware("email"), PIIMiddleware("credit_card")],
)

result = agent.invoke({"messages": [{"role": "user", "content": "My email is jane@example.com, please note that down."}]})
print(result["messages"][-1].content)

PIIMiddleware automatically detects and redacts specific categories of sensitive personal information before it ever reaches the model — a real, practical safety measure genuinely relevant to any agent handling real user input.

Example 4: dynamic model selection

Recall Module 4’s discussion of using a cheaper model for simple tasks. Middleware is the real mechanism for doing this automatically, based on the actual request.

from langchain.chat_models import init_chat_model
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware

FAST_MODEL = init_chat_model("openai:gpt-4o-mini")
POWERFUL_MODEL = init_chat_model("openai:gpt-4o")

class ModelRoutingMiddleware(AgentMiddleware):
    def wrap_model_call(self, request, handler):
        last_message = request.state["messages"][-1].content
        if len(last_message) > 200:
            request = request.override(model=POWERFUL_MODEL)
        else:
            request = request.override(model=FAST_MODEL)
        return handler(request)

agent = create_agent(model=FAST_MODEL, tools=[], middleware=[ModelRoutingMiddleware()])

wrap_model_call lets you inspect the actual request and swap the model being used for that specific call — a long, complex question genuinely routes to a more capable (and more expensive) model, while a short one stays on the cheaper default, exactly the deliberate, cost-aware routing decision worth making in a real, production application.

Example 5: dynamic tool filtering by role

from langchain.chat_models import init_chat_model
from langchain.tools import tool
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware

@tool
def read_report(report_id: str) -> str:
    """Read a business report by ID."""
    return f"Report {report_id}: Q3 revenue up 12%."

@tool
def delete_report(report_id: str) -> str:
    """Permanently delete a business report by ID."""
    return f"Report {report_id} deleted."

class RoleBasedToolMiddleware(AgentMiddleware):
    def __init__(self, user_role: str):
        self.user_role = user_role

    def before_model(self, state):
        if self.user_role != "admin":
            state["tools"] = [t for t in state.get("tools", []) if t.name != "delete_report"]
        return state

agent = create_agent(
    model=init_chat_model("openai:gpt-4o-mini"),
    tools=[read_report, delete_report],
    middleware=[RoleBasedToolMiddleware(user_role="viewer")],
)

A “viewer” role genuinely never has delete_report available to be called at all, filtered out before the model ever sees it — a real, meaningful security boundary, not just a polite suggestion the model could still choose to ignore.

Common mistakes worth avoiding

Rebuilding a custom loop instead of reaching for middleware first. Recall this module’s opening point — middleware exists specifically so you keep create_agent’s tested loop and error handling from Module 17, rather than losing them by hand-rolling your own custom control flow.

Stacking too many middleware classes without understanding their order. Middleware runs in the order you list it, and later middleware sees the effects of earlier ones. A logging middleware placed after a PII-redaction middleware will log the already-redacted version; placed before, it logs the raw, unredacted input — a genuinely meaningful difference worth being deliberate about.

Using role-based tool filtering as your only access control. Recall Example 5 — this is a real, useful safeguard at the agent layer, but it shouldn’t be your only protection. A genuinely secure system still validates permissions at the actual data or API layer too, the same defense-in-depth principle worth applying anywhere real access control matters.

What you should take away from this module

• Middleware lets you hook into an agent’s loop — before it starts, before or after each model call — without rewriting the loop itself.
• SummarizationMiddleware and PIIMiddleware are real, built-in solutions to genuinely common problems: growing token cost and sensitive data exposure.
• Dynamic model selection and dynamic tool filtering, both implemented via wrap_model_call and before_model, let an agent’s behavior adapt to the actual request or the actual user, rather than staying fixed.
• Middleware order matters — earlier middleware’s effects are visible to everything that runs after it.

Where this goes next

The next module shifts away from agents specifically and back toward a concept you already understand conceptually from your earlier RAG course: Retrieval. You’ll see exactly how LangChain implements the loaders-splitters-embeddings-vectorstore-retriever pipeline you already know the theory behind.