Section 3 — ChatClient

Begin with the problem

A friendly front door for model calls

ChatModel is the low-level engine connection. ChatClient is the fluent front desk that prepares requests, runs advisors, calls the model, and converts the result.

prompt() → advisors → call()/stream() → content/entity

What you will learn

• Create and use a ChatClient.
• Follow its fluent request chain.
• Explain what advisors add around a call.
• Choose blocking, streaming, or typed output.

Current official reference: Spring AI documentation for this topic. The examples below primarily preserve the stated 1.1.x course target. Where Spring AI 2.0 differs, the text must treat that behavior as version-specific rather than universal.

(Continues from Section 2. Target: Spring AI 1.1.x / Spring Boot 3.5.x.)

3.1 Why ChatClient Exists

Current 2.0 behavior to know first

In Spring AI 2.0, ChatClient automatically registers a ToolCallingAdvisor unless that behavior is disabled. The advisor owns the repeated model → tool → result → model loop. Older 1.1 examples may describe tool execution as happening inside a provider’s ChatModel; do not transfer that internal explanation to 2.0 unchanged.

The practical lesson is simple: always check the version beside an advisor-chain or tool-loop example. The public goal is similar, but the component responsible for the loop and the advisor ordering rules changed.

ChatModel alone gives you call(Prompt) -> ChatResponse. That’s a raw client. Nobody wants to hand-assemble Prompt objects, manage memory injection, wire tool callbacks, and manually loop tool-execution rounds in every service method. ChatClient is the fluent orchestration façade that does this — the same relationship RestClient has to raw HttpClient, or JdbcTemplate has to raw java.sql.Connection.

Real-world analogy — Restaurant Front-of-House vs. Kitchen: ChatModel is the kitchen: it takes a fully-specified order and produces food. ChatClient is the waiter: takes your loosely-specified request (“I’ll have the steak, medium rare, no onions”), fills in defaults (house wine pairing, standard sides), routes special instructions to the kitchen in the right format, and hands back the plated result — potentially going back to the kitchen mid-service if you change your mind (the tool-calling loop).

Analogy: The Restaurant Waiter / Pipeline Supervisor Think of the relationship between ChatModel (the raw model connection) and ChatClient (the orchestration wrapper) as a dining room service:

• The Raw Chef (ChatModel): Lives in the kitchen. They take a highly structured list of ingredients (a raw Prompt object) and return a plated meal (ChatResponse). They don’t know who the guest is, they don’t look up user profiles, and they don’t check for dietary safety policies.
• The Waiter (ChatClient): The waiter handles the guest at the table:
  • Takes your casual request: “Give me the daily special.”
  • Before the kitchen: Automatically appends the default system layout (“Serve with a side of vegetables”), checks your order memory log to remember your food allergies (Memory Advisor), and runs a safety check to block toxic requests (Safeguard Advisor).
  • During service (Tool Calls): If the kitchen chef asks a question (“Is the guest allergic to mushrooms?”), the waiter looks up the booking details and answers the kitchen directly (the tool-calling loop) without forcing you to walk into the hot kitchen.

📊 Visual Flowchart: The ChatClient Advisor & Interceptor Pipeline

Here is how request parameters flow through the before-hooks, call the model, and return through the after-hooks:

graph TD
    UserCall["1. User Request:<br>chatClient.prompt().call()"] --> RequestSpec["2. Compile Request Spec"]

    subgraph AdvisorPipeline ["Advisor Interceptor Pipeline"]
        RequestSpec --> Adv1Before["3. Advisor 1: before()<br>(e.g. MessageChatMemoryAdvisor ID lookup)"]
        Adv1Before --> Adv2Before["4. Advisor 2: before()<br>(e.g. SafeGuardAdvisor term filter)"]

        Adv2Before --> ModelCall["5. ChatModel: call(Prompt)"]

        ModelCall --> Adv2After["6. Advisor 2: after()<br>(Filter outgoing text content)"]
        Adv2After --> Adv1After["7. Advisor 1: after()<br>(Save prompt + response to repository)"]
    end

    Adv1After --> OutputContent["8. Plated Output:<br>Response content / entity"]

---

3.2 The Builder — What It Actually Captures

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultSystem("You are a precise, concise enterprise assistant.")
        .defaultAdvisors(
                new MessageChatMemoryAdvisor(chatMemory),
                new SimpleLoggerAdvisor(),
                new SafeGuardAdvisor(List.of("competitor-x", "internal-only"))
        )
        .defaultOptions(OpenAiChatOptions.builder()
                .model("gpt-4o")
                .temperature(0.3)
                .build())
        .defaultTools(weatherTool, orderLookupTool)
        .build();

ChatClient.Builder is a mutable accumulator for defaults that get baked into every request built from the resulting ChatClient, unless overridden per-call. Internally it holds:

DefaultChatClientBuilder
 ├── ChatModel chatModel
 ├── String defaultSystemText
 ├── Map<String,Object> defaultSystemParams
 ├── List<Advisor> defaultAdvisors
 ├── ChatOptions defaultOptions
 ├── List<ToolCallback> defaultToolCallbacks
 ├── Map<String,Object> defaultToolContext
 └── ObservationRegistry observationRegistry

.build() produces a DefaultChatClient wrapping an immutable snapshot of these defaults. Bean scope matters here: the auto-configured ChatClient.Builder bean is prototype-scoped specifically so that different services in your application can each inject their own Builder, customize it (different system prompt, different advisors) via @Bean methods, and .build() their own independent ChatClient — without one service’s customization leaking into another’s. Injecting the same ChatClient.Builder singleton and calling .build() twice with different .defaultSystem() calls in between is a classic bug: builders are stateful, and shared mutation across threads is not safe.

@Configuration
public class ChatClientConfig {

    @Bean
    public ChatClient supportChatClient(ChatClient.Builder builder, ChatMemory chatMemory) {
        return builder
                .defaultSystem(supportSystemPrompt)
                .defaultAdvisors(new MessageChatMemoryAdvisor(chatMemory))
                .build();
    }

    @Bean
    public ChatClient summarizationChatClient(ChatClient.Builder builder) {
        return builder
                .defaultSystem("Summarize input in 3 bullet points. No preamble.")
                .defaultOptions(OpenAiChatOptions.builder().temperature(0.0).build())
                .build();
        // note: a fresh Builder injection per @Bean method — prototype scope
        // guarantees this doesn't collide with supportChatClient's customization
    }
}

---

3.3 Per-Request Overrides — The Fluent Chain

String answer = chatClient.prompt()
        .system(s -> s.text("Override: respond only in French today"))
        .user(u -> u.text("What's our return policy for {product}?")
                    .param("product", "wireless earbuds"))
        .advisors(a -> a.param(CHAT_MEMORY_CONVERSATION_ID_KEY, sessionId))
        .options(OpenAiChatOptions.builder().temperature(0.1).build())
        .tools(refundLookupTool)
        .call()
        .content();

Every one of these — .system(), .user(), .advisors(), .options(), .tools() — layers on top of the Builder’s defaults, not replacing them wholesale (advisors and tools are additive lists; options and system follow last-write-wins per field where the provider’s options object supports partial merging). This request-scoped configuration is what makes one ChatClient bean safely reusable across many different calls with different needs — the mutation lives in the short-lived ChatClientRequestSpec, not the shared ChatClient.

---

3.4 Call vs. Stream vs. Entity — The Response Spec Family

ChatClient.prompt()....
        │
        ├── .call()                       → CallResponseSpec
        │       ├── .content()            → String
        │       ├── .chatResponse()       → ChatResponse (full metadata: usage, finishReason)
        │       ├── .entity(Class<T>)     → T (structured output, see Section 11)
        │       └── .responseEntity(...)  → ResponseEntity-style wrapper with raw + parsed
        │
        └── .stream()                     → StreamResponseSpec
                ├── .content()            → Flux<String> (token-by-token text)
                ├── .chatResponse()       → Flux<ChatResponse>
                └── .chatClientResponse() → Flux<ChatClientResponse> (advisor context included)

Blocking (.call()) internally still may use a reactive ChatModel implementation under the hood (most provider modules implement StreamingChatModel and expose blocking via .block() on the reactive path, or a really separate synchronous HTTP call depending on provider — OpenAI’s non-streaming endpoint is a distinct wire call, not a blocked stream). The important operational point: .call() still runs on the calling thread and will block a servlet container thread for the full model latency unless you’re on WebFlux — this is exactly why Section 12 — Streaming and this section’s reactive coverage matter for throughput under load.

Reactive (.stream()) returns Flux<...> and is the correct choice inside a WebFlux controller or anywhere you need backpressure-aware token delivery (SSE endpoints, chat UIs — see the glossary if “streaming” as a concept is new). It does not magically make a blocking-only provider implementation non-blocking — if the underlying ChatModel only implements the synchronous interface, .stream() support depends on that provider module offering a real streaming implementation (all major providers do: OpenAI, Anthropic, Ollama, and Bedrock Converse all implement true SSE/chunked streaming under StreamingChatModel).

---

3.5 Advisors — The Interceptor Pipeline, Precisely

                 ┌──────────────────────────────────────────┐
                 │            AroundAdvisorChain              │
                 │                                            │
 Request ───────▶│ Advisor1.before()                          │
                 │      Advisor2.before()                     │
                 │           Advisor3.before()                │
                 │                │                            │
                 │                ▼                            │
                 │         [ChatModel.call()]                  │
                 │                │                            │
                 │                ▼                            │
                 │           Advisor3.after()                  │
                 │      Advisor2.after()                       │
                 │ Advisor1.after()                             │
                 └──────────────────────────────────────────┘◀── Response

Advisors are ordered via Ordered/.order() (lower value = runs first, i.e., outermost in the wrap — closest to the caller on the way in, last to touch the response on the way out — identical semantics to Spring MVC HandlerInterceptor and Servlet Filter chains, deliberately). Two advisor categories:

Type	Interface	Can do
Call advisors	CallAroundAdvisor	Wrap blocking .call() requests
Stream advisors	StreamAroundAdvisor	Wrap .stream() requests, operate on Flux

Built-in advisors you’ll actually use in production:

Advisor	Purpose
MessageChatMemoryAdvisor	Injects prior conversation turns from ChatMemory before the call, persists the new turn after
PromptChatMemoryAdvisor	Deprecated as of 1.1.6 — migrate to MessageChatMemoryAdvisor with an explicit conversationId
SimpleLoggerAdvisor	Logs request/response at DEBUG — your first debugging tool, see §3.9
SafeGuardAdvisor	Blocks requests/responses containing configured sensitive terms
QuestionAnswerAdvisor	Injects VectorStore retrieval results into context (basic RAG — superseded by RetrievalAugmentationAdvisor for production pipelines, Section 7)
RetrievalAugmentationAdvisor	Full RAG pipeline advisor: query transformation, retrieval, augmentation

3.5.1 Writing a Custom Advisor

public class TenantContextAdvisor implements CallAroundAdvisor {

    @Override
    public String getName() {
        return "TenantContextAdvisor";
    }

    @Override
    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE + 10; // run early, close to the call boundary
    }

    @Override
    public ChatClientResponse adviseCall(ChatClientRequest request, CallAroundAdvisorChain chain) {
        String tenantId = TenantContext.getCurrentTenantId();

        // mutate the request BEFORE it proceeds down the chain
        ChatClientRequest enrichedRequest = request.mutate()
                .context(ctx -> ctx.put("tenantId", tenantId))
                .build();

        long start = System.nanoTime();
        ChatClientResponse response = chain.nextCall(enrichedRequest);
        long elapsedMs = (System.nanoTime() - start) / 1_000_000;

        log.info("Tenant {} chat call completed in {}ms", tenantId, elapsedMs);
        return response;
    }
}

The critical pattern: you call chain.nextCall(request) exactly once to proceed to the next advisor (or the model itself if you’re the innermost advisor) — this is what makes it an “around” advisor, structurally identical to MethodInterceptor.invoke(MethodInvocation) in Spring AOP. Forgetting to call chain.nextCall() silently short-circuits the entire pipeline — the model is never invoked. This is a real production bug pattern: an advisor with an early-return guard clause that forgets the chain call on the guarded path.

---

3.6 Memory Integration — How It Actually Wires In

ChatMemory chatMemory = MessageWindowChatMemory.builder()
        .chatMemoryRepository(new JdbcChatMemoryRepository(jdbcTemplate))
        .maxMessages(20)
        .build();

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
        .build();

// per-request, conversationId MUST be supplied (as of 1.1.6+, since
// PromptChatMemoryAdvisor's implicit-conversation-ID behavior was deprecated)
chatClient.prompt()
        .user("What did I ask you before?")
        .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, sessionId))
        .call()
        .content();

MessageChatMemoryAdvisor.adviseCall() does exactly two things around the chain call:

1. Before: chatMemory.get(conversationId) → prepend retrieved List<Message> to the current request’s messages.
2. After: chatMemory.add(conversationId, newUserMessage + newAssistantMessage) → persist the just-completed turn.

MessageWindowChatMemory caps stored messages at maxMessages using a sliding window (oldest messages evicted first) — it does not summarize by default; summarization- based memory strategies are a Section 8 topic (ChatMemory composed with a summarization advisor), since Spring AI’s built-in memory is window-based, not automatically compressive.

---

3.7 Retry and Timeout

Retry is not configured on ChatClient directly — it’s configured on the underlying ChatModel via a RetryTemplate bean, which the auto-configuration wires in by default with sane exponential backoff for transient errors (5xx, rate limits):

spring:
  ai:
    retry:
      max-attempts: 3
      backoff:
        initial-interval: 2000
        multiplier: 2.0
        max-interval: 30000
    openai:
      chat:
        options:
          model: gpt-4o

For custom retry policy (e.g., don’t retry on 400s, do retry on 429s with Retry-After header respect):

@Bean
public RetryTemplate customRetryTemplate() {
    return RetryTemplate.builder()
            .maxAttempts(4)
            .exponentialBackoff(Duration.ofSeconds(1), 2.0, Duration.ofSeconds(30))
            .retryOn(TransientAiException.class)
            .build();
}

Timeout is set at the HTTP client level, not ChatClient — for RestClient-backed providers, via a custom ClientHttpRequestFactory:

@Bean
public RestClient.Builder restClientBuilder() {
    ClientHttpRequestFactorySettings settings = ClientHttpRequestFactorySettings.DEFAULTS
            .withConnectTimeout(Duration.ofSeconds(5))
            .withReadTimeout(Duration.ofSeconds(60)); // model latency can legitimately be long
    return RestClient.builder()
            .requestFactory(ClientHttpRequestFactories.get(settings));
}

This bean is picked up automatically by OpenAiChatAutoConfiguration via the ObjectProvider<RestClient.Builder> injection point shown in Section 1 — you don’t wire it into ChatClient directly; it flows through the auto-configured ChatModel.

---

3.8 Full Execution Flow (Consolidated)

chatClient.prompt()
    .system(...).user(...).advisors(...).tools(...).options(...)
    .call() or .stream()
        │
        ▼
1. ChatClientRequestSpec accumulates builder defaults + per-request overrides
        │
        ▼
2. .call()/.stream() triggers DefaultChatClient.execute()
        │
        ▼
3. ChatClientRequest constructed: Prompt + AdvisorContext Map
        │
        ▼
4. AroundAdvisorChain built from (defaultAdvisors + per-request advisors),
   sorted by Ordered value
        │
        ▼
5. Chain executes: each advisor's before-logic runs, chain.nextCall()
   propagates inward, innermost link invokes ChatModel.call(Prompt)
   or StreamingChatModel.stream(Prompt)
        │
        ▼
6. If tool calls present in response AND internalToolExecutionEnabled
   (default true): ToolCallingManager executes, loop repeats step 5's
   innermost call with tool results appended — entirely inside this step,
   invisible to advisors above unless they specifically inspect it
        │
        ▼
7. Chain unwinds: each advisor's after-logic runs in reverse order
   (memory persistence, logging, safety filtering on the response)
        │
        ▼
8. ChatClientResponse returned to CallResponseSpec/StreamResponseSpec
        │
        ▼
9. .content()/.entity()/.chatResponse() extracts the requested shape

---

3.9 Debugging

SimpleLoggerAdvisor is your single best debugging tool — add it as the innermost advisor (highest order value, so it’s closest to the model call) to see exactly what’s sent and received after all other advisors have mutated the request:

.defaultAdvisors(
        new TenantContextAdvisor(),      // order: early
        new MessageChatMemoryAdvisor(chatMemory),
        new SimpleLoggerAdvisor()        // order: LOWEST_PRECEDENCE, logs the final request/response
)

logging:
  level:
    org.springframework.ai.chat.client.advisor: DEBUG

For advisor-ordering bugs specifically: log advisor.getOrder() for every advisor at startup and assert the resulting sorted order matches your mental model — silent misordering (e.g., a safety filter running before memory injection instead of after, missing injected context in its scan) is the most common multi-advisor production bug.

---

3.10 Common Mistakes

1. Sharing one ChatClient.Builder singleton across services and mutating it — use prototype scope (the default) and build independent ChatClients per concern.
2. Forgetting chain.nextCall()/chain.nextStream() in a custom advisor — silently short-circuits the pipeline.
3. Using .call() inside a WebFlux reactive chain — blocks an event-loop thread; always use .stream() or wrap in Mono.fromCallable(...).subscribeOn(Schedulers.boundedElastic()) if you truly need blocking semantics inside reactive code (and reconsider why).
4. Not setting ChatMemory.CONVERSATION_ID — post-1.1.6, omitting this doesn’t fall back gracefully to a sane default in all configurations; conversations can bleed across users if a fallback key collides.
5. Assuming advisor order is declaration order — it’s Ordered value order, not the order you called .defaultAdvisors().
6. No timeout configured — default RestClient timeouts can be effectively unbounded depending on Spring Boot version defaults; always set explicit connect/read timeouts for production.

---

3.11 Interview Questions

1. Why is the auto-configured ChatClient.Builder bean prototype-scoped rather than a singleton?
2. Walk through what happens if a custom CallAroundAdvisor never calls chain.nextCall().
3. What determines advisor execution order, and how does it relate to Spring AOP’s MethodInterceptor chain semantics?
4. Where is retry configured — on ChatClient or somewhere else? Justify the design choice.
5. What’s the practical difference between .call().content() and .stream().content() in terms of thread-blocking behavior under WebFlux?
6. How does MessageChatMemoryAdvisor inject and persist conversation history — name the exact two hook points in the advisor lifecycle.
7. Why was PromptChatMemoryAdvisor deprecated in favor of requiring an explicit conversationId?
8. What’s the risk of injecting the same ChatClient.Builder instance into two @Bean methods that each call .defaultSystem() differently?
9. At what point in the execution flow does the internal tool-calling loop happen, and which advisors can/cannot observe intermediate tool-call rounds?
10. How would you set a read timeout for model calls that legitimately take 45+ seconds, without setting it so high that a hung connection blocks a thread indefinitely?
11. What’s the difference between CallAroundAdvisor and StreamAroundAdvisor, and why can’t one advisor implementation trivially handle both without care?
12. Where would you place a safety/content-filtering advisor in the order chain relative to a memory advisor, and why?
13. Explain MessageWindowChatMemory’s eviction strategy — is it summarization-based or a sliding window by default?
14. What does ChatClientRequest.mutate() do, and why is direct field mutation on the original request object not supported?
15. How does .options() per-request interact with .defaultOptions() set on the Builder — full replacement or field-level merge?
16. What’s the actual class hierarchy behind CallResponseSpec.entity(Class<T>), and how does it relate to structured output (preview of Section 11)?
17. Why does SimpleLoggerAdvisor need to be placed as the innermost advisor to be maximally useful for debugging?
18. What HTTP-client-level configuration point does ChatClient retry/timeout ultimately bottom out at for an OpenAI-backed ChatModel?
19. Describe a production scenario where advisor misordering caused a real bug (a safety filter missing injected content, for example) and how you’d catch it in code review.
20. How would you unit test a custom Advisor in isolation without standing up a real ChatModel?

---

3.12 Best Practices Checklist

• Build distinct ChatClient beans per concern (support bot, summarizer, extractor) from prototype-scoped Builder injections — never share one mutated Builder.
• Always set explicit ChatMemory.CONVERSATION_ID per request; never rely on implicit/default conversation identity in multi-user systems.
• Place SimpleLoggerAdvisor (or your own structured-logging advisor) as the innermost advisor in every non-trivial pipeline.
• Set explicit connect/read timeouts on the underlying RestClient/WebClient — do not run with framework defaults in production.
• Use .stream() inside WebFlux controllers; reserve .call() for MVC/blocking contexts or background batch jobs.
• Log advisor.getOrder() values explicitly (don’t rely on default Ordered.LOWEST_PRECEDENCE) so pipeline order is self-documenting in code review.

---

3.13 Key Takeaways

• ChatClient is an orchestration façade over ChatModel; ChatClient.Builder accumulates reusable defaults, and request-scoped calls layer overrides on top.
• Advisors are a Spring-AOP-style “around” interceptor chain, ordered by Ordered value, not declaration order — forgetting chain.nextCall() silently breaks the pipeline.
• Memory, logging, safety filtering, and RAG retrieval are all just advisors — there’s no separate “memory subsystem” wired differently from any other cross-cutting concern.
• .call() blocks the calling thread regardless of context; .stream() is the reactive-safe path and the correct default for WebFlux/SSE use cases.
• Retry and timeout live at the ChatModel/HTTP-client level, not on ChatClient itself — know where to actually configure them.

---

End of Section 3. Next: Section 4 — ChatModel Providers (multi-provider integration: OpenAI, Anthropic, Gemini, Ollama, DeepSeek, Mistral, Groq, Together AI; switching providers, fallback, routing, load balancing).