Chat Models: The Foundation of Every LangChain App

You’ve used init_chat_model and .invoke() in every module so far, but always in passing — just enough to make a point about something else. This module is different. This is the one where we stop borrowing the chat model to illustrate other ideas, and actually understand it properly, on its own terms.

That’s worth doing carefully, because literally everything else in this course — tools, agents, retrieval, structured output — is built on top of this one component. If your understanding of the chat model itself is shaky, every later module inherits that shakiness. So let’s slow down here, more than usual, and build this foundation properly.

What a chat model actually is, precisely

You already know this conceptually from your earlier courses, so let’s just anchor the LangChain-specific vocabulary: a chat model in LangChain is an object that represents one specific AI model, from one specific provider, configured with a specific set of settings. Once created, that object has exactly one core job: you give it a list of messages, and it gives you back a reply.

flowchart LR
    A["List of messages\n(the conversation so far)"] --> B["Chat Model\n(configured for one provider, one model)"]
    B --> C["A single reply\n(an AIMessage)"]

Everything in this module is really just exploring that one diagram in depth: how you configure the box in the middle, and exactly what comes out the other side.

Example 1: the absolute simplest call, and what actually happens inside it

Let’s start even simpler than what you’ve seen before — passing a plain string, not a list of messages.

OpenAI:

from langchain.chat_models import init_chat_model

model = init_chat_model("openai:gpt-4o-mini")
response = model.invoke("What is the capital of France?")

print(response.content)

Gemini:

from langchain.chat_models import init_chat_model

model = init_chat_model("google_genai:gemini-2.0-flash")
response = model.invoke("What is the capital of France?")

print(response.content)

Notice we passed a plain string, "What is the capital of France?", not a HumanMessage. This works because LangChain quietly does something helpful for you here: whenever you .invoke() a chat model with a plain string, it automatically wraps that string in a HumanMessage before sending it. So this line:

model.invoke("What is the capital of France?")

is genuinely, exactly equivalent to this one:

from langchain.messages import HumanMessage
model.invoke([HumanMessage(content="What is the capital of France?")])

Both are completely valid. The plain-string version is a convenience for quick, single-turn calls; the message-list version is what you’ll reach for the moment you need a real conversation, a system prompt, or anything with more than one turn — exactly like you built back in Module 1.

Example 2: what’s actually inside the response object

Let’s look more closely at what comes back from .invoke(), because .content is only one small piece of it.

from langchain.chat_models import init_chat_model

model = init_chat_model("openai:gpt-4o-mini")
response = model.invoke("What is the capital of France?")

print("The actual text:", response.content)
print("What kind of object is this?", type(response))
print("Token usage:", response.usage_metadata)
print("Which exact model replied:", response.response_metadata.get("model_name"))

The object you get back is an AIMessage — the same message type you saw in Module 1, when we manually appended the model’s own reply back into our conversation history. That’s not a coincidence; it’s deliberate. Because the reply comes back as the same kind of object you’d add to a message list, you can immediately continue a conversation without converting anything yourself.

response.usage_metadata is genuinely worth knowing about early, since you already understand from your earlier courses why token counts matter for cost. It typically contains something like:

{'input_tokens': 14, 'output_tokens': 8, 'total_tokens': 22}

That’s a real, exact count of what this specific call cost you, available immediately, without any separate calculation.

Example 3: configuring the model with real parameters

So far we’ve only passed a model name string. Real applications almost always need to tune how the model behaves — and this is where you’ll recognize concepts from your earlier AI courses, now appearing as concrete, settable LangChain parameters.

from langchain.chat_models import init_chat_model

model = init_chat_model(
    "openai:gpt-4o-mini",
    temperature=0.2,      # lower = more focused, consistent answers
    max_tokens=150,        # a hard ceiling on how long the reply can be
    timeout=30,             # give up and raise an error after 30 seconds
    max_retries=2,          # automatically retry twice on a transient failure
)

response = model.invoke("Suggest one creative name for a coffee shop.")
print(response.content)

And the same parameters, applied to Gemini, using its own recommended defaults:

from langchain.chat_models import init_chat_model

model = init_chat_model(
    "google_genai:gemini-2.0-flash",
    temperature=0.2,
    max_tokens=150,
    timeout=30,
    max_retries=2,
)

response = model.invoke("Suggest one creative name for a coffee shop.")
print(response.content)

Let’s be precise about each one, since “just set it and see” isn’t the same as actually understanding it:

• temperature — you already know this concept: it controls how much randomness goes into picking the next word. A low value like 0.2 makes the model favor its single most likely answer almost every time; a higher value like 0.9 lets it wander into more varied, creative territory.
• max_tokens — a hard limit on how long the model’s reply is allowed to be. If the model would naturally want to say more, the response gets cut off once this limit is hit.
• timeout — how many seconds LangChain will wait for a reply before giving up and raising an error, rather than hanging forever if the provider’s servers are slow.
• max_retries — if a call fails for a temporary reason (a brief network hiccup, a momentary rate limit), LangChain will automatically try again, up to this many times, before finally giving up.

Notice something important in that second code block: every one of these parameter names stayed identical when we switched to Gemini. This is exactly the kind of consistency Module 1 promised — you’re not learning “OpenAI’s settings” and “Gemini’s settings” as two separate things. You’re learning LangChain’s settings, once.

Example 4: the same model, configured two different ways, for two different jobs

Real applications often need more than one configuration of the same model — a creative-writing task and a strict, factual lookup shouldn’t use the same temperature.

from langchain.chat_models import init_chat_model

precise_model = init_chat_model("openai:gpt-4o-mini", temperature=0.0)
creative_model = init_chat_model("openai:gpt-4o-mini", temperature=0.9)

fact_question = "What is 12 multiplied by 8?"
story_prompt = "Write one wildly imaginative opening sentence for a fantasy novel."

print("Precise:", precise_model.invoke(fact_question).content)
print("Creative:", creative_model.invoke(story_prompt).content)

Each init_chat_model(...) call creates a genuinely separate, independent object. Changing creative_model’s settings later has zero effect on precise_model — they don’t share any state at all. This is worth knowing early: it’s completely normal, and often the right design, to create several differently-configured instances of the same underlying model within one application.

Example 5: streaming — a first look

You’ll get a full module dedicated entirely to streaming soon, but it’s worth a first, small look here, because it changes how .invoke() behaves in a genuinely important way.

from langchain.chat_models import init_chat_model

model = init_chat_model("openai:gpt-4o-mini")

for chunk in model.stream("Write a two-sentence bedtime story about a sleepy robot."):
    print(chunk.content, end="", flush=True)

Notice we called .stream() here, not .invoke(). The difference matters: .invoke() waits until the entire reply is ready, then hands it to you all at once. .stream() hands you the reply in small pieces, as they’re generated, letting you print (or display) each piece the moment it arrives — which is exactly why chat apps like ChatGPT appear to “type” their answer instead of making you wait in silence. We’ll go much deeper into this soon; for now, just notice that this same method, .stream(), exists identically across providers.

Example 6: calling the model asynchronously

Similarly, you’ll get a full module on async later — but a first taste belongs here too, since it’s simply the async twin of everything you’ve already seen.

import asyncio
from langchain.chat_models import init_chat_model

model = init_chat_model("openai:gpt-4o-mini")

async def main():
    response = await model.ainvoke("What's a fun fact about octopuses?")
    print(response.content)

asyncio.run(main())

The only real difference here is ainvoke instead of invoke, and await in front of it. Conceptually, async means “don’t sit here blocking everything else while you wait for the reply — let other work happen at the same time, and come back to this the moment it’s ready.” We’ll unpack exactly why that matters for real applications in its own dedicated module.

Example 7: a first glimpse of structured output

You’ve seen the model reply with plain text so far. Real applications frequently need something more precise — a reply shaped exactly like data your code can use directly, not a sentence you’d have to parse yourself.

from pydantic import BaseModel
from langchain.chat_models import init_chat_model

class Capital(BaseModel):
    country: str
    capital_city: str

model = init_chat_model("openai:gpt-4o-mini")
structured_model = model.with_structured_output(Capital)

result = structured_model.invoke("What is the capital of France?")
print(result)
print(type(result))

Run this, and instead of a sentence like “The capital of France is Paris,” you get back a genuine, typed Python object: Capital(country='France', capital_city='Paris'). That’s a real, usable piece of data, not text you’d have to guess-parse yourself. This is genuinely one of the most useful things you’ll learn in this entire course, and it earns a full module of its own soon. For now, just notice the shape: .with_structured_output(...) wraps your existing model and changes what kind of thing .invoke() hands back.

Example 8: a first glimpse of giving the model tools

One last preview, since it’s the natural next step after structured output, and you’ll build on this constantly starting in a few modules.

from langchain.chat_models import init_chat_model
from langchain.tools import tool

@tool
def get_temperature(city: str) -> str:
    """Return the current temperature for a given city."""
    return f"It's 21°C in {city} right now."

model = init_chat_model("openai:gpt-4o-mini")
model_with_tools = model.bind_tools([get_temperature])

response = model_with_tools.invoke("What's the temperature in Tokyo?")
print(response.tool_calls)

Instead of writing a sentence, the model responds with a request to call get_temperature with city="Tokyo" — visible in response.tool_calls. It hasn’t actually run anything yet; it’s simply told you what it wants to run. Actually executing that request, and completing the full loop, is exactly what the Tools and Tool Calling modules coming up will teach you properly, step by step.

Common mistakes worth avoiding

Before wrapping up, it’s worth naming a few real, easy-to-make mistakes with chat models specifically, since you’ll be using this component in literally every module from here forward.

Not setting max_tokens, and being surprised by a long, expensive reply. Every provider has its own default ceiling for reply length, and those defaults are often far longer than what your application actually needs. Leaving max_tokens unset “to be safe” often does the opposite — a model asked an open-ended question can generate a genuinely long response, and you pay for every token of it. Set an explicit, deliberate ceiling based on what your feature actually needs.

Using the most expensive, most capable model for every single call, out of habit. Recall the precise_model and creative_model pattern from Example 4 — real applications often need several differently-tuned instances of a model, and not every task needs your most powerful (and most expensive) option. A simple classification task and a complex research summary rarely deserve the same model. We’ll return to this idea properly once we cover dynamic model selection later in the course.

Assuming temperature=0 guarantees the exact same output every single time. It gets you very close to deterministic, repeatable output, which is genuinely useful for testing and classification tasks — but most providers don’t guarantee perfect, bit-for-bit determinism even at temperature=0, due to how their own infrastructure processes requests internally. Don’t build logic that assumes byte-for-byte identical replies across repeated calls.

What you should take away from this module

You now understand the chat model itself, in real depth, not just as a black box you call .invoke() on:

• init_chat_model("provider:model-name") creates one configured, reusable object for a specific model.
• Passing a plain string to .invoke() is a shortcut LangChain provides — it silently wraps it as a HumanMessage for you.
• The reply is always an AIMessage, carrying more than just .content — including real, exact token usage in .usage_metadata.
• Parameters like temperature, max_tokens, timeout, and max_retries work identically across every provider, because they belong to LangChain’s shared interface, not any one company’s API.
• .stream() and .ainvoke() are the streaming and async twins of .invoke() — both get full, dedicated modules soon.
• .with_structured_output() and .bind_tools() both wrap your existing model to change what kind of thing you get back — a pattern you’ll see again and again throughout this course.

Where this goes next

The next module steps back from the model itself and looks specifically at Provider Abstraction — what LangChain’s shared interface genuinely buys you across OpenAI, Gemini, and other providers, and, just as importantly, where that abstraction honestly breaks down and real, provider-specific differences still matter.