System, User, and Assistant Messages

Start with the real problem

AI chat systems label messages so the model can tell application instructions, user requests, and earlier assistant replies apart.

An API is a set of rules that lets two software programs communicate. Instead of seeing a chat page, the API receives labeled message data.

Chat appears to be one conversation, but APIs send a structured sequence of messages and instructions. Understanding those layers prevents accidental mixing of product policy, user requests, and model output.

application instructions + conversation messages + current user input → model response

What you will learn

Explain system/developer, user, and assistant roles.
Compare how providers expose instruction layers.
Preserve conversation state correctly.
Avoid treating role priority as guaranteed compliance.

How this connects to current AI systems

OpenAI, Gemini, and Anthropic expose instruction and conversation layers differently, so application code should follow the selected API rather than assume one universal message shape.

1. Why This Module Exists

Every prompt so far in this course has been shown as one plain block of text. Real LLM APIs actually work differently — conversations are built from distinct, labeled messages. This module explains that structure, since it directly underlies chat apps, memory, and agents.

2. The Idea, in Plain Language

Instead of one long block of text, a real conversation with an AI is made of separate messages, each labeled with who it’s from.

System message:      sets the AI's overall behavior and rules for
                      the whole conversation (like a persona,
                      Module 5, plus operating instructions)

User message:           what the actual person (or your app, on
                      their behalf) says

Assistant message:        what the AI said back

A conversation is simply a growing list of these labeled messages, sent to the AI together each time.

3. A Concrete Example

[System]:    "You are a helpful assistant for a cooking app. Keep
             answers under 100 words and always suggest one
             substitution if an ingredient might be hard to find."

[User]:      "What's a good marinade for chicken?"

[Assistant]: "Try soy sauce, garlic, ginger, and a splash of honey.
             If you don't have fresh ginger, ground ginger works as
             a substitute."

[User]:      "Can I use this for shrimp instead?"

Notice: the second user message (“Can I use this for shrimp instead?”) only makes sense because the earlier messages are still part of the conversation. This is really how multi-turn chat actually works — the entire message history is sent to the AI again with each new message, not just the newest one.

4. Why Separate Roles, Instead of One Big Prompt?

You could, in theory, cram everything into one giant text block — but separating roles gives real, practical benefits:

The system message stays consistent across an entire conversation, without needing to be repeated in every user message
The conversation history (user and assistant messages) gives the AI real context about what’s already been discussed — directly enabling the “shrimp” follow-up question above to make sense
Applications can programmatically manage each part separately — update the system message without touching history, or trim old messages (Module 16 covers this) without losing the system-level instructions

5. A Real Example From a Developer’s Perspective

This structure is exactly what makes a multi-turn chatbot feature possible in code — not a single request, but a growing, managed list:

Turn 1:
messages = [
  {system}: "You are a customer support agent for Thread & Co."
  {user}:    "Where's my order?"
]
→ AI responds, response is added to the list

Turn 2:
messages = [
  {system}: "You are a customer support agent for Thread & Co."
  {user}:    "Where's my order?"
  {assistant}: "Can you share your order number?"
  {user}:    "It's #4471"
]
→ Now the AI has the full context needed to actually help

Every single turn, the entire growing message list is sent again — this is precisely why long conversations eventually run into context window limits, and why context management (Module 16) becomes a real, practical concern.

6. A Simple Agentic AI Example

For agents, the assistant’s messages often aren’t just text — they can represent a decision to use a tool, with the tool’s result coming back as a kind of message too:

[System]:      "You are a scheduling agent with access to a
               check_availability tool."

[User]:        "Book me a meeting room for 2pm tomorrow."

[Assistant]:      (decides to call check_availability(time="2pm
                  tomorrow"))

[Tool result]:       "Room 3B is available."

[Assistant]:            "Room 3B is available at 2pm tomorrow —
                        should I book it?"

The tool result becomes part of the ongoing message history too — the agent’s next response is generated with full awareness of what the tool returned. Module 18-19 cover tool calling and agent message flows in complete depth.

7. How Is This Used in AI?

🤖 How Is This Used in AI?

Every chat-based AI product — customer support bots, coding assistants, general chatbots — is built on exactly this system/user/assistant message structure under the hood. It’s the foundation that makes multi-turn conversation, persistent behavior (via the system message), and tool-using agents all possible.

8. When Should You Think About This Structure?

Any time you’re building something with multiple conversation turns, not just a single one-off request
Any time you want behavior (persona, rules) to stay consistent across an entire conversation without repeating it every turn
Any time you’re working with an agent or a tool-using system, where tool results become part of the ongoing message flow

9. When Is a Single Message Enough?

One-off, single-turn requests (like most examples earlier in this course) don’t need multiple message roles — a single user message is fine

10. Common Mistakes

Incorrect idea

Putting persona or behavior instructions in the user message instead of the system message.

Why it is incorrect

This works, but it means those instructions have to be repeated in every single message rather than set once for the whole conversation — Module 5’s code examples showed this exact improvement directly.

Incorrect idea

Forgetting that the full message history is resent every turn.

Why it is incorrect

This has real, direct implications for cost and context limits (Module 3, Module 25) — a long conversation isn’t “free” just because only the newest message feels short.

Incorrect idea

Not including earlier turns when they’re actually needed for context.

Why it is incorrect

If a user’s new message depends on something discussed earlier (“can I use this for shrimp instead?”), the AI really needs that earlier message included, not just the latest one.

11. Limitations

This structure organizes a conversation clearly — it doesn’t, by itself, solve what to do when a conversation grows too long for the context window (Module 16 covers that directly)
A system message sets initial behavior, but doesn’t guarantee the AI follows it with perfect consistency across a very long conversation
The exact message format and available roles can vary slightly between different AI providers’ APIs — the underlying concept is consistent, but implementation details differ (Module 26 covers model-specific differences)

The role names are not identical everywhere

The durable idea is instruction priority plus conversation history, not one universal JSON format. OpenAI APIs can distinguish developer, user, and assistant messages. Gemini commonly separates a system instruction from user/model contents. Claude commonly uses a top-level system prompt plus user/assistant messages. Check the provider’s current API instead of copying one provider’s role names into another.

Analogy: The Stage Play Script & The Director’s Notes Think of system, user, and assistant messages in terms of theatrical logistics:

The Director’s Notes (System Message): Pinned to the dressing room mirror. It says: “You are a grumpy 1920s hardboiled detective. Always speak in short, blunt sentences and use street slang. Never break character.”

This instruction stays active for the entire duration of the play (the conversation history) and sets the universal rules of behavior.

The Actor 1 Lines (User Message): “Did you locate the stolen diamond, inspector?”

The Actor 2 Lines (Assistant Message): “Footprints in the rain. Diamond’s gone. Suspect left town.”

The Actor 1 Follow-up: “Where did they go?”

The Script History: The script accumulates these lines sequentially. When Actor 2 needs to deliver line 4, they review the entire script from page 1 to remember who they are and what has been said so far.

📊 Visual Chart: The Labeled Message Array Structure

Here is how conversation turns compile into structured message lists sent to LLM APIs:

graph TD
    classDef sys fill:#2ecc71,stroke:#333,stroke-width:1px,color:#fff;
    classDef usr fill:#3498db,stroke:#333,stroke-width:1px,color:#fff;
    classDef ast fill:#f1c40f,stroke:#333,stroke-width:1px,color:#fff;

    subgraph MessageList ["Structured API Input Array"]
        M1["{ 'role': 'system', 'content': 'You are a supportive tutor.' }"]:::sys
        M2["{ 'role': 'user', 'content': 'Help me with question 1.' }"]:::usr
        M3["{ 'role': 'assistant', 'content': 'Sure! Question 1 asks...' }"]:::ast
        M4["{ 'role': 'user', 'content': 'What is the first step?' }"]:::usr
    end

    MessageList --> LLMEngine["LLM Next-Token Prediction Engine"]
    LLMEngine --> M5["{ 'role': 'assistant', 'content': 'The first step is...' }"]:::ast

12. Quick Reference — The Whole Idea in One Diagram

System message   -> sets behavior/rules for the WHOLE conversation
User message        -> what the person (or app) actually says
Assistant message      -> what the AI said back

Full message list resent EVERY turn
   ↓
Enables: persistent behavior, multi-turn context, tool-result
         integration for agents

13. Prompts in Code — Calling an LLM

Here’s how system, user, and assistant messages actually look when calling an LLM through code — building a real, growing conversation.

Example 1 — Simple

A single system message plus one user message — the basic structure.

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=150,
    system="You are a helpful assistant for a cooking app. Keep "
           "answers under 100 words.",
    messages=[
        {"role": "user", "content": "What's a good marinade for chicken?"}
    ]
)
print(response.content[0].text)

Example 2 — Intermediate

A multi-turn conversation, manually building the growing message list across two turns.

import anthropic

client = anthropic.Anthropic()
SYSTEM_PROMPT = "You are a helpful assistant for a cooking app. Keep " \\
                "answers under 100 words."

messages = [{"role": "user", "content": "What's a good marinade for chicken?"}]

response1 = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=150,
    system=SYSTEM_PROMPT, messages=messages,
)
assistant_reply = response1.content[0].text
print("Assistant:", assistant_reply)

# Add the assistant's reply, then the next user message, to the growing list
messages.append({"role": "assistant", "content": assistant_reply})
messages.append({"role": "user", "content": "Can I use this for shrimp instead?"})

response2 = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=150,
    system=SYSTEM_PROMPT, messages=messages,
)
print("Assistant:", response2.content[0].text)

Example 3 — Production Grade

A reusable Conversation class that manages the growing message history properly, including a basic safeguard for context length (previewing Module 16).

import anthropic

client = anthropic.Anthropic()

class Conversation:
    def __init__(self, system_prompt: str, max_turns: int = 20):
        self.system_prompt = system_prompt
        self.messages = []
        self.max_turns = max_turns

    def send(self, user_message: str) -> str:
        self.messages.append({"role": "user", "content": user_message})

        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=300,
            system=self.system_prompt,
            messages=self.messages,
        )
        reply = response.content[0].text
        self.messages.append({"role": "assistant", "content": reply})

        # Basic safeguard: keep the conversation from growing unbounded
        if len(self.messages) > self.max_turns * 2:
            self.messages = self.messages[-(self.max_turns * 2):]

        return reply

chat = Conversation(
    system_prompt="You are a helpful assistant for a cooking app. "
                   "Keep answers under 100 words."
)
print(chat.send("What's a good marinade for chicken?"))
print(chat.send("Can I use this for shrimp instead?"))

Wrapping the message list inside a Conversation class keeps the system prompt set once, handles the growing history automatically, and includes an early, simple safeguard against unbounded growth — the real production concern Module 16 covers in full.

When to use it—and when not to

Use it when:

building chat applications and agents.
separating product policy from user content.

Do not rely on it when:

a single stateless request needs no history.
message roles are being used as the only security control.

14. Interview Questions

Q: What are the three main message roles used in modern LLM APIs, and what does each one do?

Ans: The system message sets overall behavior, persona, and rules for the entire conversation. The user message contains what the actual person (or the application on their behalf) says. The assistant message contains what the AI generated in response. A conversation is simply a growing, ordered list of these labeled messages, sent to the model together with each new request.

Q: Why is it generally better to put persistent behavior instructions in the system message rather than repeating them in every user message?

Ans: The system message is set once and applies across the whole conversation, so behavior stays consistent without needing to be retyped every turn. Putting the same instructions in every user message instead works, but it’s redundant, easy to forget on some turns, and makes the persona or rules harder to update consistently across an entire conversation.

Q: Why does a long, multi-turn conversation become more expensive and eventually hit limits, even if each individual message is short?

Ans: Because the full message history — every previous user and assistant message — is resent to the model with every new request, not just the newest message. As a conversation grows, the total token count being sent each turn grows too, directly increasing cost and eventually approaching the model’s context window limit, even though any single message in isolation might be quite short.

Q: How do tool results fit into the system/user/assistant message structure in an agentic system?

Ans: A tool’s result is typically fed back into the ongoing message history as its own kind of entry (sometimes represented as part of an assistant message or a dedicated tool-result message, depending on the specific API), so that the model’s next response is generated with full awareness of what the tool returned. This lets the agent’s tool calling and reasoning form a connected, ongoing conversation rather than disconnected, isolated requests.

15. What You Should Remember

Real conversations are built from labeled messages — system, user, and assistant — not one plain block of text.
The entire message history is resent with every new turn, which is exactly why long conversations have real cost and context-limit implications.
System messages are the right place for persistent behavior (persona, rules) that should apply across an entire conversation.

16. Quick Practice

Sketch out (in plain text, no need for real code) a 3-turn conversation for a fitness app assistant: one system message, and two user/assistant exchanges where the second question depends on context from the first.

17. Next Step

Next: Module 16 — Context Management — what happens as a conversation (or any prompt) grows too large, and the real strategies for managing that limit deliberately.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026