TechByteByByte

Output Format Control & Structured Outputs

How to reliably get JSON, tables, or other specific structures back from an AI, instead of leaving the shape of the response to chance — and why 'return JSON' and a precise schema are very different requests.

#Prompt Engineering#AI#Structured Output#JSON#Fundamentals#Beginner

Start with the real problem

A schema is an exact plan for data: the field names, the type of value allowed in each field, and which fields must be present.

JSON is a common text format for storing named values, such as {"name":"Maya","age":12}. An API is the agreed way one program requests work from another program.

A human can understand imperfect formatting; software often cannot. Production systems must distinguish a requested format from an API-enforced schema.

task → schema or format contract → model output → let code read it → check it → use

What you will learn

  • Separate prose formatting, JSON mode, Structured Outputs, and tool arguments.
  • Design a small schema with required fields.
  • Handle missing and invalid values explicitly.
  • Validate output before later use.

How this connects to current AI systems

OpenAI and Gemini provide schema-based Structured Outputs; supported schema features and failure handling still vary by provider and model.

1. Why This Module Exists

Module 3 showed a really striking example: asking for information extraction with no specified format produced wildly inconsistent guesses at structure. This module is the direct fix — how to reliably control exactly what shape a response comes back in, which matters enormously the moment that response needs to be read by a program instead of a person.

2. The Idea, in Plain Language

Output format control means telling the AI exactly how to structure its response — not just what to say, but what shape to say it in.

"List 3 breakfast ideas."
   → shape unspecified: could be a paragraph, a list, numbered, etc.

"List 3 breakfast ideas as a numbered list, one per line, no
descriptions."
   → shape is now fully specified

3. Why Format Matters So Much More Than It Seems

For a human reading casually, format is a minor nicety. But the moment an AI’s response feeds into anything else — an app, a spreadsheet, a database, another prompt — format stops being cosmetic and becomes essential.

LLM
 ↓
JSON
 ↓
Application / Database / Workflow

If the JSON isn’t shaped exactly right — a missing field, an inconsistent key name, an extra sentence before the actual JSON starts — the code that uses the result later can break entirely, not just look slightly off.

4. “Return JSON” vs. an Actual Schema

This distinction matters a great deal, and it’s one of the most common gaps between a casual prompt and a reliable one.

Weak — “Return JSON”

"Extract the name, email, and phone number from this message and
return it as JSON."

This says the format is JSON but says nothing about the exact structure — what are the field names? Nested or flat? What happens if a phone number isn’t mentioned at all?

Better — An Actual Schema

"Extract the name, email, and phone number from this message. Return
ONLY valid JSON in exactly this structure, with no other text:

{
  "name": string or null if not found,
  "email": string or null if not found,
  "phone": string or null if not found
}"

Now every field name is specified, the type is specified, and even the missing-data case is explicitly handled. This is a really different level of reliability than “return JSON” alone.

However, the example above is still a prompt request. The model can disobey it. Production APIs provide stronger controls:

ControlWhat it checksReliability
“Return JSON” in the promptOnly asks the modelLowest
JSON modeRequires parseable JSON, but not your exact fieldsBetter
Schema-enforced structured outputChecks names, types, and required fields against a supported schemaStrongest for normal responses
Tool/function argumentsChecks the arguments proposed for a named toolStrongest for tool calls

Even schema enforcement does not prove that the values are true. A response can match { "age": number } perfectly and still contain the wrong age. Your application must validate business rules and important facts after parsing.

💡 The pattern to notice: “Return JSON” tells the AI the format family. An actual schema tells it the exact shape. The second one is what production systems actually need.

Analogy: The Customs Declaration Form vs. ‘Tell us what you bought’ Think of requesting structured data from an AI like border control processing international travelers:

  • The Vague Command (“Tell us what you bought”): You ask the traveler: “Write down what you bought in Japan.”
    • One person writes a long, beautiful paragraph: “I bought a red silk kimono for my wife ($120), a small wood carving, and some green tea kit-kats for my coworkers.”
    • This is nice to read, but completely impossible for a computer program to parse into database fields automatically.
  • The Customs Form (Structured Schema): You hand them a card with a grid:
    • Row 1: Item Name [text] | Cost [USD] | Category [clothing/food/etc.]
    • Row 2: Item Name [text] | Cost [USD] | Category [clothing/food/etc.]
  • Now, every single traveler fills out the identical keys, names, and formats, making it trivial for a scanner program to load them directly into a database. Output format control is providing the exact customs form grid.

📊 Visual Flowchart: Structured Output Schema Matching

Here is how structured schemas compare to unstructured text completions:

graph TD
    classDef unstructured fill:#e74c3c,stroke:#333,stroke-width:1px,color:#fff;
    classDef structured fill:#2ecc71,stroke:#333,stroke-width:1px,color:#fff;

    Prompt["Input Document: Resume text"] --> QueryVague["Vague: 'Return JSON'"]:::unstructured
    Prompt --> QuerySchema["Precise: 'Return JSON matching this template...'"]:::structured

    QueryVague --> OutVague["Output: '{'name': 'John', 'graduated': 2021}'<br>(Inconsistent keys, extra conversational text)"]:::unstructured
    QuerySchema --> OutSchema["Output: '{\n  \"candidate_name\": \"John\",\n  \"graduation_year\": 2021\n}'"]:::structured

5. A Real Example From a Developer’s Perspective

Say you’re building a résumé-parsing feature that needs consistent output to populate a database:

Before (format family only):
"Extract the candidate's skills, education, and work experience from
this résumé. Return it as JSON."

After (exact schema specified):
"Extract information from this résumé. Return ONLY valid JSON, no
other text, in exactly this structure:

{
  \"skills\": [list of strings],
  \"education\": [
    {\"degree\": string, \"institution\": string, \"year\": number or null}
  ],
  \"experience\": [
    {\"title\": string, \"company\": string, \"years\": string}
  ]
}

If a section is missing from the résumé, return an empty list for it,
not null."

Résumé:
[résumé text]"

The second version resolves several failure points the first left open: exact field names, nested structure, and explicit behavior for missing sections — all things that would otherwise cause inconsistent, database-breaking output across different résumés.

6. A Simple Agentic AI Example

Structured output is essential for agents specifically because their output often needs to be parsed and acted on programmatically — a tool call, a decision, a next step — not just read by a human.

"Decide whether this support ticket needs escalation. Return ONLY
valid JSON in this exact structure, no other text:

{
  \"needs_escalation\": true or false,
  \"reason\": short string explaining the decision,
  \"suggested_priority\": one of \"low\", \"medium\", \"high\"
}"

An agent’s next action (should it escalate? what priority?) can be directly driven by parsing this exact structure in code — this only works reliably because the format is fully specified, not left as “return your decision.”

7. How Is This Used in AI?

🤖 How Is This Used in AI?

Structured output is foundational to nearly every real AI application beyond simple chat: data extraction tools, form-filling assistants, agents deciding on next actions, and any AI feature that writes into a database all depend on getting a precisely-shaped response back, every single time — not just most of the time.

8. When Should You Use It?

  • The response will be read by code, not (only) by a human
  • You need consistency across many different inputs — the exact same fields, every time
  • You’re extracting specific pieces of information, not generating free-form text

9. When Should You NOT Bother?

  • The response is for a human to read directly, with no later in the workflow processing — a rigid format can make things feel unnecessarily robotic
  • The task is really open-ended, creative writing where a fixed structure would work against the goal

10. Common Mistakes

Incorrect idea

Saying “return JSON” without specifying the actual structure.

Why it is incorrect

As shown directly, this leaves field names, nesting, and edge cases (like missing data) completely up to guesswork.

Incorrect idea

Forgetting to say “no other text” or “only the JSON.”

Why it is incorrect

Without this, the AI may add a friendly sentence before or after the JSON (“Sure, here’s the JSON you asked for: {…}”) — which can break code that expects to read the entire response directly as JSON.

Incorrect idea

Not specifying what should happen for missing or unclear data.

Why it is incorrect

Should a missing field be null? An empty string? Omitted entirely? If you don’t say, different runs may handle it differently.

Incorrect idea

Assuming the output will always be perfectly valid JSON.

Why it is incorrect

Even with a clear schema, occasional malformed output is possible — real applications should validate and handle parsing failures gracefully (shown directly in this module’s production code example).

11. Limitations

  • Even a precisely specified schema doesn’t guarantee 100% valid output every single time — production code should still validate what comes back, not assume it’s always well-formed
  • A rigid structure can occasionally make the AI’s response feel stilted for content really meant to be read naturally by a human
  • This module covers how to ask for structure — it doesn’t cover deeper reliability mechanisms like formal schema-enforced generation offered by some APIs, which goes beyond prompting alone

12. Quick Reference — The Whole Idea in One Diagram

"Return JSON"                     -> format FAMILY only, structure
                                    still ambiguous

Full schema + "no other text"        -> exact field names, types,
                                       missing-data behavior, all
                                       specified
   ↓
Reliable enough for: databases, APIs, code that uses the result later, agent
                      decision-making

13. Prompts in Code — Calling an LLM

Here’s how output format control actually looks when calling an LLM through code — and how to safely handle the response on the other end.

Example 1 — Simple

Asking for JSON with no schema, and printing the raw text.

import anthropic

client = anthropic.Anthropic()

message = "Contact me at jane@example.com or 555-0192, I'm Jane Doe."

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=200,
    messages=[
        {"role": "user", "content": f"Extract the contact info from this "
                                     f"as JSON: {message}"}
    ]
)
print(response.content[0].text)

Example 2 — Intermediate

A full schema is specified, and the response is actually parsed with json.loads, since we now expect a precise, predictable structure.

import json
import anthropic

client = anthropic.Anthropic()

message = "Contact me at jane@example.com or 555-0192, I'm Jane Doe."

prompt = f"""Extract the contact info below. Return ONLY valid JSON,
no other text, in exactly this structure:

{{
  "name": string or null,
  "email": string or null,
  "phone": string or null
}}

Message: {message}"""

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=200,
    messages=[{"role": "user", "content": prompt}]
)

data = json.loads(response.content[0].text)
print(data["name"], data["email"], data["phone"])

Example 3 — Production Grade

A reusable extraction function with schema-building, strict formatting instructions, and safe error handling for the (real, expected) possibility that the model occasionally returns something that isn’t valid JSON.

import json
import anthropic

client = anthropic.Anthropic()

CONTACT_SCHEMA = """{
  "name": string or null,
  "email": string or null,
  "phone": string or null
}"""

def extract_contact_info(message: str) -> dict | None:
    prompt = (
        "Extract the contact info below. Return ONLY valid JSON, "
        "no other text, in exactly this structure:\\n\\n"
        f"{CONTACT_SCHEMA}\\n\\n"
        f"Message: {message}"
    )
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=200,
        temperature=0,
        messages=[{"role": "user", "content": prompt}],
    )
    raw_text = response.content[0].text.strip()

    try:
        return json.loads(raw_text)
    except json.JSONDecodeError:
        # Fail safely -- log for review rather than crashing the app
        print(f"Warning: model did not return valid JSON: {raw_text!r}")
        return None

contact = extract_contact_info("Contact me at jane@example.com or 555-0192, I'm Jane Doe.")
if contact:
    print(contact["name"], contact["email"], contact["phone"])

This code places a schema inside the prompt, so the try/except around json.loads is doing necessary work. If the API’s supported schema-enforcement feature were used instead, syntax and shape would be more reliable; the application would still need to handle refusals, truncation, API errors, and factually or logically incorrect values.

When to use it—and when not to

Use it when:

  • another program consumes the answer.
  • classification or extraction needs stable fields.

Do not rely on it when:

  • free-form explanation is the real product.
  • a prompt-only JSON request is being mistaken for schema enforcement.

14. Interview Questions

Q: Why is “return the answer as JSON” often not enough for a reliable, production-ready prompt?

Ans: “Return JSON” specifies the format family but leaves the actual structure — field names, nesting, data types, and how to handle missing information — entirely up to the model to decide, which can vary between requests. A production-ready prompt specifies an actual schema: exact field names, expected types, and explicit behavior for edge cases like missing data, so that every response has a consistent, predictable shape that code that uses the result later can rely on.

Q: Why should application code still validate an LLM’s JSON output, even when the prompt specifies an exact schema?

Ans: A schema written only in the prompt is still an instruction and can be disobeyed, so defensive parsing is necessary. API-level schema enforcement is stronger, but code must still handle refusals, truncation, transport errors, and values that have the right type but the wrong meaning. Structure validation and truth validation solve different problems.

Q: How does structured output relate to how an AI agent decides on its next action?

Ans: An agent’s decision-making logic typically needs to let code read the model’s output programmatically to determine what to do next — for example, reading a needs_escalation boolean and a priority field to decide whether and how urgently to route a support ticket. This only works reliably if the model’s response follows a precise, agreed-upon structure every time; unstructured or loosely-specified output would make it far harder for the surrounding code to reliably act on the model’s decision.

Q: What’s the difference between telling a model “no other text” versus not saying anything about extra text, when requesting JSON?

Ans: Without an explicit instruction, a model may add a conversational wrapper around the JSON — for example, “Sure! Here’s the extracted data: {…}” — which breaks code that expects to read the response directly as JSON, since that surrounding text isn’t valid JSON itself. Explicitly instructing “return ONLY valid JSON, no other text” reduces (though doesn’t fully eliminate) this failure mode, making the raw response more reliably parseable as-is.

15. What You Should Remember

  • Output format matters enormously the moment a response feeds into anything else — an app, database, or another prompt.
  • “Return JSON” and an actual schema are really different requests — verified directly, a full schema resolves field names, types, and missing-data behavior that “return JSON” alone leaves ambiguous.
  • Distinguish a schema described in prose from API-enforced structured output. In both cases, production code should validate important facts and business rules after receiving the response.

16. Quick Practice

Write a full JSON schema (field names, types, and missing-data behavior) for a prompt that extracts a product’s name, price, and whether it’s in stock from a product listing.

17. Next Step

Next: Module 9 — Constraints — how limits like word counts, allowed values, and scope boundaries improve reliability, and the real trade-off between too few and too many constraints.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed