TechByteByByte

Function Calling Deep Dive

Closing Level 3: the precise mechanics of how an LLM's structured decision becomes an executable function call — and the critical clarification that the LLM never literally executes anything itself.

#AI Agents#AI#Function Calling#Level 3

Begin with the problem

Function calling is a request, not remote control. The model produces a tool name and arguments; trusted application code validates and executes them.

JSON schema → function request → argument validation → application call → function result → model

What you will learn

  • Follow function calling from tool schema to model request, application execution, and returned result.
  • Understand why the model requests a function but the application executes it.
  • Validate names and arguments before any function runs.
  • Handle malformed calls, tool failures, retries, and duplicate requests safely.

Current real-system grounding: Google’s tool documentation shows the critical difference between provider-executed built-in tools and custom functions executed by your application.

These official links document available product features. They do not reveal every provider’s private implementation, hidden reasoning, default setting, or internal limit.

1. The problem this module solves

Module 6 covered tools broadly. This module zooms in on the specific mechanism — function calling — that makes tool selection structured and reliable, and gives the single most important clarification in this entire section its own dedicated, explicit treatment.

2. Why Function Calling Was Introduced

BEFORE function calling: getting a model to reliably indicate "call
                         THIS tool with THESE parameters" meant
                         parsing free-form TEXT the model generated
                         -- fragile, since the model's
                         phrasing could vary in ways that broke
                         simple text parsing.

Function calling: models are SPECIFICALLY trained to produce
                  STRUCTURED output (JSON) when a task calls for
                  invoking a function -- directly connecting to
                  your Prompt Engineering course's structured output
                  coverage, now applied specifically to tool
                  invocation.

3. The Structured Tool Invocation

{
  "function_name": "process_refund",
  "arguments": {
    "order_id": "4471",
    "amount": 49.99
  }
}

This structure is reliable in a way free-form text never was — the application can directly parse function_name and arguments without needing to interpret ambiguous natural language.

4. The Critical Clarification — What the LLM Does NOT Do

This is the single most important idea in this module, worth stating with maximum directness:

The LLM does NOT literally execute the external function. The LLM’s ENTIRE contribution is generating the structured JSON above — a real DECISION about which function to call and with what arguments. The APPLICATION (your code) is what actually calls process_refund(order_id="4471", amount=49.99) and runs real logic against a real database or API.

flowchart LR
    LLM[LLM] -->|"decides: {function_name, arguments}"| App[Application]
    App -->|executes REAL function| Ext[External System / API]
    Ext -->|result| App
    App -->|feeds result back| LLM

This directly extends Module 5’s “LLM decides, application executes” boundary — function calling is precisely the mechanism through which that boundary is implemented in practice.

5. Function Schema — Precisely What the Model Is Given

{
    "name": "process_refund",
    "description": "Processes a monetary refund for a given order.",
    "parameters": {
        "type": "object",
        "properties": {
            "order_id": {"type": "string", "description": "The order to refund"},
            "amount": {"type": "number", "description": "Refund amount in USD"},
        },
        "required": ["order_id", "amount"],
    },
}

This is the same schema concept from Module 6, expressed in the specific structured format most function-calling APIs expect — directly connecting the conceptual tool schema to its real, implementable form.

6. Parameter Validation — A Necessary Safety Step

The LLM's generated arguments are NOT automatically trustworthy:

- It might GENERATE a value of the wrong TYPE (a string where a
  number was expected)
- It might OMIT a required parameter
- It might HALLUCINATE a plausible-but-incorrect value (Module 5,
  Section 6's hallucination risk, applied directly here)

This is precisely why Module 6, Section 5’s validation step exists as a separate stage — the application should validate the LLM’s generated arguments BEFORE executing anything against a real system, exactly the same principle as validating any untrusted user input.

7. A Real Developer Example

TechCorp’s refund-handling flow, with the LLM/application boundary made completely explicit:

1. LLM receives: goal + state ("customer disputes non-delivery,
   order confirmed delivered by carrier")
2. LLM GENERATES (does NOT execute): {"function_name":
   "process_refund", "arguments": {"order_id": "4471", "amount":
   49.99}}
3. APPLICATION receives this JSON
4. APPLICATION VALIDATES: order_id is a real string, amount is a
   real positive number -- both required fields present
5. APPLICATION checks: does this function require human approval
   (Module 6, Section 9)? YES -> route to a human reviewer
6. ONLY after human approval does the APPLICATION actually call
   process_refund(order_id="4471", amount=49.99)
7. The REAL result (success/failure) is fed back to the LLM as the
   NEXT observation

Notice: the LLM’s job ended at step 2. Every subsequent step is the application’s responsibility.

8. A Simple Agentic AI Connection

Every framework covered in Module 22 implements function calling exactly this way internally — when you see LangChain or LangGraph “call a tool,” what’s happening underneath is this exact LLM-generates-JSON, application-executes-function pattern, wrapped in convenient abstractions.

9. How Is This Used in AI?

🤖 How Is This Used in AI?

Function calling is the standard mechanism underlying essentially every production agent’s tool invocation — models are specifically trained to reliably produce this structured output format, which is precisely why it replaced fragile, free-form text parsing as the real industry standard for connecting LLM reasoning to real executable actions.

10. Real-World Applications

  • Any agent system where the LLM needs to reliably trigger real actions (database writes, API calls, file operations)
  • Structured data extraction (a related use of the same mechanism, covered in your Prompt Engineering course)
  • Multi-tool agent systems needing reliable disambiguation between several available functions

11. Common Mistakes

Incorrect idea: Believing the LLM directly runs the function.

Why it is incorrect: As shown directly in Section 4, this is the most important misconception this module exists to correct.

Incorrect idea: Executing LLM-generated arguments without validation.

Why it is incorrect: As shown directly in Section 6, generated arguments can be malformed or hallucinated — validate before executing, always.

Incorrect idea: Treating function calling as fundamentally different from the “tool calling” concept in Module 6.

Why it is incorrect: They’re the same underlying idea — function calling is the specific, structured mechanism that implements tool calling in practice.

12. Limitations

  • Even with structured function calling, the LLM can generate arguments for a real, existing function that are still factually wrong (a validly-typed but hallucinated order ID) — structural validation catches malformed input, not necessarily factually incorrect input
  • Function calling reliability varies by model — some models are more consistently accurate at this than others

13. Quick Reference

flowchart TD
    Context[Goal + State + Function Schemas] --> LLM[LLM]
    LLM -->|generates JSON decision| Parse[Application Parses JSON]
    Parse --> Validate{Validate<br/>Arguments}
    Validate -->|Invalid| Reject[Return Error as Observation]
    Validate -->|Valid| Approval{Requires Human<br/>Approval?}
    Approval -->|Yes| Human[Human Review]
    Approval -->|No| Execute[Application Executes<br/>Real Function]
    Human -->|Approved| Execute
    Execute --> Result[Real Result]
    Result --> LLM

14. Code — Implementing the Complete Function-Calling Boundary

🎯 Target of this example: implement Section 7’s complete real developer example directly — the LLM produces a structured decision (never executing anything), while the application parses, validates, and only then executes the real function, exactly Section 4’s critical clarification made into observable, working code.

Example 1 — Simple

import json

def simulate_llm_function_call_decision(user_intent: str) -> dict:
    """Simulates what an LLM produces during function
    calling -- a structured JSON DECISION, not an executed result
    (Section 3-4)."""
    if "refund" in user_intent.lower():
        return {
            "function_name": "process_refund",
            "arguments": json.dumps({"order_id": "4471", "amount": 49.99}),
        }
    return {"function_name": None, "arguments": None}

def application_executes_function(decision: dict, function_registry: dict) -> dict:
    """This is the APPLICATION's job -- executing what the
    LLM decided. The LLM did NOT run this itself (Section 4's
    critical clarification)."""
    if decision["function_name"] is None:
        return {"executed": False, "reason": "No function call decided"}

    fn = function_registry.get(decision["function_name"])
    if fn is None:
        return {"executed": False, "reason": f"Unknown function: {decision['function_name']}"}

    args = json.loads(decision["arguments"])
    result = fn(**args)
    return {"executed": True, "result": result}

def process_refund(order_id: str, amount: float) -> str:
    return f"Refunded ${amount} for order {order_id}"

registry = {"process_refund": process_refund}

decision = simulate_llm_function_call_decision("Customer wants a refund")
print(f"LLM's decision (NOT yet executed): {decision}")

execution = application_executes_function(decision, registry)
print(f"Application's execution: {execution}")

Expected Output:

LLM's decision (NOT yet executed): {'function_name': 'process_refund',
'arguments': '{"order_id": "4471", "amount": 49.99}'}
Application's execution: {'executed': True, 'result': 'Refunded
$49.99 for order 4471'}

What we conclude from this example: the LLM’s decision is just a JSON string at first — no refund has actually happened yet at that point. Only application_executes_function runs process_refund — exactly Section 4’s clarification made directly, unambiguously observable as two distinct steps.

Example 2 — Intermediate

import json

def validate_arguments(function_name: str, arguments: dict, schema: dict) -> dict:
    """Directly implements Section 6's validation step -- checking
    the LLM's generated arguments BEFORE any execution happens,
    exactly like validating any untrusted input."""
    required = schema.get("required", [])
    missing = [p for p in required if p not in arguments]
    if missing:
        return {"valid": False, "error": f"Missing required parameters: {missing}"}

    type_map = {"string": str, "number": (int, float)}
    for param, value in arguments.items():
        expected_type = schema["properties"].get(param, {}).get("type")
        if expected_type and expected_type in type_map:
            if not isinstance(value, type_map[expected_type]):
                return {"valid": False, "error": f"Parameter '{param}' should be {expected_type}, "
                                                  f"got {type(value).__name__}"}
    return {"valid": True, "error": None}

refund_schema = {
    "properties": {"order_id": {"type": "string"}, "amount": {"type": "number"}},
    "required": ["order_id", "amount"],
}

# A malformed LLM decision -- amount is a string, not a number
malformed_args = {"order_id": "4471", "amount": "forty-nine dollars"}
valid_args = {"order_id": "4471", "amount": 49.99}

print("Validating malformed arguments:", validate_arguments("process_refund", malformed_args, refund_schema))
print("Validating valid arguments:", validate_arguments("process_refund", valid_args, refund_schema))

Expected Output:

Validating malformed arguments: {'valid': False, 'error': "Parameter
'amount' should be number, got str"}
Validating valid arguments: {'valid': True, 'error': None}

What we conclude from this example: the malformed arguments (representing an LLM hallucinating or misformatting a value) are correctly caught BEFORE any attempt to execute the real function — exactly Section 6’s principle: never trust generated arguments as automatically correct, validate them the same way you’d validate any untrusted input.

Example 3 — Production Grade

import json
from dataclasses import dataclass
from enum import Enum

class FunctionCallOutcome(Enum):
    EXECUTED = "executed"
    VALIDATION_FAILED = "validation_failed"
    UNKNOWN_FUNCTION = "unknown_function"
    NO_CALL_DECIDED = "no_call_decided"

@dataclass
class FunctionCallResult:
    outcome: FunctionCallOutcome
    detail: str
    result: object = None

class FunctionCallHandler:
    """A production-style handler implementing Section 7's COMPLETE
    flow -- parse the LLM's decision, validate, and ONLY THEN
    execute -- with every outcome explicitly classified, directly
    supporting Module 21's observability."""

    def __init__(self, registry: dict, schemas: dict):
        self.registry = registry
        self.schemas = schemas

    def _validate(self, function_name: str, arguments: dict) -> tuple:
        schema = self.schemas.get(function_name, {})
        required = schema.get("required", [])
        missing = [p for p in required if p not in arguments]
        if missing:
            return False, f"Missing required parameters: {missing}"

        type_map = {"string": str, "number": (int, float)}
        for param, value in arguments.items():
            expected_type = schema.get("properties", {}).get(param, {}).get("type")
            if expected_type in type_map and not isinstance(value, type_map[expected_type]):
                return False, f"Parameter '{param}' should be {expected_type}, got {type(value).__name__}"
        return True, None

    def handle(self, llm_decision: dict) -> FunctionCallResult:
        function_name = llm_decision.get("function_name")
        if function_name is None:
            return FunctionCallResult(FunctionCallOutcome.NO_CALL_DECIDED, "LLM did not decide to call a function")

        if function_name not in self.registry:
            return FunctionCallResult(FunctionCallOutcome.UNKNOWN_FUNCTION, f"'{function_name}' not registered")

        arguments = json.loads(llm_decision["arguments"])
        is_valid, error = self._validate(function_name, arguments)
        if not is_valid:
            return FunctionCallResult(FunctionCallOutcome.VALIDATION_FAILED, error)

        # ONLY NOW does real execution happen
        result = self.registry[function_name](**arguments)
        return FunctionCallResult(FunctionCallOutcome.EXECUTED, "Function executed successfully", result)

def process_refund(order_id: str, amount: float) -> str:
    return f"Refunded ${amount} for order {order_id}"

handler = FunctionCallHandler(
    registry={"process_refund": process_refund},
    schemas={"process_refund": {"required": ["order_id", "amount"],
                                 "properties": {"order_id": {"type": "string"}, "amount": {"type": "number"}}}},
)

good_decision = {"function_name": "process_refund", "arguments": json.dumps({"order_id": "4471", "amount": 49.99})}
bad_decision = {"function_name": "process_refund", "arguments": json.dumps({"order_id": "4471", "amount": "bad_value"})}

for label, decision in [("Valid decision", good_decision), ("Malformed decision", bad_decision)]:
    outcome = handler.handle(decision)
    print(f"{label}: [{outcome.outcome.value}] {outcome.detail}")

Expected Output:

Valid decision: [executed] Function executed successfully
Malformed decision: [validation_failed] Parameter 'amount' should be
number, got str

What we conclude from this example: the FunctionCallOutcome enum gives every possible result of the LLM-decides-application-executes pipeline an explicit, classifiable label — exactly the kind of structured tracking a real production agent system needs to distinguish “the LLM’s decision was fine but execution failed” from “the LLM’s decision itself was malformed,” directly building on Module 6’s tool-error classification.

15. Interview Questions

Q: State precisely what an LLM does and does not do during function calling.

Ans: The LLM’s entire contribution is generating a structured decision — typically JSON specifying a function name and arguments — reasoning about which function should be called and with what parameters given the current context. The LLM does not literally execute the function itself; it has no ability to actually run code or call an external API. The application receiving this generated decision is responsible for parsing it, validating the arguments, and actually executing the real function against real systems.

Q: Why does function calling represent a real improvement over having the model produce free-form text describing what action to take?

Ans: Free-form text requires the application to parse and interpret potentially ambiguous natural language to figure out what the model intended — fragile, since phrasing can vary in ways that break simple parsing logic. Function calling has models specifically trained to produce structured, directly parseable output (like JSON) when a task calls for invoking a function, letting the application reliably extract the function name and arguments without needing to interpret natural language at all.

Q: Why is validating an LLM’s generated function arguments a necessary step, rather than an optional safety check?

Ans: Generated arguments aren’t automatically trustworthy — the LLM could generate a value of the wrong type, omit a required parameter, or hallucinate a plausible-looking but incorrect value. Executing these arguments against a real system without validation risks real errors or unintended consequences. Validating generated arguments the same way you’d validate any untrusted input — checking required fields are present and correctly typed — is a necessary safety step before any real execution occurs.

Q: Design a function-calling handler that clearly distinguishes between different kinds of failures a real production team would want to monitor separately.

Ans: I’d classify outcomes into distinct categories: no function call was decided at all, the LLM specified a function that doesn’t exist in the registry, the generated arguments failed validation (wrong type or missing required fields), or the function executed successfully. Each of these represents a different failure mode with a different root cause and fix — an unknown function suggests a tool description or prompt problem, a validation failure suggests the model is generating malformed arguments, and tracking these separately (as distinct, explicit statuses rather than one generic “failed”) lets a team diagnose and address the actual, specific underlying issue.

16. What You Should Remember

  • Function calling’s real improvement over free-form text: models produce structured, reliably parseable output, not ambiguous natural language.
  • The LLM decides; the application executes — verified directly by observing the LLM’s decision exist as a plain JSON string before any actual function call happens.
  • Generated arguments must be validated before execution — verified directly by catching a malformed, wrongly-typed argument before it ever reaches the real function.

17. Quick Practice

Design a function schema (following Section 5’s format) for a new capability, then write out, step by step exactly like Section 7’s TechCorp example, what the LLM would generate versus what the application would then do with that decision.

18. Next Step

Next: Module 8 — Reasoning and Planning — Level 4 begins here: why agents sometimes need to plan ahead across multiple steps before acting at all, using a complex, multi-part task as the running example.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed