Zero Hallucination AI Agents: Pydantic & Tool Calling with Gemma 3 (Lab 2)

Watch Full Video on YouTube: https://www.youtube.com/watch?v=xfMGtXuYGj0

Download notebook to follow along: https://github.com/MysteryBytes-Labs/agentic_ai/tree/main/local-agentic-ai-gemma3/

The Fragility of Raw Prompting in Autonomous Agents

When transitioning an LLM from a simple conversational chatbot into an autonomous agent, the interaction model changes fundamentally. Instead of outputting freeform text for human consumption, the model must output structured data intended to drive software tools, execute terminal commands, or query production databases.

In early agent prototypes, developers typically rely on raw prompting:

"Answer the user's question. If you need a tool, output JSON in this format: 
```json { "tool": "get_weather", "location": "Tokyo" } ```"

In production, raw prompting fails catastrophically:

1. Malformed Syntax: Models frequently emit trailing commas, unescaped quotes, or conversational markdown wrappers like Here is the JSON you requested: { ... }.
2. Missing Required Fields: An agent may invoke a database update tool but forget the required primary key or timestamp.
3. Type Mismatches: Passing string numerals ("101") into Python functions expecting native integers or floats crashes the downstream API.
4. Hallucinated Functions: The model invents plausible-sounding but non-existent tools (e.g., calling search_weather_forecast() when only get_weather exists).

To build resilient, autonomous agents, we need formal data contracts enforced at the generation layer.

Execution Pattern Comparison

The Constrained Decoding Breakthrough: By passing a Pydantic model's JSON Schema to Ollama's format parameter, the inference runtime physically restricts token sampling at the logit level. The model cannot emit tokens that violate the grammar of the schema.

4-Phase Closed-Loop Execution Architecture

Our agent follows a disciplined, 4-phase execution loop:

┌────────────────────────────────────────────────────────┐
│                   User Instruction                     │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
┌────────────────────────────────────────────────────────┐
│       Phase 1: Constrained Tool Decision (Ollama)      │
│  Gemma 3 analyzes intent & emits AgentToolDecision     │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
┌────────────────────────────────────────────────────────┐
│       Phase 2: Runtime Pydantic Validation             │
│  Validates schema, coerces types, checks boundaries    │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
┌────────────────────────────────────────────────────────┐
│       Phase 3: Deterministic Tool Dispatcher           │
│  Python executes the matched function (Local/Mock)     │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
┌────────────────────────────────────────────────────────┐
│       Phase 4: Final Synthesis & Markdown Response     │
│  Gemma 3 synthesizes tool observations into answers    │
└────────────────────────────────────────────────────────┘

Step 1: Environment Setup & Ollama Client

We use the official ollama Python library connected to our local daemon running on localhost:11434.

pip install -q ollama pydantic ipython

from pydantic import BaseModel, Field, ConfigDict
from typing import Optional, Literal, Dict, Any
import json
import ollama
from IPython.display import Markdown, display

# Initialize local Ollama client
client = ollama.Client(host="http://localhost:11434")
MODEL_NAME = "gemma3:4b"

print(f"✅ Connected to Ollama at {client._client.base_url}")
print(f"  Target Model: {MODEL_NAME}")

Step 2: Runtime Validation & Automatic Type Coercion

Pydantic models guarantee data integrity through automated type coercion and boundary constraints. Notice how Pydantic automatically converts string integers ("101" and "28") into native Python integers while validating age boundaries:

class UserProfile(BaseModel):
    user_id: int = Field(..., description="Unique user identifier")
    name: str = Field(..., min_length=2, description="Full user name")
    email: str = Field(..., description="Contact email address")
    age: Optional[int] = Field(None, ge=18, le=120, description="Age between 18 and 120")

# Demonstrate auto-coercion ("101" -> 101, "28" -> 28)
user = UserProfile(user_id="101", name="Alex Morgan", email="[email protected]", age="28")
print(json.dumps(user.model_dump(), indent=2))

Step 3: Defining Discrete Tool Parameter Schemas

Instead of allowing arbitrary inputs, we define explicit Pydantic models for each tool the agent can access:

1. Environmental Telemetry Schema (WeatherToolParams)

Constrains temperature units to an explicit enumeration (celsius or fahrenheit) using Python's Literal type:

class WeatherToolParams(BaseModel):
    """Retrieve current weather conditions, temperature, and attire recommendations."""
    location: str = Field(..., description="The target city, e.g. 'Tokyo' or 'New York'")
    unit: Literal["celsius", "fahrenheit"] = Field("celsius", description="Temperature scale")

2. Mathematical Calculation Schema (CalculatorToolParams)

class CalculatorToolParams(BaseModel):
    """Perform precise arithmetic calculations and evaluations."""
    expression: str = Field(..., description="Math expression to calculate, e.g. '25 * 48'")

Step 4: The Unified Master Routing Contract

To give the agent autonomous control over whether to invoke a tool or reply directly, we encapsulate all possibilities into a unified master decision contract: AgentToolDecision.

class AgentToolDecision(BaseModel):
    """Unified decision schema for agent tool selection and parameterization."""
    tool: Literal["get_weather", "calculator", "none"] = Field(
        ..., description="Selected tool name, or 'none' if no tool is required"
    )
    reasoning: str = Field(..., description="Step-by-step reasoning for this decision")
    weather_params: Optional[WeatherToolParams] = Field(None, description="Parameters if get_weather is selected")
    calculator_params: Optional[CalculatorToolParams] = Field(None, description="Parameters if calculator is selected")

# Generate JSON Schema for Ollama constrained decoding
agent_schema = AgentToolDecision.model_json_schema()
print("✓ Unified Agent Decision Schema compiled.")

Step 5: Sandboxed Tool Implementations & Dispatcher

We implement safe, deterministic Python backends for our tools without requiring external third-party API keys:

# 1. Environmental Telemetry Database
def get_weather_data(location: str, unit: str = "celsius") -> Dict[str, Any]:
    city = location.split(",")[0].strip().title()
    database = {
        "Tokyo": {"temp_c": 19, "condition": "Partly Cloudy", "humidity": "62%", "clothing": "Light jacket"},
        "New York": {"temp_c": 22, "condition": "Sunny", "humidity": "45%", "clothing": "T-shirt and sunglasses"},
        "London": {"temp_c": 14, "condition": "Light Rain", "humidity": "80%", "clothing": "Raincoat and umbrella"},
        "Toronto": {"temp_c": 16, "condition": "Breezy", "humidity": "55%", "clothing": "Sweater or windbreaker"}
    }
    record = database.get(city, {"temp_c": 20, "condition": "Clear", "humidity": "50%", "clothing": "Casual wear"})
    temperature = record["temp_c"] if unit == "celsius" else round(record["temp_c"] * 9/5 + 32, 1)
    
    return {
        "location": city,
        "temperature": f"{temperature}°{'C' if unit == 'celsius' else 'F'}",
        "condition": record["condition"],
        "humidity": record["humidity"],
        "recommended_attire": record["clothing"]
    }

# 2. Sandboxed Calculator (Safe Expression Evaluator)
def calculate_expression(expression: str) -> Dict[str, Any]:
    try:
        # Whitelist allowed mathematical characters to prevent code injection
        allowed_chars = set("0123456789+-*/(). % ")
        if not all(c in allowed_chars for c in expression):
            raise ValueError("Contains disallowed characters")
        
        # Evaluate without access to built-in functions
        result = eval(expression, {"__builtins__": None}, {})
        return {"expression": expression, "result": result, "status": "success"}
    except Exception as e:
        return {"expression": expression, "error": str(e), "status": "error"}

# Central Tool Dispatcher Registry
TOOL_REGISTRY = {
    "get_weather": lambda p: get_weather_data(p.location, p.unit),
    "calculator": lambda p: calculate_expression(p.expression)
}

Step 6: Compiling the Autonomous Agent Orchestrator

The run_agent() function executes the complete decision-dispatch-synthesis loop:

def run_agent(user_query: str) -> str:
    print(f"\n💬 User Query: \"{user_query}\"")
    
    # Phase 1: Structured Decision via Ollama Constrained Decoding
    response = client.chat(
        model=MODEL_NAME,
        messages=[
            {
                "role": "system",
                "content": (
                    "You are an autonomous AI agent with access to external tools. "
                    "Analyze the user request. When selecting a tool, populate the corresponding "
                    "parameter object (weather_params for get_weather, calculator_params for calculator). "
                    "If no tool is needed, set tool to 'none'."
                )
            },
            {"role": "user", "content": user_query}
        ],
        format=agent_schema  # Enforces physical token conformance to Pydantic JSON Schema
    )
    
    # Phase 2: Validate Output with Pydantic
    decision = AgentToolDecision.model_validate_json(response["message"]["content"])
    print(f"🤖 Agent Reasoning: {decision.reasoning}")
    print(f"🔧 Tool Selected  : {decision.tool}")
    
    # If no tool is needed, return the model's direct reasoning
    if decision.tool == "none":
        return decision.reasoning
        
    # Phase 3: Execute the matched tool from the registry
    if decision.tool == "get_weather":
        tool_output = TOOL_REGISTRY["get_weather"](decision.weather_params)
    elif decision.tool == "calculator":
        tool_output = TOOL_REGISTRY["calculator"](decision.calculator_params)
    
    print(f"⚙️ Tool Output    : {tool_output}")
    
    # Phase 4: Final Response Synthesis
    synth_response = client.chat(
        model=MODEL_NAME,
        messages=[
            {
                "role": "system", 
                "content": "You are an articulate technical assistant. Formulate a clean, formatted Markdown answer based on the tool result."
            },
            {"role": "user", "content": user_query},
            {"role": "assistant", "content": f"Tool execution result: {json.dumps(tool_output)}"},
            {"role": "user", "content": "Please present the final answer to my question based on this data."}
        ]
    )
    
    return synth_response["message"]["content"]

Step 7: Multi-Scenario Benchmark Evaluation

We test our agent across three distinct intents:

Scenario 1: Telemetry Lookup (Weather in Tokyo)

ans1 = run_agent("What is the weather in Tokyo in celsius?")
display(Markdown(ans1))

• Reasoning: Need to retrieve current temperature for Tokyo in Celsius.
• Tool Selected: get_weather with location="Tokyo", unit="celsius".
• Result:
  • Temperature: 19°C (Partly Cloudy)
  • Humidity: 62%
  • Recommended Attire: Light jacket

Scenario 2: Precise Arithmetic Calculation

ans2 = run_agent("Calculate 25 * 48.")
display(Markdown(ans2))

• Reasoning: Mathematical multiplication required.
• Tool Selected: calculator with expression="25 * 48".
• Result: 1200 (Calculated deterministically with zero math hallucination).

Scenario 3: Direct Knowledge (No Tool Required)

ans3 = run_agent("What is the capital of France?")
display(Markdown(ans3))

• Reasoning: Factual question that can be answered directly using internal weights.
• Tool Selected: none.
• Result: Answered directly with zero unnecessary tool dispatches.

Key Takeaways

1. Logit-Level Grammar Enforcement: Passing format=PydanticModel.model_json_schema() eliminates JSON parse errors, syntax errors, and missing parameters.
2. Unified Routing Contracts: Encapsulating tool names, parameters, and reasoning into a single AgentToolDecision model allows the LLM to explain its rationale while maintaining strict typing.
3. Defense-in-Depth: Combining local constrained decoding with Python-side boundary checks guarantees that malicious or corrupted queries cannot execute arbitrary code.

Masterclass Series Roadmap

• Lab 1: High-Performance Local LLMs with Gemma 3 & Token Streaming
• Lab 2 (This Guide): Pydantic Schemas & Constrained Tool Calling
• Lab 3 (Next Up): OpsAgent — Autonomous DevOps & Hardware Telemetry Copilot (psutil + Safe Process Triage)
• Lab 4: StockAgent — Autonomous Financial Market Screener (Yahoo Finance Scraper + Institutional Memos)

Download the complete companion notebook lab2_pydantic_tool_calling.ipynb and follow along with the code.