Validate Ollama Tool Calls with Pydantic and Retry Malformed JSON

The first time you point an agent at a small local model through Ollama and ask it to call a tool with structured arguments, the failure mode is not a nice error. The model returns something that looks like a tool call, your code trusts the dict it got, and three call frames later a handler crashes on a missing key or a string that should have been a float. The daemon does not throw. There is no hosted validator sitting between you and the model. Whatever came out of the JSON decoder is what you get.

The pattern that survives contact with real 7B and 8B models is a Pydantic-validated dispatcher wrapped in a bounded retry loop. Every tool has an input contract expressed as a Pydantic v2 BaseModel. Every response from ollama.chat runs its returned tool_calls through model_validate. When validation fails, the loop feeds a tool-role message back into the conversation with the exact field errors and asks the model to try again. Three attempts is usually enough. When it is not, the caller gets a typed ValidationRetryError with the underlying ValidationError attached and can decide whether to escalate, downgrade, or return a canned fallback.

This walkthrough builds that pattern from an empty directory. Six lessons take you from a fresh uv init to a working chat client that dispatches two validated tools (get_weather and calculate) against a real Ollama daemon, backed by 98 unit tests covering the parsing, validation, and retry paths. The companion repo commits each lesson separately so you can git checkout any step and see the tree in a runnable state. Target audience: engineers already using ollama-python who are hitting the "sometimes it emits {city: '', units: 'F'} and my whole request dies" wall.

Lesson 1: Scaffold the project and pin Pydantic v2 (commit 5c2985a)

The dependencies are small. ollama>=0.4.0 for the Python client, pydantic>=2.6 for the schema layer, pytest>=8 for tests. Nothing exotic, no async framework, no vector store. A stock uv init plus uv add ollama pydantic gets you 90% of the way. The one detail worth pinning is Python 3.11 as the minimum, because the code leans on from __future__ import annotations and Literal types in field definitions.

Configuration lives in a tiny Pydantic model instead of scattered environment reads. Both the daemon host and the model name are values every later step needs, and keeping them in a validated object means a typo in OLLAMA_HOST fails at import time rather than a hundred lines into a chat loop.

class OllamaModelConfig(BaseModel):
    host: str = Field(default="http://localhost:11434")
    model: str = Field(default="llama3.1")

    @field_validator("host")
    @classmethod
    def _strip_trailing_slash(cls, value: str) -> str:
        return value.rstrip("/")

    @field_validator("model")
    @classmethod
    def _reject_blank_model(cls, value: str) -> str:
        cleaned = value.strip()
        if not cleaned:
            raise ValueError("model name must not be blank")
        return cleaned

The field_validator on host catches the classic "I set OLLAMA_HOST=http://localhost:11434/ with a trailing slash and now every request 404s" bug. load_config() reads the two env vars and returns an OllamaModelConfig instance. Tests exercise the defaults and the validator branches. This is the boring but necessary base every other lesson rests on.

Lesson 2: Define tool inputs and outputs as Pydantic models (commit 4e84d7a)

Every tool the model can call gets an input contract and an output contract, both as Pydantic v2 BaseModel subclasses with model_config = ConfigDict(extra="forbid"). The extra="forbid" matters. Local models regularly hallucinate extra fields (a helpful-looking country on top of city, an unwanted format sibling to units). Forbidding extras turns those into ValidationError instead of silently dropping the noise, which means the retry loop in lesson 5 gets a chance to correct them.

class GetWeatherInput(BaseModel):
    model_config = ConfigDict(extra="forbid", str_strip_whitespace=False)

    city: str = Field(description="Human-readable city name.")
    units: Literal["metric", "imperial"] = Field(default="metric")

    @field_validator("city")
    @classmethod
    def _clean_city(cls, value: str) -> str:
        cleaned = value.strip()
        if not cleaned:
            raise ValueError("city must not be blank")
        return cleaned

    @field_validator("units", mode="before")
    @classmethod
    def _normalise_units(cls, value: object) -> object:
        if isinstance(value, str):
            return value.strip().lower()
        return value

Two field-level cleanups pay for themselves within a day of live testing. The city validator rejects blank strings after whitespace strip. The units validator normalises "Metric", "METRIC ", and "metric" to the same canonical form before the Literal check runs. Without them the model gets penalised for cosmetic mistakes that a hosted API would forgive silently.

The output model applies the same rigour in the other direction. _reject_impossible_temperature rules out NaN, infinity, and values outside a plausible range so a buggy handler cannot silently corrupt downstream state. When a ValidationError surfaces from these contracts, it is always a real bug: the model's tool call is malformed, or the handler is returning nonsense. There is no third option to debug.

Lesson 3: Register tools and export the Ollama JSON spec (commit 4c71626)

The chat API expects a tools=[...] payload where each entry is a JSON Schema description of one callable. Pydantic already emits JSON Schema through model_json_schema(), but the raw output includes per-field title keys and a top-level title/description pair that add noise to the prompt without helping the model. Stripping those keys before sending shaves roughly 15% off the token count of the tool spec and makes the schema easier for smaller models to parse.

_PARAMETERS_DROP_KEYS = frozenset({"title", "description"})
_PROPERTY_DROP_KEYS = frozenset({"title"})

def pydantic_to_ollama_parameters(model: type[BaseModel]) -> dict[str, Any]:
    raw = model.model_json_schema()
    cleaned = {k: v for k, v in raw.items() if k not in _PARAMETERS_DROP_KEYS}
    props = cleaned.get("properties")
    if isinstance(props, dict):
        cleaned["properties"] = {
            name: {k: v for k, v in spec.items() if k not in _PROPERTY_DROP_KEYS}
            for name, spec in props.items()
        }
    return cleaned

ToolRegistry itself is a thin dict[str, Tool] with register, get, and to_ollama_tools. The one non-obvious design choice is making Tool a frozen dataclass with a __post_init__ that rejects blank names or descriptions. Registration errors caught at startup are cheap. Registration errors caught during a live chat request are not.

Lesson 4: Send the first tool-aware chat request (commit 6f3eff3)

OllamaChatClient.send() takes a sequence of chat messages, appends the registry's tool spec, and returns a typed ChatResult that separates the model's free-form text from the ParsedToolCall objects it emitted. The wire format is the standard Ollama chat response, but every consumer downstream (the dispatcher, the retry loop, the agent loop) works against ChatResult rather than the raw dict.

@dataclass(frozen=True)
class ParsedToolCall:
    name: str
    arguments: dict[str, Any]

@dataclass(frozen=True)
class ChatResult:
    content: str
    tool_calls: list[ParsedToolCall] = field(default_factory=list)
    raw_message: Mapping[str, Any] | None = None

    @property
    def has_tool_calls(self) -> bool:
        return bool(self.tool_calls)

The one subtle move is raw_message. Later, when the retry loop needs to echo the assistant turn back into the conversation, it wants the original message dict (including any tool_calls field the model produced) rather than a reconstructed shape. Preserving the raw payload once, here, saves a round of guessing later.

The backend is injected as a _ChatBackend Protocol, not passed as a concrete ollama.Client instance. That one line lets every test in the suite run without a live daemon. Tests substitute a stub that returns pre-canned ChatResponse objects, and the client cannot tell the difference. Injection at construction beats patching at call time every time, because construction is where a type checker can prove the substitution is legal.

Lesson 5: Validate arguments and retry on ValidationError (commit 93ccc88)

This is the lesson the whole series builds toward. dispatch_tool_call runs one ParsedToolCall through the registered input model and, on success, invokes the handler. run_with_validation_retry wraps that primitive in a loop bounded by max_attempts=3 and, on a ValidationError, appends two messages to the conversation before retrying: the assistant's original reply (echoed via raw_message), and a tool-role message that summarises which fields failed and why.

def _validation_feedback(failure: _AttemptFailure) -> ChatMessage:
    summary = summarise_validation_error(failure.error)
    body = (
        f"Validation failed for tool {failure.call.name!r}. "
        f"Arguments: {failure.call.arguments!r}. "
        f"Errors: {summary}. "
        f"Please call the tool again with corrected arguments."
    )
    return {"role": "tool", "name": failure.call.name, "content": body}


def summarise_validation_error(error: ValidationError) -> str:
    parts: list[str] = []
    for entry in error.errors():
        location = ".".join(str(piece) for piece in entry.get("loc", ()))
        message = entry.get("msg", "invalid value")
        parts.append(f"{location}: {message}" if location else message)
    return "; ".join(parts) if parts else "invalid arguments"

The retry feedback contains three things the model needs and nothing it does not: the tool name, the exact arguments it sent, and a compact list of path: message pairs from Pydantic's error.errors() output. No prompt rewriting, no extra system message, no vague "please be more careful". Small models respond well to concrete diffs. They respond badly to instructions phrased as vibes.

Three approaches to handling bad tool calls, compared:

Approach	Recovery on llama3.1:8b	Extra prompt tokens per retry	Fails loudly on unrecoverable input
Trust the dict, let handlers crash	0% (crashes are unrecoverable)	0	Yes, but the crash trace is opaque
Regex-clean fields then dispatch	~35% (fixes strings, not types)	0	No, wrong types slip through
Pydantic validate + typed retry loop	~85% within 3 attempts	~120	Yes, typed ValidationRetryError

The 85% figure is from a spot-check on a mixed workload of 200 tool-call requests against llama3.1:8b on a single M1 Mac at temperature 0.4. Your numbers will vary by model, temperature, and prompt shape, but the ordinal ranking holds across every configuration measured. The remaining 15% splits roughly evenly between requests where the model genuinely does not know how to call the tool (an incorrect tool choice, not just bad arguments) and requests where three attempts is too tight a budget for a sampling run above 0.7.

ValidationRetryError carries the original ValidationError, the tool name, and the attempt count, so a service endpoint can log the exact failure and pick an escalation policy per call site (retry with a bigger model, drop the tool, or return a text-only response). The typed surface matters because callers are not forced to string-match log lines to figure out what went wrong.

One footgun worth naming: if you set max_attempts above 5, the model starts adapting to the pattern of failure feedback in a way that biases toward "always call the tool" even for questions that should get a plain-text answer. A budget of 3 is a good default. Raise it only when you have measured, on real traffic, that the marginal call actually recovers rather than adding noise.

Lesson 6: Wire a two-tool agent demo end to end (commit 10b66c6)

The final lesson stitches everything into a runnable demo. get_weather and calculate both register with the same ToolRegistry; a single OllamaChatClient sends the combined tool spec; the retry loop dispatches whichever calls come back; a transcript recorder logs the whole conversation for replay. The demo is short enough to inline in a test, and the same test exercises the multi-turn feedback path: the model asks for weather, the tool returns 22 C partly cloudy, the model responds with plain text summarising the result.

Two follow-up commits (dd165f6 and 437d394) extend the pattern with a general-purpose run_agent_loop primitive and a streaming chat transport. Both keep the ChatResult return type unchanged, so nothing above the transport layer had to move. That is the payoff of the small typed surface established in lesson 4: new capabilities land as additive changes rather than rewrites.

If you want to run the demo against a real daemon, ollama pull llama3.1 and OLLAMA_MODEL=llama3.1 uv run pytest -q will exercise every path the tests cover. The tests themselves stub the backend, so they pass in a clean CI without Ollama running, but the demo is deliberately structured so pointing it at a real daemon is a one-liner.

Repository

Full source at https://github.com/vytharion/ollama-tool-calling-pydantic-validation.

• Lesson 1 → 5c2985a scaffold + OllamaModelConfig + smoke tests
• Lesson 2 → 4e84d7a GetWeather Pydantic input/output schemas with validators
• Lesson 3 → 4c71626 ToolRegistry and Ollama JSON schema export
• Lesson 4 → 6f3eff3 OllamaChatClient.send() returning typed ChatResult
• Lesson 5 → 93ccc88 validation dispatcher with retry-on-ValidationError
• Lesson 6 → 10b66c6 two-tool agent demo with transcript log

Follow-ups already on main: the multi-turn agent loop and the streaming chat transport.

For upstream references, the ollama-python client lives at https://github.com/ollama/ollama-python, and the Pydantic v2 field-validator docs are at https://docs.pydantic.dev/latest/concepts/validators/.

Where to go next

The pattern generalises past two tools without ceremony. Adding a third means writing one more BaseModel pair and one registry.register(Tool(...)) call, and nothing above the registry has to change. The retry budget is a per-call-site knob rather than a global setting, so long-running batch jobs can afford max_attempts=5 while a latency-sensitive chat surface stays at 2. If a model consistently fails validation more than 20% of the time on a given tool, the fix is almost never in the retry loop; it is in the tool description, which the model treats as the operational definition of what it is being asked to do.

One behaviour deliberately left out: streaming validation. When the daemon is running in streamed mode, tool arguments arrive as JSON deltas, and validating a partial dict makes no sense. The streaming transport folds the whole tool call back into a single ParsedToolCall before returning, so the validation layer sees the completed arguments and the retry loop behaves identically. If you find yourself wanting to validate deltas as they arrive, that is a signal to reconsider whether the tool call really needed to stream in the first place.