Case Study: Pydantic AI Tool Context

The fixture is a Pydantic AI support-routing agent.

Each run passes tenant-local data through Pydantic AI dependencies: tenant, region, ticket priorities, and escalation policies. The generated code created the agent with deps_type=SupportDeps, but registered both tools with @agent.tool_plain. Those tools cannot receive RunContext, so they returned global fallback values.

Both runs used Codex GPT-5.5 against the same fixture. The prompt was:

Fix this Python fixture so `pytest` succeeds, preserving tenant-aware Pydantic AI routing tools.

The target package was pydantic-ai 2.0.0b7.

Case study replay

Pydantic AI tenant routing tools

model Codex GPT-5.5

Watch Replay Clean

$

Fix this Python fixture so `pytest` succeeds, preserving tenant-aware Pydantic AI routing tools.

Without GitHits

Costly

tokens

0

time

0s / 189s

1. Ready. Click "Watch Replay" to start.

With GitHits

Efficient

tokens

0

time

0s / 99s

1. Ready. Click "Watch Replay" to start.

Result

Run	Time	Tokens	Tools
With GitHits	99s	393,469	21
Without GitHits	189s	901,661	28

Both runs produced a passing patch. The GitHits run used 508,192 fewer processed tokens and finished 90 seconds sooner.

Failure

The tools were registered with tool_plain:

@agent.tool_plain
def lookup_ticket(ticket_id: int) -> str:
    return f"{DEFAULT_TENANT}:{ticket_id}:{DEFAULT_PRIORITY}"

@agent.tool_plain
def escalation_policy(ticket_id: int) -> str:
    return (
        f"{DEFAULT_TENANT}:{DEFAULT_PRIORITY}:"
        f"{DEFAULT_POLICY}:{DEFAULT_REGION}"
    )

tool_plain is correct for tools that do not need the run context. These tools depend on SupportDeps.

The tests checked data flow:

• acme and globex must route differently.
• lookup_ticket and escalation_policy must read the same per-run dependencies.
• Unknown tickets should still fall back to normal and standard, but the fallback must keep the active tenant and region.

Fix

Use context-aware tools and read ctx.deps:

from pydantic_ai import Agent, RunContext

@agent.tool
def lookup_ticket(ctx: RunContext[SupportDeps], ticket_id: int) -> str:
    priority = ctx.deps.priorities.get(ticket_id, DEFAULT_PRIORITY)
    return f"{ctx.deps.tenant}:{ticket_id}:{priority}"

@agent.tool
def escalation_policy(ctx: RunContext[SupportDeps], ticket_id: int) -> str:
    priority = ctx.deps.priorities.get(ticket_id, DEFAULT_PRIORITY)
    policy = ctx.deps.escalation_policies.get(priority, DEFAULT_POLICY)
    return f"{ctx.deps.tenant}:{priority}:{policy}:{ctx.deps.region}"

The patch depends on three package facts:

• @agent.tool is the decorator for tools that receive RunContext.
• RunContext[SupportDeps] gives the tool access to the active dependency object.
• Both tools need to read ctx.deps; fixing only one still leaves inconsistent routing.

Trace

The replay shows seven GitHits tool calls:

• Two search calls for Pydantic AI docs around agent.tool, RunContext, deps, and tool_plain.
• Three docs_read calls on the current tools documentation.
• One code_grep call for the def tool( implementation.
• One code_read call on the package source where Agent.tool is defined.

Those calls gave the agent the package contract before editing: use @agent.tool when a tool needs RunContext, then access per-run dependencies through ctx.deps.

The no-GitHits run had to reconstruct the same information from the local environment. It searched installed pydantic_ai internals, found the package path, read exports, read agent source, read run-context source, patched, tested, cleaned local artifacts, reread the file, and tested again.

That local probing accounts for most of the 508k-token gap.

Evidence

A passing fixture test proves the local behavior for the tested cases. The docs and source establish that the patch uses the intended Pydantic AI mechanism.

The GitHits trace had a short evidence chain:

• Docs showed the context-aware tool pattern for the current package.
• Source search found the Agent.tool implementation surface.
• Source reads confirmed that this was the right API boundary.
• pytest verified tenant-specific behavior in the fixture.

The package has multiple valid tool decorators. The evidence points to the one that matches the data-flow requirement.

The final patch did not rewrite the routing model, change the tests, or move tenant data into prompts. It changed the decorator and dependency access in the two tools.

Accuracy Risk

The incorrect alternatives are close to the correct patch:

• Keep tool_plain and add globals.
• Put tenant data into the prompt.
• Capture dependencies in a closure instead of using Pydantic AI’s run context.
• Fix lookup_ticket but leave escalation_policy on defaults.
• Change the tests to match global fallback behavior.

All of those preserve the original bug or create a more brittle fixture.

The GitHits run found the package mechanism in docs and source before editing. The no-GitHits run found the same mechanism by probing the installed package.