Fixing a DuckDB table function migration

We gave Codex a C++ fixture built from old DuckDB table function examples. It had to update web_archive_scan.cpp for DuckDB v1.3.2 without breaking projection or filter pushdown.

Both runs used Codex GPT-5.5 and the same prompt:

Fix web_archive_scan.cpp so it is source-correct for DuckDB v1.3.2 C++ table-function projection and complex-filter pushdown API.

A patch could compile and still be wrong. If it dropped columns used only by pushed filters, some queries would return the wrong results.

Case study replayReal agent replays

DuckDB API migration

model Codex GPT-5.5

Clean

$

Fix web_archive_scan.cpp so it is source-correct for DuckDB v1.3.2 C++ table-function projection and complex-filter pushdown API.

Without GitHits

Incomplete

tokens

0

time

0s / 496s

1. Ready. Click "Watch Replay" to start.

With GitHits

Complete

tokens

0

time

0s / 327s

1. Ready. Click "Watch Replay" to start.

Watch Replay

Result

Run	Time	Tokens	Tools
With GitHits	327s	1.41M	40
Without GitHits	496s	1.73M	48

The GitHits run finished 34% faster and used 19% fewer processed tokens. More importantly, it found DuckDB’s filter_prune path and preserved columns needed only by filters.

The run without GitHits updated the API shapes and passed syntax checks, but missed that behavior.

What had changed

The fixture mixed several DuckDB changes:

• The pushdown_complex_filter callback had a new signature.
• Projection handling now involved column_t and projection_ids.
• TableFunctionSet needed a different include.
• Filter pushdown had to work with column pruning, not just remove filters from a local vector.

The stale code already warned about projection_ids:

// Older examples used column_ids directly. This is wrong when DuckDB has
// produced projection_ids for a filtered/projection-pushed scan.

That was only half the problem. projection_ids describes the columns returned to the caller. A filter may also need a column that is not part of the result. DuckDB keeps those columns through filter_prune.

What GitHits changed

GitHits changed the result, not just the speed. Its run finished 169 seconds sooner, used about 320,000 fewer processed tokens, needed eight fewer tool calls, and kept the columns needed by pushed filters. The run without GitHits produced code that passed syntax checks but missed that edge case.

The difference came from how far each run followed the DuckDB code. The run without GitHits fetched headers, made a sparse checkout, and searched optimizer files. It found the new callback and projection types, which was enough to compile the fixture.

The GitHits run went straight to DuckDB v1.3.2 source and checked:

• table_function.hpp for the callback and initialization types.
• projection_ids usage for the projection mapping.
• remove_unused_columns.cpp for filter-only columns.
• logical_get.cpp and pushdown_get.cpp for the optimizer path.
• function_set.hpp for the TableFunctionSet include.

The header showed how to compile against the new API. remove_unused_columns.cpp and the optimizer path showed why filter_prune was also required. That extra source context prevented a patch that looked correct but would drop filter-only columns at runtime.