Case Study: DuckDB Table Function API Migration

The fixture is a C++ DuckDB table-function migration.

Both runs used Codex GPT-5.5 against the same fixture. The prompt was:

Fix web_archive_scan.cpp so it is source-correct for DuckDB v1.3.2 C++ table-function projection and complex-filter pushdown API.

The stale fixture came from older DuckDB table-function examples. The patch had to update web_archive_scan.cpp for DuckDB v1.3.2 while preserving projection and filter pushdown behavior.

A patch could compile and still be wrong if it dropped columns needed only by pushed filters.

Case study replayReal agent replays

DuckDB API migration

model Codex GPT-5.5

Clean

$

Fix web_archive_scan.cpp so it is source-correct for DuckDB v1.3.2 C++ table-function projection and complex-filter pushdown API.

Without GitHits

Incomplete

tokens

0

time

0s / 496s

1. Ready. Click "Watch Replay" to start.

With GitHits

Complete

tokens

0

time

0s / 327s

1. Ready. Click "Watch Replay" to start.

Watch Replay

Result

Run	Time	Tokens	Tools
With GitHits	327s	1.41M	40
Without GitHits	496s	1.73M	48

The GitHits run was about 34% faster and used about 19% fewer processed tokens. It also found the filter_prune path used for filter-only column pruning.

The no-GitHits run updated API shapes and passed syntax checks. It missed the path that tells DuckDB how to retain columns required only by pushed filters.

Failure Surface

The broken fixture mixed several kinds of API drift:

• The pushdown_complex_filter callback signature had changed.
• Projection handling needed to account for DuckDB’s column_t and projection_ids behavior.
• TableFunctionSet needed the right include path.
• Complex filter pushdown had to cooperate with column pruning instead of only erasing filters from a local vector.

The no-GitHits run fetched headers, made a sparse checkout, inspected optimizer files, and ran syntax checks. Most of that work was source acquisition and orientation: branch, header, optimizer file, usage site, and version.

The GitHits run moved from the fixture into version-specific DuckDB source evidence:

• table_function.hpp for the v1.3.2 callback and init inputs.
• projection_ids usage to understand projection mapping.
• remove_unused_columns.cpp to confirm how filter-only columns are preserved.
• logical_get.cpp and pushdown_get.cpp to connect the table function API to optimizer behavior.
• function_set.hpp to settle the TableFunctionSet include.

That evidence covered both the API shape and the planner behavior.

Missed Behavior

The fixture contained this warning in the stale code:

// Older examples used column_ids directly. This is wrong when DuckDB has
// produced projection_ids for a filtered/projection-pushed scan.

That comment points at the right area, but it is incomplete. projection_ids explains visible output columns. It does not fully explain columns needed only to evaluate filters that have been pushed into the scan.

The no-GitHits result did enough source work to avoid compilation errors. It stopped before the filter-pruning behavior.

In this fixture, compile-correct was not enough. The behavior depended on the optimizer path, not only on the table-function header.

Evidence Path

The required context was DuckDB v1.3.2 source in the files that implement table functions and optimizer pruning:

• The new pushdown_complex_filter shape.
• The projection mapping between requested output columns and table function state.
• The role of filter_prune when filters reference columns that are not otherwise projected.
• The include boundary for TableFunctionSet.

The no-GitHits run inspected real source too, but spent more of the run getting and locating it. The GitHits run reached the relevant files earlier and followed the planner path far enough to catch the column-pruning edge case.