// Proposal · v3 of three

The Ratchet

Both earlier designs assume the job is to build an assembly line that produces code, with model judgment as the quality gate. That assumption is the ceiling. Models reviewing models is weakly correlated with correctness; execution is the only thing that isn’t. So build the machine that manufactures verifiers, and treat code as the cheap part.

Written 2026-08-27 ·

This one is not ready to build, and that is the point. It depends on something nobody here has solved: running a full test corpus several times over, in parallel, cheaply enough to do it on every work item. Until that is costed, the rest of this design is an argument rather than a plan — so the first thing you can fund is answering that question, and a legitimate outcome is that the answer kills the idea.

Reviewers are weak discriminators

A model reviewing another model’s code shares most of its blind spots. When the reviewer catches something, it is usually a slip, a convention, or a missing case that was visible on the surface. Deep defects — a race, an off-by-one under load, a guard on the wrong side of a branch — are exactly the ones that read fine.

This project’s own history contains the cleanest example. A dry-run promised 116 worktree removals; the real run, on the identical population seconds later, removed none. The skip it should have hit is gated on the run not being a dry run, so a dry run falls through and reports what it would have deleted. Nobody lost data — but two code reviews had already cited that dry-run transcript as their verification. The reviews were not lazy; the evidence was lying. The rule written down afterwards is really a statement about where truth comes from.

So the question a design should optimize is not “how do we review better?” but “how much of our judgment can we convert into something that executes?” Every check that becomes executable is permanent, free to re-run, uncorrelated with model blind spots, and makes every future change safer. Every check that stays a reviewer’s opinion is paid for again on every PR and forgotten the moment the PR closes.

The inversion. Today tests are a byproduct of features: build the thing, then write tests for it. Here the executable criterion is the primary artifact and the implementation is the byproduct. That sounds like a slogan until you notice what it does to “done”: it becomes mechanical. A model is very good at making a specific failing check pass without breaking a corpus. It is much less good at knowing when it is finished.

Three mechanisms

01

Spec first, and it must fail

No implementation starts without an executable criterion that currently fails. Review moves to the criterion, before code exists, where changing your mind is nearly free.

02

Generate many, select by running

Produce K implementations against the same failing criterion, in parallel, and let execution do the discriminating. Fix cycles get replaced by discard. Model judgment is spent only on survivors, and only on what execution cannot decide.

03

The ratchet never slips

Every defect found anywhere is converted into a permanent executable check before the item closes. The corpus only grows, so safety compounds instead of depending on vigilance.

one item, end to end 
item: "worktree reaper removes trees that still have an open PR"

# 1 — criterion first, must fail before anything is written
criterion: reaper_spares_open_pr_head()      -> FAILS  (evidence: run#8812)
review: is this the right invariant?         model judgment, pre-code, cheap

# 2 — K candidates, selected by execution not opinion
candidates: 4 in parallel, varied framing + effort
  c1 -> criterion PASS, corpus PASS     survives
  c2 -> criterion PASS, corpus 3 FAIL   discard
  c3 -> criterion FAIL                  discard
  c4 -> criterion PASS, corpus PASS     survives
review: c1 vs c4 on design only              one call, two known-correct options

# 3 — ratchet: the finding outlives the PR
reviewer noted "reaper assumes worktree == branch"
  -> promoted to property: corpus += reaper_no_branch_assumption()
  -> now runs on every future change, forever

Notice what happened to the review chain. It did not get longer or shorter — it moved. Judgment is spent twice: once on the criterion before code exists, once on two implementations already known to be correct. Nobody reviews a diff hoping to spot a bug.

The problem we have not solved

Everything above rests on one assumption: that verification is cheap and fast. If running the corpus is slow, generate-and-select collapses — you cannot afford to run four candidates against it, and the design degrades into an expensive way to do what the other two already do. That is a hard prerequisite, not a detail, and it is not solved.

Worse, the problem compounds with the design’s own success. The ratchet means the corpus only grows. Every defect ever found becomes a permanent check, so the thing you must run K times per item gets larger every week. A design whose safety compounds also has a cost that compounds, and nobody has drawn that curve.

There are candidate answers. None of them is worked out:

Publishing this design without that section would be dishonest, and pretending the choice is made would be worse. So the first fundable item below is not a build — it is answering the question, with a real possibility that the answer is “too expensive” and the design gets shelved.

Where it fits, and where it doesn’t

Of the last 200 commits’ file-touches: 26% Python, 20% test code, 12% shell, 3% web — and 16% markdown plus 4% config. So roughly 60% is code or tests, which this design handles well, and 35% is documentation, config and plans, which it has nothing useful to say about. That needs an explicit second lane rather than pretending everything is mechanizable:

What you can fund

This is not a migration ladder like the other two, because the design does not deserve one yet. Phase 00 is the blocker. Phases 01–03 are the mechanisms. Phase 04 is the trial that decides whether any of it pays for itself.

  1. 00

    Cost the verification problem

    $350

    This is the blocker, and nothing else here is worth starting until it is answered. What does it actually cost to run the corpus K times in parallel, on what, and how fast? Candidates include remote build infrastructure such as Namespace, plain local parallelism, and a tiered corpus that runs a fast subset per candidate and the full set once on the survivor. None of these is costed and none is chosen. The deliverable is a number and a recommendation, not a build — and a real possible outcome is “too expensive, design shelved”.

  2. 01

    The criterion-first gate

    $450

    No implementation starts without an executable criterion that currently fails — a test, a property, a type, an invariant, a benchmark threshold. Review moves to the criterion, before code exists, where changing your mind is nearly free. “Is this the right thing to prove?” is a far better use of model judgment than “is this code good?”

  3. 02

    Generate and select by running

    $600

    Models are much better at producing K candidates than at judging one. Produce several implementations against the same failing criterion, in parallel, and let execution discriminate. Fix cycles — the most expensive thing in the loop — get replaced by discard. Judgment is spent only on survivors, and only on what execution cannot decide: design, naming, fit. Depends entirely on phase 00.

  4. 03

    The corpus that only grows

    $400

    Every defect found anywhere — a review finding, a revert, an incident, a flaky test — becomes a permanent executable check before the item closes. A class of bug caught once becomes structurally impossible rather than watched-for. This also needs a documented pawl-release: checks can be retired, but only with an explicit record of what class of regression is being re-admitted.

  5. 04

    The one-subsystem trial

    $700

    A month of criterion-first and generate-select on a single subsystem, measured against the rest of the repo on fix-cycle count and defect density. This is the design’s own recommendation and it is the honest way to adopt it. You are funding an experiment that can legitimately conclude the idea does not pay for itself.

ALL

Fund the whole experiment

$2200

All five. The parts add up to $2,500, so this is $300 less than buying them separately — and it is still an experiment, not a delivery.

What would make me wrong

If you want one sentence. Build the fleet rewire — it is the substrate for all three and none of it is wasted. Then run criterion-first and generate-select on a single subsystem for a month and measure. If the numbers move, that is your answer, and it will be a better answer than any of these three documents.