The destination is the only witness

An agent asks an API to create a shipment. The API creates it. The response never arrives. Right there, in the gap, is the failure I built Irrevon for, and it has a property that separates it from every other failure in the stack: the only party that knows what happened is the one that stopped talking to you.

The caller's options are both wrong. Retry, and if the destination committed, you have shipped twice. Don't retry, and if the destination didn't commit, a real order silently never happens. Nothing in the caller's own state distinguishes the two cases, because the distinguishing fact was written on the other side of a wire that went quiet. The site puts it in five words on the home page: agents retry, destinations don't forgive.

I own this project. I created it, I maintain it, and it is public under Apache-2.0, published to PyPI as irrevon==0.1.0 with an immutable GitHub Release on 2026-07-25. Python 3.13 and up, Postgres underneath, a local read-only workbench, a deterministic demo, and a benchmark harness.

Here is the part that usually goes in a footnote, so I'll put it up here instead. There are no performance results in this write-up: no duplicate rates, no latencies, no accuracy figures, because none exist. Nothing in the benchmark is frozen, no confirmatory run has happened, and neither provider adapter has ever been called against a live destination. Every behavioral claim below rests on a synthetic reference destination I also wrote, which is exactly the kind of evidence you should discount. The preregistration says so in its own header:

STATUS: DRAFT — NOTHING IN THIS DOCUMENT IS FROZEN.

What I did build is a mechanism, a state model, and a measurement apparatus designed to be hostile to me. That last part is the interesting one, and it's where I spent the most care.

Why an LLM caller breaks the old assumptions

Distributed systems have known about lost responses for decades, and the standard defenses all lean on one assumption that agents quietly violate. The site states it plainly:

Classic distributed systems assume a deterministic caller: if a request is retried, the same bytes are resent, so argument hashes and caller-supplied idempotency keys hold. An LLM agent breaks that assumption — it regenerates the request itself.

That is the whole wedge. A conventional client retries by resending a buffer. An agent retries by thinking again, and thinking again produces different words, a different reference ID, a different argument shape for the same business intent. A survey of twelve agent frameworks reported on that same page found none enforcing exactly-once at the tool boundary.

So I went through the five things people reach for, which the README lists, and asked what each one actually proves. A request hash identifies request bytes, which is not the same thing as one business intent, so it collapses two different orders that happen to serialize alike and splits one order the agent reworded. A freshly generated UUID is a new UUID the moment the agent reconstructs the tool call, which is the exact moment you needed it stable. A provider idempotency key is genuinely good, and it helps only when that provider supports keys for that operation, retains them long enough to cover your recovery window, and applies the semantics you assumed rather than a similar-sounding one. A durable workflow engine retries with admirable reliability and will still hand a duplicate to a destination that does not deduplicate, because its exactly-once guarantee covers its own state and not somebody else's database. And an outbox proves you handed the intent off, which says nothing about what the destination committed after the response was lost, the only question you had.

None of those are wrong. They solve adjacent problems well. What none of them do is answer the post-crash question, and the post-crash question is the one where the money is.

I wrote a whole essay arguing that agents need a nervous system rather than a bigger brain, where I named the idempotency key and receiver-side dedup as the things that close the window a durable runtime merely narrows. Irrevon is me taking my own advice seriously enough to find where it stops working, which is the moment the receiver won't dedupe and the caller can't keep a key stable.

What the destination lets you know

Before any mechanism, a boundary. Everything a client can learn about an irreversible effect it may have caused is bounded by what the destination exposes, and no amount of caller-side cleverness moves that boundary. The engine names three tiers, and the canonical table renders all three including the ones where it has nothing to offer.

C1 destinations accept a caller-supplied idempotency key with a defined replay window. Duplicates are prevented natively, and the table's own entry for what Irrevon adds is "the expected null". Stripe is the reference example, and their idempotent requests documentation describes the semantics everyone else gets compared against.

C2 destinations have stable external references and a list or status query, and no dependable native idempotency. This is the engine's primary scope: duplicates are detected by querying, and re-dispatch is safe only after absence is confirmed. Shipment APIs, message APIs, order-creation mutations.

C3 destinations expose no key, no query, and no stable identifier. Lost and orphaned effects are undetectable, full stop. The table's word is UNDETECTABLE and the framing is an impossibility boundary demonstrated openly. On C3 the engine has no adjudication path at all: it parks the record, escalates, and only a human may settle it. That is not a limitation I plan to engineer away, because it isn't an engineering problem. The information was never emitted.

The tier is a description rather than a grade. The adapter guide says it directly: the tier is not a quality grade, it is what the destination exposes. A payment API is C1 because someone there did the work; a shipping API is C2 because they exposed a status endpoint and not a key. Neither fact grades their engineering, and both fully determine what I can promise on mine.

Identity from facts the model did not choose

The mechanism starts with the least reversible decision in the codebase: effect identity comes from stable upstream business facts and nothing else.

intent_id is the lowercase-hex SHA-256 over the RFC 8785 canonical JSON bytes of exactly three members: effect_type, scope, and stable_ids. That's it. The order ID, the invoice ID, the authorization ID, plus what kind of effect this is and whose namespace it lives in. The ledger binds that same value as the effect_id primary key, with CHECK (effect_id ~ '^[0-9a-f]{64}$') in the migration mirroring the INTENT_ID_RE pattern in Python so the two cannot drift.

What is deliberately absent from that tuple is the model's prose. The parameters payload, the thing the agent actually wrote, is never an input to identity or to idempotency-key derivation. When the agent re-synthesizes its arguments for the same order the identity is unchanged, the retry collapses onto the same record, and the divergent parameters are kept as a parameter_variants row: evidence for the denial rather than a reason to treat this as new work.

Rules like that decay into comments unless something enforces them, so there are three enforcers. The first is a taint canary. The registrar wraps incoming parameters in ModelTainted, an opaque container where every implicit conversion a derivation path might reach for (__str__, __bytes__, iteration, indexing, formatting) raises TaintViolation. There is exactly one sanctioned exit, reveal_for_adapter(), called at the adapter boundary where the wire request gets built. Model output cannot physically flow into a hash without exploding a test. The second is an import-linter contract declaring that the identity module may not import the module handling raw model arguments, so the isolation is checked in CI rather than remembered. The third is JSON Schema at the trust boundary.

One detail in that module I'm fond of out of proportion to its size: the guard is if type(value) is not str rather than isinstance, because a str subclass would satisfy isinstance and could carry tainted or non-canonical content through the derivation path wearing a string's face. Two words of Python standing in for a paragraph about what a type actually promises.

From intent_id the rest follows. operation_id = intent_id + ':' + step, where the step is allocated by the ledger and never by the model, and the idempotency key IS that operation_id verbatim. No second derivation, no extra inputs. Postgres enforces the relation structurally rather than trusting the application to keep it: CHECK (operation_id = effect_id || ':' || step::text). If the two ever disagree, the row does not exist. The site's own summary of the rule is that no key-derivation path reads model output, and it calls that a conformance test rather than a convention, which is the distinction that matters.

The mirror image of a lesson from Dumbly sits next to this: semantic similarity may propose a duplicate and cannot act on the proposal, because the advisory module emits a ClassifierProposal type no mutation API accepts and a second import-linter contract keeps every authority module from importing it. A classifier that can merge two irreversible effects on a hunch is a worse bug than the one it was hired to catch.

Persist before dispatch, and never hold a lock across a wire

The second rule is the boring one that carries the most weight: nothing crosses the irreversible boundary before a durable record of the intent exists. Ledger.register_intent is a single transaction that inserts the effect record, appends the authority, and opens the execution, writing the None→INTENDED and INTENDED→PERSISTED transitions together. Two named crash seams bracket the commit, crashpoint("persist.pre_commit") and crashpoint("persist.post_commit"), so a test can SIGKILL the process in either position and assert the difference. The claim the site makes is precise about which side of the commit it covers: a crash before persistence is provably effect-free. A crash after it leaves a record that recovery is obligated to adjudicate.

Then the claim protocol, which is where I think most homegrown versions of this go wrong. The claim transaction commits before any wire I/O happens. The module docstring is blunt about it: locks are held only inside short claim transactions and NEVER across wire I/O. Lock order is fixed everywhere, scope_slots then effect_records then findings, and isolation is plain READ COMMITTED, because correctness is carried by locked SQL functions, unique arbiters and durable claims rather than by an isolation level. Default dispatch deadline is 30 seconds. Holding a row lock across a network call turns one slow destination into a stalled system, and it's tempting because it makes the happy path easy to reason about and the hung-destination path impossible.

Single-writer discipline is enforced in three independent layers, which I mention because "we only run one writer" is the kind of assumption that quietly stops being true. A session advisory lock, SELECT pg_try_advisory_lock(hashtext('irrevon_single_writer')), where boot raises StorageUnavailable if the lock isn't held and recovery re-checks it independently rather than trusting boot. Per-slot serialization on (scope, effect_type): upsert the slot row, SELECT ... FOR UPDATE it, lock the effect row, then consult the open_attempts view (attempts with no receipt row) and raise a retryable ScopeBusy if one is live. And the database itself, with unique arbiters including UNIQUE (execution_id, attempt_no), a UNIQUE on dispatch_receipts.attempt_id, a partial unique index making orphan-sweep emission idempotent, and UNIQUE NULLS NOT DISTINCT (execution_id, from_state) so at most one transition may ever leave a given state. That last one carries a comment calling itself defense in depth rather than the enforcement, which is accurate: the locked SQL functions are the enforcement, and the constraint catches me if I write a new path that forgets them.

Multi-writer is designed and deliberately not implemented: SKIP LOCKED leases, heartbeats and epoch fencing are all specified and all absent, and single writer is stated as a documented scaling limit in three separate places on the site, which is where a limit belongs when you'd rather people hear it from you than from production.

AMBIGUOUS is a state, not an error

The lifecycle has one encoding, in statetable.py, with the SQL enums and seed rows generated and validated from it rather than hand-copied. Seven states: INTENDED, PERSISTED, DISPATCHED, SETTLED_COMMITTED, SETTLED_FAILED, AMBIGUOUS, CANCELLED. Three are terminal per execution: the two settled states and CANCELLED.

AMBIGUOUS is not terminal, and that's the design decision the whole engine turns on. Most systems have an error state where unknown outcomes go to die, and an error state is a claim: it says the effect did not happen. When the response was merely lost, that claim is a coin flip presented as a fact. So the transport mapping refuses to make it. _SETTLE_MAP sends OK to SETTLED_COMMITTED, FAILED to SETTLED_FAILED, and both TIMEOUT and LOST to AMBIGUOUS. The adapter policy closes the obvious loophole: unknown or unrecognized destination outcomes map to LOST or TIMEOUT and therefore to AMBIGUOUS, never to FAILED, and only error shapes the capability declaration's citations prove are side-effect-free may map to FAILED. An adapter author who wants to call something a clean failure has to cite the destination's documentation for the claim.

There are 24 legal edges, each a 4-tuple of (from, to, cause, actor), so cause and actor are part of edge legality rather than metadata hanging off it, and anything outside the frozenset is illegal. A settle claiming a cause no actor is licensed to produce cannot be written at all. Leaving AMBIGUOUS requires reconciliation or replay evidence, from four named causes, none of which is inference. SETTLED_FAILED permits opening a new execution at step+1, which yields a new operation_id and therefore a new idempotency key, so a retry after a proven failure is structurally a different operation rather than the same one sent twice.

The second decision I'd defend hardest is that there are three dimensions and they are never collapsed. Dimension A, the execution lifecycle above, answers what we did. Dimension B, reconciliation classification (UNRECONCILED, CONFIRMED_UNIQUE, DUPLICATE, LOST, CONTRADICTED, ORPHANED), answers what the destination says happened. Dimension C, resolution status (OPEN, then COMPENSATED or REDISPATCHED or ACCEPTED_AS_IS or ESCALATED_HUMAN, then CLOSED), answers what a human or a policy decided. The rationale on the site is the sentence I'd keep if I could keep one:

The three dimensions are deliberately never collapsed into one status color or one flat machine: an effect can be lifecycle-settled and classification-contested at once, and the record keeps both facts.

Two consequences fall out of that separation. The pair ("DUPLICATE", "REDISPATCHED") is absent from the legal resolutions, with the reason written in the source: redispatching a duplicate manufactures more. And an orphan, a real effect at the destination with no ledger record, is representable only as a finding, because a state machine keyed on ledger records structurally cannot express an effect that has no ledger record.

Absence is the hard direction

Proving an effect exists is easy: query the destination, find one, settle committed and cite the probe. Proving an effect does not exist is where the work is, because "I asked and didn't find it" has at least three explanations and only one of them means it isn't there.

So reconciliation queries in a fixed key strength order. First the stamped client reference, the operation_id itself, usable only if the capability declaration has a non-null client_ref_field. Second the receipt's destination_ref, if a receipt came back with one. Third, declared queryable keys projected from the stable IDs, which is specified and not implemented for the shipped profile: a stub, and an honest one. If no covering key exists, the effect parks and escalates rather than guessing.

Confirmed absence then requires three conjuncts, all of them. Key coverage. Enough elapsed time: now - claimed_at must be at least the declaration's status_settlement_lag, because asking a destination about a write before its own read path has caught up is a question you already know the wrong answer to. And a second status_query with the same keys, a re-read gap later, that also returns ABSENT. Both probes get recorded in status_probes and cited by ID in the settle transition's evidence, and the ledger auditor, a read-only global-invariant checker run after every integration and fault test, independently verifies that the citations are there and legal.

Absence also has to come from an authoritative answer rather than a quiet one. The reference destination requires an HTTP 404 and the Stripe draft requires 404 plus error.code == "resource_missing", and a failed query is INDETERMINATE, never coerced to ABSENT. There are also zero retries anywhere in the adapter layer, on purpose: one function call is one wire attempt, because hidden retries corrupt attempt accounting and manufacture duplicates on exactly the tier this engine exists for. An INDETERMINATE result parks, with the reason in a one-line comment I like more than the paragraph I'd have written: retry-with-backoff is the ops loop's job.

The design constants are visible and arguable: probe deadline 10s, stuck threshold 300s, absence re-read gap 60s, redispatch evidence bound 600s. One more branch is where the engine declines to be clever. If a declaration's status_settlement_lag is null, meaning the destination publishes no visibility bound, a reconciled-absent settle is still permitted, but the LOST finding is auto-routed to ESCALATED_HUMAN and automatic redispatch is forbidden, because absence without a stated visibility bound supports human judgment and nothing else. The whole recovery posture compresses to four words on the site: never re-dispatch on belief.

Nothing gets dropped when evidence arrives late, either. _record_outcome_txn wraps the receipt and its transition in a savepoint, and on a unique violation or a DT001/DT002 error it inserts into late_responses and computes contradicts_settle: settled FAILED but the outcome says OK, or settled COMMITTED but the outcome says FAILED. That flag enqueues an audit reconcile. A late contradiction is the most useful signal the system can receive and the easiest to discard, since by the time it lands the record already looks finished.

The evidence surface

If the ledger is the argument, the ledger has to be hard to quietly edit, so append-oriented enforcement sits in three layers. The trigger layer attaches irrevon_no_rewrite() BEFORE UPDATE OR DELETE ... FOR EACH STATEMENT on all 25 fact and seed tables, raising ledger is append-only: % on % is forbidden. Statement-level rather than row-level, so it fires even on an UPDATE that would have matched zero rows, which is the version of the mistake that otherwise looks like it worked.

The privilege layer revokes everything from PUBLIC, then grants the app role SELECT everywhere and INSERT on evidence tables only. No INSERT on effect_executions, effect_transitions, findings or finding_resolutions: those are reachable exclusively through four SECURITY DEFINER functions owned by the migration role, which raise custom SQLSTATEs DT001 through DT005 mapped to typed Python errors. UPDATE and DELETE are granted to no application role on any fact table. GRANT UPDATE does appear on two tables, only because SELECT ... FOR UPDATE requires it for claim-transaction row locking, and the header comment says so outright so the next reader doesn't think they found a hole. Layer three is a separate read-only role for the workbench, with default_transaction_read_only = on and deliberately no EXECUTE on the locked functions. Migrations themselves are sha256-journaled: a journaled file whose content changed on disk is a hard integrity error, not a warning.

One note on wording, since I have been inconsistent in public. The site says "append-only" and the README says "append-oriented". The README is the more precise one: rows are never rewritten and corrections arrive as new rows, and calling that append-only invites a reader to imagine a guarantee living one layer deeper than it does.

The evidence has to be readable, too, so irrevon serve is a local workbench. It binds only 127.0.0.1 with no --host flag and no environment override, validates the Host header, implements GET and HEAD only with everything else returning 405, and connects through the SELECT-only role. The web build's own README states the property I wanted: no build of this app can mutate anything.

Stable-identifier keys stay visible while their values render as deterministic SHA-256 digests, so you can see an effect is keyed on an order_id and a customer_ref without the screen naming the customer, and raw values require the explicit local inspect --reveal. The keys are what you need to debug identity and the values are what you need to violate someone's privacy, and those were never the same requirement.

The measurement problem, and a benchmark that can refuse me

Everything above is a design, which is a hypothesis wearing a lot of tests. The question that actually matters is whether any of it reduces duplicate, orphaned and lost effects at the destination compared to what a competent engineer would already build, and I could not think of an honest way to answer that without building an instrument capable of telling me no.

IrrevonBench measures one fault-conditional construct: how often duplicate, orphaned, lost-legitimate, contradicted, ambiguous-unresolved and falsely-suppressed effects occur at the destination, per capability tier. Destination reliability, model quality, end-to-end task success, plan correctness and safety-policy compliance are named non-targets, because a benchmark that quietly measures adjacent things is how you get a result that means nothing.

A trial is one business intent handed to an arm as an Episode, and the arms package strips the labels first. The docstring sets the rule: fixture ORACLE LABELS ARE DELIBERATELY ABSENT, and an arm is handed exactly what a production integration would be handed. Seed derivation excludes arm identity, so cross-arm pairing is structural rather than a convention someone maintains: seed(cell, replicate) is the first 8 bytes of SHA-256 over the master seed, the domain tag irrevonbench/v1, the cell ID and the replicate index. The development master seed is nothing-up-my-sleeve, SHA-256 of the string irrevonbench master_seed_dev provisional v1, which I recomputed against the committed manifest.

Seven fault kinds hang off arm-neutral timeline anchors so no arm gets a differently-shaped world: crash-before-persist at T_FIRST_DURABLE, crash-after-effect-before-response and response-lost at T_RESPONSE, and retry-storm, semantic-resynthesis, stale-authorization and branch-cancellation at T_DISPATCH. Crash-before-persist is special-cased in the runner so the arm is never invoked at all: the process dies before any durable write, and nothing may exist. Semantic re-synthesis is deterministic-template rather than model-driven, described in the fixtures as the ACRFence failure shape without a model, with the business identifiers deliberately reappearing under different keys so identity-by-key-path can never shortcut the stable-id projection. A real model there would have bought realism and sold determinism, and determinism is what makes two arms comparable.

The baseline ladder runs B0 through B7 plus R, where the feature flags ARE the arm definitions and every flag set is hashed into the run manifest as config_digest. B0 is naive identical retry. B1 is argument-hash dedup, which honestly suppresses identical retries and is defeated by re-synthesis. B2 is an agent-generated key, regenerated on restart. B3 is a stable workflow-issued operation ID, which a C2 destination accepts and does not honor, the wedge under test. B4 is recorded as mechanically identical to B3 against the reference destination and kept as a distinct ladder entry anyway, with the deviation written into its spec. B5 is a durable journal. B6 is status-check-before-retry, degrading honestly to UNAVAILABLE on C3. B7 refuses to run rather than ship a strawman, since a fair model-assisted matcher needs budget parity with R's advisory classifier and that is a Stage-B item. R is the engine.

The preselected primary comparator is the composite B5+B3+B6, described as the strongest realistic conventional stack, and superiority has to reject against both B5 and the composite. Baseline strength is locked by tests, including one whose docstring says the failure must SURFACE and orders "never patch this assertion to pass", a test_lock_composite_is_strong_on_queryable_c2 that requires the composite to show zero duplicates where it honestly should, and test_b7_refuses_instead_of_running_a_strawman. Weakening a baseline is the cheapest way to win a benchmark, so the tests fail when the baselines get weak rather than when they get strong.

Then two independent oracles, which is the part I'd want a reviewer to attack first. The first is destination read-back. Ground truth is the reference destination's control_state() truth API, which no arm may touch. Duplicate detection is attribution rather than diffing: the primary mechanism is a canonical payload digest map, the fallback is stable-id projection, matches = [key for key, ids in targets if ids and ids <= values], and attribution happens only when exactly one intent matches. More than one is counted as ambiguous_attributions and never guessed, because as the oracle puts it, a wrong attribution is worse than a declared one. Neither mechanism reads any arm-chosen reference, so no arm can influence the denominator or the attribution it is judged by. Excess is then duplicate_excess, the sum of max(0, n − 1) over the per-intent effect counts, which for four intents producing 1, 2, 1 and 3 destination effects gives 3.

The second oracle is causal effect histories, credited in ADR-0032 to jepsen.history and Elle. Per run it compiles client events in the arm's own action order plus destination effects in the destination's authoritative request order, then checks eight named linear-time invariants: H1 duplicate-effect, H2 orphan, H3 lost-legitimate, H4 effect-after-cancellation, H5 unauthorized-effect, H6 false-suppression, H7 claim-contradiction, H8 effect-despite-pre-persist-crash. Three of those, H4, H5 and H8, are firewalled from confirmatory use: reported per run, labeled, never pooled, because adding a confirmatory metric is a preregistration change and not a harness change.

The guard between them is the single best thing in the project, and it fits in under forty lines. cross_check_metrics requires the metric numerators, computed independently from read-back counts, to equal the invariant-violation counts from the history checker. Any discrepancy is fatal, and the run finalizes with invalid_class = "harness-corruption" under a comment refusing the tempting escape hatch: two independent oracles disagreeing is independently proven harness corruption, never silently preferred one way. The benchmark doc states the motive in the sentence I'd put on a poster: a metric bug that under-reports duplicates, the classic favorable-measurement failure, is therefore mechanically build-breaking, in both directions. It is pinned by a counterexample test per invariant plus test_cross_check_fires_on_doctored_metric, which doctors a metric and asserts the run dies.

Oracle isolation has three layers of its own. An import-linter contract forbids arm modules from importing the oracle, fixtures, metrics, analysis or runner. A second forbids the engine from importing the bench at all, so the system under test has no code path to detect that it is being benchmarked. And a textual token scan fails any arm module that so much as mentions control_state, control_schedule, control_oob_create or control_log, which catches the string-based workarounds a type system waves through. The reference destination also carries an enrichment_quirk that stores normalized representations specifically to break digest-only attribution, run as a differential so the fallback path is exercised by an adversary rather than by hope.

Metric names are fixed, in metrics.py: duplicate_effect_rate, orphaned_effect_rate, lost_legitimate_effect_rate, false_suppression_rate, unresolved_ambiguous_rate, false_reconciliation_rate, human_review_rate, compensation_correctness. The first three are primary, and denominators come from the fixture or the oracle, never from arm-side counts. The non-numeric cell marks are pre-specified and never zero-filled: N/A, N/A-by-arm, NOT-DETECTABLE, expected-null, fixture-truth-only. Applicability is declared per arm rather than inferred, so an arm with no concept of ambiguity reports N/A-by-arm instead of a flattering zero. A table that renders a zero and a blank identically has lost the argument it was built to settle.

Statistics, the kill criterion, and the power I do not have

The statistics module is stdlib only, no scipy and no numpy, so the confirmatory analysis runs on a clean machine from the published wheel: Student-t via the regularized incomplete beta, paired t, TOST in the Schuirmann and Lakens paired form, Wilson intervals per Brown, Cai and DasGupta, an exact paired sign-flip permutation test with an honest floor of one over two-to-the-number-of-seeds, Holm-Bonferroni and Benjamini-Hochberg, and log relative risk with the Haldane-Anscombe correction. The module refuses opinions it isn't entitled to: no default margins or thresholds, every equivalence margin and alpha a required argument, and bench analyze --verdict exits 2 rather than defaulting them. A statistics library that ships a default alpha will eventually pick your alpha for you.

The unit of inference is the seed. Trials are nested inside seeds and are not independent replicates, which is the error I most expected to make and the one the preregistration is most careful about. Bootstrap percentile CIs are rejected for confirmatory use at this sample size, citing under-coverage in small samples and the arithmetic that n = 5 admits only 126 distinct resamples.

The equivalence margin is where a benchmark usually smuggles in its conclusion, so δ is derived from a named adopter profile rather than chosen. A reference adopter with a million irreversible C2 dispatches per year, an ambiguous-fault incidence of 0.005 per dispatch, 200 dollars per incident and a 10,000-dollar annual adoption cost breaks even at 10,000 / (1,000,000 × 0.005 × 200) = 0.01, so δ = 0.01 absolute. C1 gets its own margin, δ_C1 = 0.005, because payments-grade severity means a higher cost per incident and therefore a smaller tolerable difference. The worst-cell gate is 10 percentage points. All three are human freeze parameters, and no code path may choose them.

Which brings me to the credibility moment, where the preregistration says something about itself that I would rather it didn't. Write the per-seed paired-difference variance as a function of the pooled event rate p, the within-seed cross-arm correlation ρ, the number of trials per seed per arm N, and the seed-level heterogeneity standard deviation τ:

$$\sigma^2 = \frac{2p(1-p)(1-\rho)}{N} + \tau^2$$

Plug in the planned design. Four confirmatory cells at 60 trials each gives N = 240; take p = 0.05, ρ = 0.5, ten seeds, δ = 0.01 and τ = 0. TOST power at a true difference of zero comes out around 0.45. That is a coin flip, and the preregistration's own bolded words are that the equivalence leg is honestly underpowered at 60 trials per cell. At 160 per cell, N = 640 and power rises to roughly 0.96 at τ = 0, about 0.89 at τ = 0.005, and collapses for τ at or above 0.01, which no trial count fixes. And τ is deliberately not estimated from the development pilots, because using pilot data to pick the parameter that decides whether the test can fail is the move the whole document exists to prevent. So the honest reading of that equation is that the equivalence leg's power is unknown rather than 0.45.

Then the falsification rule, which is the moral center of the thing and is published verbatim:

the kill fires iff, for every primary metric on the confirmatory stratum, either (a) the preselected composite comparator is statistically equivalent to R — TOST against the §5.3 margin, never CI overlap — or (b) it is statistically better than R; and (c) no confirmatory cell shows the comparator worse than R by at least the worst-cell gate (§5.3) — a pooled equivalence coexisting with a large local regression is reported as inconclusive with a localized signal, not as a kill and not as a win. If the kill fires, Irrevon is unnecessary and the project is reframed as a teaching artifact (master doc §14). The benchmark is explicitly not designed so the system must win; clause (b) exists so a baseline that beats R kills at least as surely as one that ties it.

Clause (b) is the one I'd point at: a benchmark that only kills on a tie has an author who never considered losing. There is also a registered null, H0-C1, saying that on C1 destinations the engine is expected to show no advantage over native idempotency, reported as prominently as any win. Predicting your own irrelevance on a whole class of destinations, in writing, before running anything, is the cheapest credibility available and almost nobody buys it.

The refusals are mechanical rather than aspirational. CLI exit codes are 0 success, 1 unexpected, 2 usage, 3 declared outcome, 4 integrity refusal. bench run in confirmatory mode exits 4 unless both freeze registrations verify and the fixture manifest says frozen, under a docstring naming the failure mode it replaced: a directory's mere existence gates nothing. Freeze verification fails on a missing, unparseable or schema-invalid record, on the wrong stage, on any REQUIRED-HUMAN sentinel anywhere in the document, on a byte-hash mismatch against the preregistration, or on drift in any of six required analysis files. Drafts written by --draft-out are sentinelled by construction and can never verify. The module also records its own ceiling: hash binding plus structural attestation proves a registration is consistent with the tree and cannot prove who created it.

The invalid-run taxonomy removes the other selection lever. Harness-INVALID classes are only seed-mismatch, reset-verification-failed and harness-corruption, so arm-attributable failures are explicitly VALID and scored and an arm that crashes cannot be quietly reclassified as a bad run. Interrupted runs are finalized INVALID and retained, and after three attempts a slot is reported as unresolved-slot rather than dropped. Holdout generation refuses to write inside the repository at all, raising PermissionError("holdout artifacts never enter this repository"), and make bench-integrity fails on any artifact carrying split: "holdout", on the arm oracle-token scan, and on any document claiming freeze_status=frozen without a verifying Stage-B record, with the message "freezing is a human act". Three Make targets, benchmark-stage-b, sandbox-stage-m4 and release-gate, unconditionally echo REFUSED and exit 1, under a comment stating why they are unconditional: neither a credential nor the existence of a registration-shaped file can turn a no-op into a green benchmark run.

Two of those refusals are verifiable by absence right now, which is the cleanest kind of verification. docs/registrations/ does not exist in the tree, so both freeze verifications fail at their first branch and the exit-4 refusal is unconditionally live today. And bench/runs/ contains only a README, so no registered run artifact exists. A test pins that state, so the emptiness is asserted rather than merely current.

The disclosure in the preregistration's §0.2 is the most unusual thing in the repository, and I still find it uncomfortable. Synthetic engineering pilots have already happened against the reference destination, so the design cannot honestly be called pristine or described as preceding all observation, and the document says so about itself: calling the design "before every observation" or saying "no benchmark/fault trial has occurred" would therefore be false. It then leaves the ruling open rather than resolving it in its own favor. Whether the remaining separation suffices, Stage A still preceding every live-sandbox observation and Stage B still preceding every confirmatory one, or whether a design reset or independent review is required before freezing, sits explicitly in the review queue, and the last line is the one that governs me: no agent resolves that scientific-integrity ruling. Whatever counts those pilots produced are disclosed non-confirmatory harness-development evidence with no registered run artifact behind them, and they are not results.

The site volunteers counter-evidence against its own importance too. Irreversible actions are about 0.8% of nearly a million sampled tool calls, per Anthropic's February 2026 measurement, model-classified estimates with the caveat kept on the record. That is a high-severity, low-frequency problem, and the page says the consequence out loud: if you need a metered platform, "this isn't it." The novelty claim is narrowed to match, a literature-search inference rather than a priority claim: "None of the primitives are new. The combination and its measurement are."

The honest residue

Twelve things are wrong or missing, and a list of a system's real gaps written by the person who built it is worth more than any of its claims.

First, there is no identity-collision guard. The exact JCS preimage is persisted in effect_records.contract_canonical, which is precisely the material a guard would need, and nothing compares it. Today a SHA-256 collision would take the ON CONFLICT (effect_id) DO NOTHING path and be adjudicated as a re-registration, and if the effect class, adapter and branch happened to match, two genuinely distinct intents would merge silently. It is a preimage-strength assumption that is assumed rather than checked, and the check is roughly one if away. I know exactly what to write, which makes not having written it worse rather than better.

There is no version byte, salt or domain-separation prefix inside the identity hash. schema_version is a required field on the intent contract and is not part of the identity tuple, so a future change to the tuple shape would silently remap every ID in existence with no detectable discontinuity. Every hash would still validate, and every one would mean something new.

A blocking time.sleep(absence_reread_gap_s) sits inline in the confirmed-absence path, defaulting to 60 seconds. On a single-writer worker that stalls the loop for a minute per absent effect, while the worker docstring elsewhere describes the non-blocking design of revisiting work on later cycles. The protocol is right and the implementation of one clause of it is a sleep.

_park_or_query takes an adapter, a config and an actor and uses none of them. It unconditionally parks. Harmless, and misleading to anyone reading the signature to learn what the function decides.

The advisory lock keys on hashtext(), an undocumented internal Postgres function whose stability across major versions is not contractual. It works, and it works on a promise nobody made me.

Single writer only. Multi-writer is designed and not implemented, as above.

Both provider adapters, Stripe C1 and EasyPost C2, are credential-gated drafts never once live-called, so every behavioral claim here rests on a synthetic destination whose behavior I chose. That is the single largest discount a reader should apply.

The differential guard covers five of the eight metrics. unresolved_ambiguous_rate, human_review_rate and compensation_correctness have no independent cross-check, so the guard I called the best thing here protects the primary metrics and leaves three secondary ones on trust.

Four harness document formats, history, conformance, comparison and verdict, are declared in the code with no entry in _SCHEMA_BY_FORMAT and no schema file, so they are never validated. The second oracle's own output artifact is unschematized, which is a funny place to find the gap.

false_suppression_rate is probably unexercisable on the current matrix. It needs an arm outcome of suppressed on a trial the fixture labels legitimate, and the two faults where an arm legitimately suppresses, stale-authorization and branch-cancellation, are exactly the ones the fixture builder marks legitimate = False. The conventional arm's only other path to suppressed sits behind a branch its own comment calls defensive and impossible on a first send. The metric has a definition, a name, a cross-check, and no route to a nonzero numerator.

The committed development matrix is far narrower than the preregistered one: 9 cells, 2 replicates, 4 trials each with faults on two of the four, all against the synthetic destination, and the manifest I checked lists 18 workloads at freeze_status: unfrozen-dev. The preregistered design is 7 faults by 4 effect classes by 3 tiers at 60 or 160 trials per cell across 10 seeds on a live sandbox, and the confirmatory stratum has no fixtures in the repository at all. The distance between those two matrices is the distance between a working harness and a scientific instrument, and it is mostly not code.

And ADR-0030 through ADR-0034, the decisions authorizing the entire benchmark design, are still status proposed rather than accepted, as is ADR-0020, the one authorizing the identity derivation everything else rests on. The repository states the consequence itself, in its own status page: "Repository automation and this status page never convert implementation into ratification." Shipping code is not the same act as agreeing the design was right, and I built the process so that I cannot confuse the two even when it would be convenient.

What happens next

The next thing that matters is not a feature. It is a Stage-A freeze a person has to perform, on parameters no code path may choose, on a document that currently says in capital letters that nothing in it is frozen. Until then the exit-4 refusal stands, and I would rather find out I built something unnecessary than never build the instrument that could have told me.