03 / bench

A panel that publishes how little separation it has.

Every case is put to several models that never see one another's reasoning. Which models sat is part of the record of that case, because the same procedure run by a different set of models produces materially different outcomes.

The panel is composed of independently prompted, separately operated models. What VERDX guarantees is that the seats do not see one another's opinions and run as separate calls. It does not guarantee that their errors are uncorrelated, and correlated error across models cannot be eliminated.

Rulings are auditable, not repeatable: the evidence hash, criteria hash, prompt template hash, model ids, ruling text and every minority opinion are recorded, so anyone can re-examine a ruling against the same inputs. Inference servers batch dynamically, so an identical call is not guaranteed to return identical text.

Minority votes

Vote-level, not case-level. The theoretical ceiling is set by panel size: with a panel of N, at most floor((N-1)/2)/N of all votes can be in the minority. Compare the observed value against the ceiling before calling it high or low, and read the per-model breakdown -- one seat holding most of the minority votes means that seat is aligned differently, which is a signal to revisit panel composition.

Not decided

Two different events arrive under one verdict label and are reported apart here. agreed_undecidable is a panel that agreed the case cannot be decided on this evidence -- a finding about the dispute, and the outcome the panel is asked to choose rather than guess. no_majority is a panel that never converged: a tie, or no outcome holding a majority. That is a failure of the panel, not a finding about the case. Summed into one number, every convergence failure would sit inside a legitimate finding and the rate would look like considered restraint.

Insufficient evidence

n/a (n=0)

Ambiguous standard

n/a (n=0)

Out of scope

n/a (n=0)

Out of scope

What this system does not do.

• We cannot eliminate correlated error across models.

  Frontier models share training data and failure modes. Prompting them separately raises the number of seats, not the number of independent opinions, and a panel that agrees is not thereby right.

• We cannot fully prevent adversarial evidence crafting.

  Evidence is submitted by the parties to the dispute, which gives an adversary write access to the panel's input. Evidence is rendered as data inside a structured envelope, hashed at intake, and flagged when injection detection trips — none of that is the same as stopping it.

• VERDX rulings are produced by models only; no human review layer is provided.

  An appeal convenes a wider panel of models. It does not convene people. Nothing here is a court decision and nothing here is legal advice.