8.3 KiB
Per-case findings (drop-in for the Worked Examples section)
These are drafted to sit directly under each existing ### COVID / ### LHC black holes
/ ### Eggs heading, above the graph and transcript links. Each is one run of
crux-research followed by the full adversarial-graph-review loop, with minimal
prompting. Numbers are pulled from the session transcripts in this directory.
COVID — reproducing the 23-orders-of-magnitude spread
The run reconstructed the early-2024 Rootclaim debate (Wilf vs. Miller, judged for zoonosis) as a single shared 47-node evidence tree — raw facts only, hung under one lab-origin spine — rated by eight perspectives: the six analysts Scott Alexander compares (Rootclaim, Miller, both judges, Scott, Daniel Filan) plus Michael Weissman and the executing model's own puppet. The headline test the prompt set was whether the famous ~23-order-of-magnitude spread across independent Bayesian analyses would emerge as a propagated output rather than an asserted one. It did: the six analysts rating the same tree propagated to root odds spanning from Miller at P(lab) ≈ 10⁻²⁰ to Rootclaim at ≈ 0.999 — about 23 orders of magnitude, with every perspective landing within ~1 OOM of its published number. Removing Miller collapses the spread to ~6 OOM, matching Scott's "~7 without Peter."
Three things the map surfaced that prose summaries of the debate do not:
- The entire disagreement is manufactured at essentially one node. Every analyst agrees the Huanan-market early-case clustering exists; they disagree by ~44 dB (−50 dB for Miller, −6 dB for Rootclaim) on what it is worth. The debate is a likelihood dispute over agreed facts sitting on a shared prior that favors nature — not an empirical dispute.
- Miller's ~10⁻²⁰ is structurally exposed as over-multiplication. The tree shows it is built by stacking a −50 dB market factor, −48 dB two-introduction, −40 dB no-backbone and −35 dB base-rate factor as if independent — the exact correlated- evidence failure the independence intermediates are designed to catch. This vindicates Scott's "trolling" bracket node by node rather than by assertion.
- The tooling's own crux metric has a blind spot.
get_disagreementskeys on confidence divergence, but the market crux lives in edge-weight divergence — so the built-in metric under-ranks the single most important crux. That is a genuine, honestly reported limitation of the current Assessment layer.
The adversarial review then hardened the graph over multiple rounds. It caught four real defects — a sign/text contradiction that had both lab-side analysts propagating too hot, a duplicated datum (the Sept-2019 database withdrawal counted under two clusters), Miller's incoherent null grandchild weights, and a correlated-timing over-count — plus a shared- source correlation between the market and two-introduction clusters (both trace to Worobey/Pekar). Removing the correlated over-counts honestly shrank the reported spread from ~23 to ~21 OOM and moved Rootclaim from +31 dB to +27 dB (554:1), almost exactly its in-debate 533:1 figure. Two honest caveats stand: the judges' and Filan's ratings are inferred from their written decisions, not stated by them, and the shrinkage shows the raw 23 was itself slightly inflated by correlation.
LHC black holes — separating the physics from the epistemics
The prompt deliberately split the question into two hypotheses: H1, the physics claim that micro black holes posed no meaningful risk, and H2, the epistemic claim that declaring the case "closed" was justified at the certainty expressed. A 105-agent deep-research workflow (every finding adversarially verified against primary sources) grounded a 34-node graph rated by five perspectives: the executing model, the LSAG/CERN 2008 position, the Rössler/Wagner litigants, the x-risk epistemologists (Ord, Hillerbrand & Sandberg), and a historical-precedent lens built from Trinity (LA-602) and the RHIC strangelet review.
The result the map makes legible: the disagreement was never about black holes. Every perspective that engages the evidence converges above 0.90 on the physics (own posterior 0.97; even the safety case's sharpest published critics sit ~0.90), yet they diverge by up to 0.80 on whether the institutional certainty was warranted (own posterior on H2: 0.45). The physics rests on three layered arguments — production needs speculative extra dimensions; those same theories predict evaporation in ~10⁻²⁷ s; and cosmic rays have run harsher collisions on white dwarfs and neutron stars for billions of years — but the epistemic case has real cracks: the original cosmic-ray argument had a genuine gap (collider products can be slow and trapped) patched only months before startup, the review was CERN reviewing CERN, and the courts dismissed every lawsuit on standing, never on the merits.
The adversarial loop is the most instructive of the three cases because it did not
cleanly converge — and reports that honestly. Severity fell monotonically (round 1: two
P1 findings; round 3: one P1; rounds 4–5: P3 only), and the round-5 referee moved to
"accept with optional minor revisions" after hand-verifying the entire load-bearing spine,
but no round returned zero valid findings, so by the skill's own definition this is a
cap-out, not a clean bill of health. The correlated-evidence correction was dramatic:
the model's propagated root fell 0.981 → 0.939 → 0.892 across the repairs, closing on its
stated 0.85 — the round-1 referee's verdict was that the original 0.98 was "the sound of
correlated evidence being summed as if independent." The loop also caught its own
adversaries: two findings were rejected as confabulation after the model re-read the
cited edges in the live graph and found the claimed defects simply weren't there (this is
the hallucination-based check working as intended). Finally, the run hit a real engine
limitation — positively-weighted supports can't pull a node below the 50% prior, and
there's no way to gate one branch's contribution on another — and filed it as issue #39
(a proposed gate edge relation), which is itself an example of the workflow compounding
into product improvements.
Eggs — the everyday question, and a clean convergence
The eggs case is the "ordinary question" of the three: no adversarial debate to reconstruct, so the hard part is deciding what is even worth answering and how we can know it. The run built a graph around a single proposition ("for the general adult population, are eggs net-good / net-bad / fine in moderation") and rated it across four perspectives. Unlike LHC, this one converged in round 5 (of a 5-round cap): the final root sits at a stated 0.85, conservatively below the 0.881 it propagates to, and the round-5 adversary independently reproduced the propagation arithmetic by hand before accepting.
What makes it a good Assessment-layer demonstration is that every finding across rounds 1–4 was valid — the rejection ledger stayed empty until the final round — and most of them were correlated-evidence catches, the contest's named criterion:
- The root started as a four-way XOR question and was reframed into a single proposition (a Structure fix).
- Two studies were double-counted along two paths each; the duplicate edges were archived so each study pays out once.
- Three overlapping CVD meta-analyses (Frontiers / Godos / Zhao, which share underlying cohorts) were down-weighted from 8/7/6 dB to 6/2/2 dB and cross-annotated with shared-source undercuts — textbook "correlated evidence treated as independent."
- The LDL-cholesterol surrogate endpoint was capped so a biomarker couldn't stand in for a hard outcome at full strength.
- An incoherent node (H2 propagating to 0.946 while stated at 0.40) was repaired by adding a propagating attack for the surrogate-to-outcome gap, bringing it back to 0.46.
The run also shows disciplined restraint: one round-5 recommendation was deliberately not applied because its literal remedy (converting an attack to an undercut) would have made the discount stop propagating and re-inflated H2 — undoing an earlier round's fix. That is the builder correctly refusing a plausible-but-wrong suggestion, documented rather than silently dropped.