# sites/system-one/data

Generated by `tools/extract_system_one.py` on 2026-10-16T14:46:25+00:00. Every file is

```json
{"meta": {"id", "title", "sources": [{"repo", "path", "sha256"}], "extracted_at", "notes": []}, "data": ...}
```

Numbers keep full precision. Rates are fractions (0.00384502), not percents, unless a field name says `_pct`.
Where the old site printed two numbers for one quantity, both are kept with a `variant` label:

| quantity | variant A | variant B |
|---|---|---|
| recommended-stack block-only F1 (S2) | 0.737728 short-circuit (what every Deploy panel measures) | 0.751734 escalate-on-confirm (the recommendation; policy re-analysis) |
| recommended stack false blocks | 14 (FPR 0.00414, stack incl. judge) | 13 (FPR 0.00384502, the board's shared budget = OpenJev alone) |
| DiffusionGemma S2 latency p50 | 3.66 s (Q3 run, Deploy latency table) | 7.35 s (Q2 run, Risks comparison table) |
| SecJudge shipped block-only F1 | 0.20724154 (board; severity-head variant) | 0.20893372 (isattack variant scorecard) |

Dropped by owner decision: the 0.12854 S3 stand-in / escalate-on-confirm row (nowhere in these files).

## Chapter files (index.html)

| file | chapter | shape of `data` |
|---|---|---|
| `verdict.json` | 1 The answer | `recommended_stack`, `headline.tiles[]` (before/after/after_escalate for F1, FPR, judge traffic), `headline.stack_short_circuit` / `stack_escalate_on_confirm`, `best_single_model`, `production_weighted` |
| `leaderboard.json` | 2 Which model | `corpus` (436 positives, 3,381 negatives, budget 13 FP) and `models[]`: rank, model, licence, parameters{parameters,basis,band}, f1/precision/recall + `_ci95`, true_blocks, false_blocks, auc, shipped_block_f1, scorecard{...}, `note`. `curve` is null for every model (see Gaps): the board is shown at the fixed shared budget, no slider. |
| `cascade.json` | 3 Cascade (Sankey) | per corpus `s2`/`s3`: `rules_tier`, `variants{one_sided, two_sided_0.05 … 0.30}` each with `nodes`, `links[]` (`source`,`target`,`value`; tier→disposition links are the pooled measured mix, `pooled: true`), `decided_by`, `final_mix`, `tiles{block_f1, block_fpr, judge_rate, review_rate, …}`, `three_way_confusion`; plus `judge_alone`, `rules_small_no_judge`, `escalate_on_confirm_two_sided_0.30`, `tier_orders` |
| `policies.json` | 3 Broad ↔ Production grouped bars | `policies[]`: corpus, policy (judge_alone / rules_small / one_sided / two_sided_0.30 / escalate variant), block_f1, block_fpr, confirm_rate, judge_rate, block_confusion, any_f1 |
| `pareto.json` | 4 Cost | `judge_price{s2,s3}` (usd per case/call), `policies[]` (19 per corpus): small_model, policy, judge_rate, judge_usd_per_1000_cases, block_f1, block_fpr, standing, published{…} |
| `calculator.json` | 4 Calculator | the embedded `calc` + `thr` series the existing calculator reads (per threshold f1/leak/llm/review for both corpora), `judge_price`, defaults |
| `per_event.json` | 4 Per call vs per session | `rows[]` per corpus × model: per_event_benign_block_rate, per_trajectory_benign_block_rate, ratio, benign_events_blocked, benign_sessions_killed, unsafe_recall_block_only |
| `faults.json` | 5 Fail-open | `groups[]` (outcome group → scenarios), `scenarios[]` (29: dispositions per lens, error codes), `gate_scope`, `tamper` (11 tampers, 5 accepted), `judge_faults` (12 modes) |
| `benign_fpr.json` | 5 Benign coding traffic | `rows[]` gate × model × context with per_event_fpr and wilson95; `sweeps` (full Lane A/B threshold sweeps per candidate) |
| `repeat.json` | 5 Repeatability | `rows[]` per model: corpus_wide / flagged_only flip rates and counts, `jev_deep_repeatability` (16 replays) |
| `intent.json` | 6 Intent | `cells[]` (32: backend × C0/C7 × case/event × block/any with proof_backed, agentdojo_prior and CIs), `grade_a_recall[]`, `summary`, `claims` |
| `caveats.json` | 7 | `rows[]` merged from the three caveat tables (page, assumption, rests_on, breaks_it, status) |
| `recommendations.json` | 7 | `recommendations[]` (12), `fixes[]` (6), `merged[]` (15 unique, original wording; `same_as` links duplicates) |

## Appendix files (appendix.html)

| file | shape of `data` |
|---|---|
| `corpora.json` | `s2`, `s3` (cases, decisions, grades, scorable, prevalence, all-allow accuracy, block-everything F1, grade-A share), `stages[]` funnel, `grades[]` definitions, `screening_pilot` |
| `questions.json` | `format_in_cascade[]`, `jev_all_formats[]`, `instructions[]`, `question_variants[]`, `contexts[]`, `protocol` (questions-v2, contexts-v1 verbatim), `fail_open_audit` |
| `source_recall.json` | `rows[]` model × source: positives, blocked (at the budget), recall, recall_wilson95 |
| `calibration.json` | `rows[]` model × bucket: cases, positives, mean_predicted, observed_rate, observed_rate_wilson95 |
| `confusion.json` | `panels[]` per model: 3×3 truth × decided counts and shares |
| `latency.json` | `rows[]` per run: latency_ms{p50,p95,p99,max,mean,count}, spend_usd, input_tokens, errors (both DiffusionGemma S2 runs); `flex_tier` |
| `adjudication.json` | agreement table, pairwise agreement, bias check, routing cost, pattern classes, case skeletons, `disagreement_classes` |
| `explorer.json` | the 2,133-row disagreement queue verbatim (fields, index tables, rows) |
| `ledger.json` | `experiments[]` (23), `restated_figures[]` (17), s1 upload inventory, model parity |
| `sources.json` | 13 sources with licence/grades/url/host, code paths, seeds, revisions, HF dataset ids, GitHub pin |
| `glossary.json` | `terms[]` (glossary table + every tooltip definition) |
| `_parity.json` | for each of the 761 published figure keys: published value, reproduced value, status (matched / mismatched / missing + reason) |

## Parity

694 matched, 1 mismatched, 0 missing of 761 published figure keys (see `_parity.json`).

## Gaps

- `outputs/curves/curves-s2.json` (per-model PR curves `arms.<arm>.pr`, `at_budget`, `calibration.buckets`, `per_source`) is in neither HF dataset, and no file under stages/s2/ or stages/jev-parity/ carries those keys. By owner decision (design/decisions.md #7) the curves are NOT recomputed or approximated: `leaderboard.json` carries only each model's operating point at the fixed shared budget (P/R/F1 with intervals, true/false blocks, AUC), read from the old pages' data tables (board, fig-cleveland, fig-pr), and `calibration.json` / `source_recall.json` come from the fig-calibration and fig-source-recall tables. Consequence for the page: the budget slider and the 'every other budget' PR small multiples are dropped; the leaderboard is shown at 13 false blocks only. What is missing for a slider: every model's full (threshold, tp, fp) sequence. The recomputation path that rebuilds it from the published S2 prediction files is kept in the script behind `RECOMPUTE_CURVES = False`; it reproduced every published at-budget count, AUC and calibration bucket exactly for the 11 models whose files are published (open-jev-qwen-27b's file is absent from both datasets, see `stages/s2/h200-27b/README.md`).
- The S3 stand-in / escalate-on-confirm figure 0.12854 is dropped everywhere (design/decisions.md #4); the recommended stack keeps both S2 variants with labels: 0.73773 short-circuit at 14 false blocks (FPR 0.00414) and 0.75173 escalate-on-confirm, against the board's budget of 13 (FPR 0.00384502).
- F1 bootstrap intervals (2,000 family resamples) are carried from the Space: the seed is published (741983) but the resampling code is not in the dataset.
- The realdet scorecards for OpenJev S3 and DiffusionGemma S2/S3 (`outputs/deterministic-real/realdet-s3-openjev.json`, `realdet-s2-diffgemma.json`, `realdet-s3-diffgemma.json`) are absent; their lenses come from `stages/deterministic-real/report.json` ('after' records), which carries every block/detect metric and rate but no three-way confusion, so the S3 Sankey final mix is derived from review_rate and the block confusion.
- The 200-case pilot arms (instruction and context variants, the Von 1.0.1 and OpenJev pilots) come from `stages/s1-n1000/runs/*/metrics.json` and `stages/s1-n1000/metrics-von-questions.json`; the context run's manifest carries no `complete: true`, which the old site discloses as 'not settled by this site's own gate'.
- Not reproduced, by design (keys with a reason in `_parity.json`): the prefix-cache / prompt-padding experiment (`cache.*`, `px.*`, `pd.*`: gpu-host-evidence files not published), the cross-replay `cross.*` statistic (undocumented; neither cross-model nor case-level recounts of predictions/repeat reproduce it), the provider-spend roll-up (`spend.*`: 325 manifests, most unpublished; the three headline totals are carried in latency.json), the context-benign catalog (`cbc.*`: file not published), `tb.cal.*.below` (formula not recoverable), `calc.dpcscen`, `pilot.multiaxis`, and the dropped S3 stand-in keys. None of these is a chart value in the plan.
- One mismatch: `x.kev.allow` (kev-9b answers allow on 96.9% of cases as shipped). The case-level histogram in rescoring-remine/kev-s2/comparison-s2.json gives 3,708 of 3,817 = 97.14%; no population tried (all cases, benign cases, events) gives 96.9%.
- The flex service-tier comparison (200 cases) is carried from decide.html; the outputs/flex scorecards are not in the dataset.
