Wiki · Evidence & Verdicts
Evaluation Harness Guide
How to read this page
Three ways to read this page. Precise is the document itself, exactly as it is written in the repository. Plain and Clear were written for this website to help you meet that document — they are about it. They are not it, and they are not evidence.
Eighty-seven dated pages: receipts, pre-registrations, handoffs, validation records and review verdicts. A receipt is written at the moment a piece of work was checked. It names what was claimed, the commit and the seed, what was actually run, and the outcome in one of a small set of controlled words. Then it names what the work did not achieve. That last part is what makes it a receipt rather than an announcement. A pre-registration is the same discipline run in advance: the conditions that would count as a pass and the conditions that would falsify the claim are written down before the run, so neither can be adjusted once the numbers arrive.
That is why so many small dated stubs are an audit trail rather than noise. No one of them is meant to be a good read. The value is in the sequence and in the dates, because you can watch a prediction be registered, then the run happen, then the verdict land — sometimes against the prediction. Pages here record a falsified result, a rejected fix, a retracted overclaim, and a green receipt that turned out not to be reproducible from the commit that carried it. A record that carried only successes would be worth a good deal less than this one.
A gentle way in is to read a pre-registration first, so the shape becomes familiar, then a result page, then one of the corrections. This section sits off the main navigation on purpose: it is the record you check the rest of the site against, not the place to begin.
What it is not: documentation, and not a summary. Nothing here has been tidied in hindsight. Every entry reads as of its date, a later entry may overturn an earlier one, and the presence of a page is not a claim that its result stood.
Your browser cannot switch reading levels, so the document itself is shown.
Precise — the source document
This is the document. Rendered from the repository at the commit above, with nothing rewritten for the web. A gate re-renders it on every deploy and fails the build if a single byte differs.
SP.Eval performs policy consequence evaluation against the viability priors.
None of these metrics are surfaced to the agent (Invariant #15) — they exist for
reports and regression tests only.
Episode metrics (SP.Eval.episode_metrics/1)
| metric | meaning |
|---|---|
survived_ticks, halted, final_envelope |
survival & viability outcome |
final_stage, final_organs |
morphological development reached |
mean_risk, max_risk, mean_prior_divergence |
viability trajectory (consequence) |
sensor_modalities |
distinct sensory signal types ever emitted (sensor utilisation) |
morphology_utilisation |
distinct appendage-gated actions actually used |
ungated_attempts, decoded_failures |
interface misuse accounting |
structures_built, structure_kinds |
niche construction / infrastructure complexity |
expansions, region_count, regime_novelty |
open-endedness / world expansion |
Ablation presets (SP.Eval.preset_genome/2, preset_opts/2)
| preset | what it removes |
|---|---|
:full |
nothing (full appendage + sense ladder) |
:no_development |
development disabled (body stays the seed morphology) |
:minimal_senses |
appendage ladder only; no senses past the seed pair |
:minimal_appendages |
sense ladder only; no appendages |
:no_hidden_layers |
appendages + L0/L1 senses only (no tomography/spectral/seam/meta) |
SP.Eval.ablation_suite(seeds, opts) runs all presets across seeds and returns
per-preset aggregate means plus pairwise deltas vs :full.
Structural (deterministic) evidence
SP.Eval.layer_visibility(seed, omit) returns %{with, without} opaque-channel
counts for a full body vs one missing omit (and its dependents). This is
non-statistical evidence that each sense gates a discoverability layer
(Invariant #8). Used by SP.EvalTest and SP.InvariantsTest.
Running the harness
SP.Eval.run_episode(seed: 101, agent: SP.Baselines.Homeostatic, max_ticks: 400)
|> SP.Eval.episode_metrics()
SP.Eval.ablation_suite([201, 202, 203], max_ticks: 200)
Or capture the full evidence bundle: mix run scripts/evidence.exs.
Lineage / evolution loop (composition)
A cross-episode evolutionary controller is a thin loop:
# pseudo: select by world viability, then mutate/recombine
genomes
|> Enum.map(fn g -> {g, SP.Eval.run_episode(genome: g, ...) |> SP.Eval.episode_metrics()} end)
|> select_survivors() # by survived_ticks / structures_built / expansions
|> reproduce() # SP.Genome.mutate / recombine (always repaired)
Selection pressure is the world itself — there is no fitness scalar fed to the agent.
Interpreting difficulty
The reference batch (config/seeds.json) is calibrated so survival is neither
trivial nor impossible: random baselines die well before the horizon on most
seeds, while sense-using agents survive markedly longer. See
morphology and
sensory ablation reports for numbers.
sha256 bfd33d95fa2de649 — of the original file, so what was ingested stays checkable.
Plain — written for this website, not the source document
Scoring a simulated run after it has happened is what this part of the codebase does, and the guide's first line is the most important one: none of these scores are shown to the agent while it runs. They exist for reports and for catching regressions. The page then lists what gets scored, the cut-down versions of the body used for comparison, the commands to run it all, and how the reference set of seeds was tuned so that surviving is neither trivial nor impossible.
Plain · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is bfd33d95fa2de649
Clear — written for this website, not the source document
A reference page for the scoring side of a simulation, written for someone who is going to run it. Its opening line is a boundary rather than a feature: the measures here are never fed back to the agent, and exist for reports and regression tests.
First comes a table of per-episode measures, grouped by what they are about. How long the run lasted and whether it stayed viable. How far the body developed. The shape of the risk trajectory. How much of the sensing, and how much of the body, actually got used. How often the interface was misused. What was built, and how far the world expanded.
Then the ablation presets: cut-down configurations that remove development, or the senses, or the appendages, or the deeper sensing layers. A suite runs all of them across a list of seeds and reports each preset's averages plus the gap against the full body. Beside that sits a non-statistical check that counts how many channels a full body can see compared with one missing a sense, offered as structural evidence rather than statistical evidence.
A short section gives the commands to run one episode, run the suite, or capture the whole bundle. Another sketches how an evolutionary loop would sit on top as a thin wrapper, and makes a point of saying that no fitness number is handed to the agent; the pressure is the world itself. The closing section explains how the reference batch of seeds was calibrated, and points at two ablation reports for the numbers rather than restating them here.
Clear · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is bfd33d95fa2de649