Spec — Binocular true-signal vision, the perceptual layer

The Colony & the Method · docs/specs/sensorium.md @ 44baf03d5041 (gen2-runtime) — opens the published snapshot ac338733bbba

Three ways to read this page. Precise is the document itself, exactly as it is written in the repository. Plain and Clear were written for this website to help you meet that document — they are about it. They are not it, and they are not evidence.

Eighty-four pages about the colony. Each agent is an Elixir process holding a generative model and doing inference, attached to a body that logs into a Minecraft world as an ordinary player. Around that sit the broadcast suite that films them and the runbooks that keep the whole thing running. There are typed specifications for each organ of the model, plus the world and genome specs. There are also the adversarial review personas used to attack a proposed change before it ships.

It is for the reader curious how a running system is put together and how it is held to account. The accountability half is the more distinctive. There is a lab protocol governing evidence and attribution, and a claim fence that restricts the vocabulary a claim is allowed to use. There is a public gate log. And there is a standing invitation to reproduce any verdict from the commit and the seed named in its receipt.

Start with the public read, then the lab protocol, then the falsification invitation. If you want the mathematics rather than the operations, go straight to the typed organ specs.

What it is not: a description of a mind, and not all one kind of document. A large part of this corpus is design and planning — specs marked as proposed rather than applied, organs designed but not built, plans that were later superseded — and each page states which it is. A specification is not a running system, and these pages are careful about the difference; the reader should be too. Eight documents were withheld from publication because they describe private infrastructure.

PlainClearPrecise

Your browser cannot switch reading levels, so the document itself is shown.

This is the document. Rendered from the repository at the commit above, with nothing rewritten for the web. A gate re-renders it on every deploy and fails the build if a single byte differs.

Part II of A4. Reads with generative_model.md (backbone) + curriculum_removal.md (Part I). Design-only; ship gate = formal /lab-team-review + owner go-ahead. Corrections folded from docs/receipts/a4_lab_team_review.md. RED-B runs only AFTER RED-A has a verdict (one-cure-at-a-time).

Goal

Replace the hand-crafted vision_index block-name bins (mc_codec.ex:106-117 — the tampering the cookbook forbids) with a full human binocular eye delivering TRUE signals through the existing learned-scene port. Rule: "swap only A" — the categorical engine (model.ex/factors.ex) is untouched.

II.1 The full human eye apparatus (per eye — real retina physics, Python BODY zone)

Reuse the sensorium Eye pipeline (uni-sensorium/src/uni_sensorium/body/eye.py), each stage a real biological mechanism, arithmetic-only (three-zones purity, AST-guarded: no fft/trig-synth/clock/RNG/autodiff): retinal contrast gain-control (center-surround; ADR-0011, 0.20→1.00), saccade + accommodation (_fixate, ADR-0010, fixation load-bearing +0.269), aperture integration (8×8→64 receptor Signal), and active mental rotation (torsion ODE, the Shepard-Metzler act — passive invariant features were measured NEGATIVE and kept OFF; honor that). Output per eye: Signal ∈ R^64, ≥0, Σ=1 (Hellinger), stays FEP-side.

II.2 Binocular — two eyes + cortex-side stereo fusion (new; no stereo code exists yet)

Two Eye instances (L/R) from two POV frames (the wire supports N streams/UNI). Stereo/disparity fusion is computed cortex-side in Python (where pixels legally live). Honors two real retinas + stereopsis while keeping the blanket clean.

II.3 StateSpace — discrete states only; `:depth` IDENTIFIABILITY (blocker #1a)

Faithful to V1→ventral("what")/dorsal("where"), the brain gains two discrete factors, both learned:

:scene — the "what" (ventral/IT): the learned scene-state prototype-id from the fused percept. Port EXISTS (mc_codec.ex:52-54); A_scene 12×12 categorical, Dirichlet-learned.
:depth — the "stereo where" (dorsal): a NEW categorical factor over depth/disparity states.
(Deferred) :gaze — active "where to look" as an EFE saccade policy; needs the motor loop; next rung.

Identifiability (blocker #1a — load-bearing). genome.ex:89-92 documents that a single-modality no==ns uniform-A factor is non-identifiable (qs stuck uniform, counts smear, A never leaves uniform) unless seeded with init_a: :diagonal. The old spec declared :depth (and :scene) with uniform A — a stuck-uniform :depth is an inert smuggled feature, and it would make the load-bearing G-VIS-4 stereo ablation FALSIFY for the WRONG reason (non-identifiability, not decorative stereo). Fix: declare :depth with init_a: :diagonal (a weak diagonal prior that only breaks the uniform symmetry so the factor is identifiable; online learning still refines it — exactly the motor-cortex remedy), and audit :scene for the same. Additionally, pre-register an activation-gate probe that A_depth/A_scene LEAVE uniform and qs concentrates on held-out MC frames — before any RED-B behavioural tick.

II.4 ObservationChannels — cardinality ENFORCED, not clamped (blocker #12)

Wire: two POV ports → two vision_forward streams → cortex (two patch-Markov + a stereo fuser) → <UNI_PERCEPT_DIR>/<user>.json (scene_state + depth_state) → body.js σ channels → bridge.ex parse_sense → MCCodec.outcome(:scene|:depth). The cardinality contract must RAISE, not clamp (blocker #12). Today mc_codec.ex:54 outcome(:scene,s) merely idx(...)-CLAMPS an out-of-range cortex index onto the top bin — a cortex/genome n_states mismatch silently folds indices ≥12 onto bin 11 and corrupts the A_scene Dirichlet counts with no error. Fix: add @depth_states + a def depth_states accessor (mirroring genome.ex:42-43), a :depth codec clause mirroring outcome(:scene,s), and a raise-on-mismatch assertion at the bridge/startup boundary for BOTH :scene and :depth (@scene_states == cortex n_states).

II.5 `:depth` bin-EDGE provenance — the central honesty fix (blocker #1b)

RED-B's premise is that hand-crafted bins are forbidden. Dirichlet-learning A_depth learns the MEANING of a bin by co-occurrence; it does NOT learn the bin EDGES. If the depth/disparity bins are fixed designer thresholds, they are categorically identical to the forbidden vision_index thresholds — hand-authored discretization laundered through a learned A. :depth bin EDGES must be UNSUPERVISED-clustered cortex-side (like the patch-Markov :scene prototype clustering — the reason :scene escapes the sin), so the boundaries are learned from the disparity statistics, not authored. (Alternative, only if justified: fixed edges as retina-level receptor quantization — NOT semantic labels — with a pre-registered ablation showing the boundary placement is non-load-bearing.) Until this is stated, the binocular organ's central honesty claim is unproven and G-VIS-4 tests the wrong object. A_scene/A_depth otherwise learn online; no labels reach the mind (three-zones: labels only in school/tests).

II.6 Seams — motor-tail reindex (blocker #3) + bridge wire + byte-identity (blocker #14)

Motor tail fence (blocker #3 — silent-corruption; MUST land SAME PR as :depth). motor_config (mc.ex:135-136) does obs |> Enum.take(-5) and destructures [aim,reach,contact,dig,motion], asserting they "are always the final 5 factors" (mc.ex:133-134). That invariant is ALREADY a lie for any :motor_cortex+:metabolism genome (energy/satiety are appended after the motor block, genome.ex:109-111); it only latently survives because no live lineage combines them (motor_step gated at mc.ex:126). :depth is the THIRD append-last organ on the crack. Fix: reindex motor_config BY NAME using the active_modalities index pattern strategist_config already uses (mc.ex:425), not by tail position + a regression test that a motor+metabolism (and motor+depth) lineage still reads aim/reach/contact/dig/motion.
Bridge wire (blocker #14): append :depth at a NEW FIXED rest[] slot AFTER the motor block (rest[7..11]) and the continuous channels (rest[12..14]) — e.g. rest[15] — so default/motor/vision bodies that omit it degrade to "0"; do NOT reuse/shift the fixed scene/motor positions (bridge.ex:31-34). Add a bridge.ex parse_sense decoder-order test proving a :binocular_cortex body's motor channels are byte-unshifted and a non-vision body degrades gracefully.
Gating + byte-identity: add :depth under a :binocular_cortex organ (requires :sight_cortex); :scene already exists via vision_primary/0. Default 12-factor lineage untouched ⇒ byte-identity green; extend decider_byte_identity to a 13/14-factor vision lineage. Express path uncapped (@factor_cap 12 gates only runtime add_factor); all factors share nu (factors.ex:59-63).

II.7 RED-B — paired, single-variable, UN-BUNDLED, numeric (blockers #5, #6)

Pin curriculum: IDENTICAL in both arms (blocker #5) — the between-arm variable is the vision organ ONLY (not curriculum-source). novelty_gain equal.
Un-bundle the two changes (blocker #5). Treatment swaps vision_index→true-pixel :scene AND adds :depth = two changes. Pre-register a monocular true-pixel :scene-only intermediate arm (or explicitly designate the G-VIS-4 monocular condition as it), so true-signal (arm2−control) and binocular/:depth (arm3−arm2) are each independently attributable. Report per-factor novelty-term contribution / match effective cardinality so a plateau-break is not confounded by epistemic-surface inflation (two new high-cardinality learned factors carry more unlearned Dirichlet counts than the bins control).
Offline activation gate BEFORE live T0 (blocker #6). No stereo code exists yet, so RED-B's numerals can't come from any existing receipt. Require a stereo-cortex OFFLINE held-out gate (G-VIS-0 signal purity + a NEW stereo free-energy-drop held-out receipt) green with numerals pinned before RED-B live T0.
Load-bearing ablations = pre-registered NUMERALS (blocker #6), each a FALSIFIES (replace "collapses"): G-VIS-1 fixation-off Δrecognition ≤ pinned floor; G-VIS-2 gain-off low-contrast ≤ floor; G-VIS-3 passive (no mental-rotation) rotation-AUC ≤ floor; G-VIS-4 monocular/stereo-off depth-discrimination AUC ≤ floor (the binocular claim dies if monocular matches — valid only after :depth is identifiable, §II.3); G-VIS-5 no-regression (committed vision bars stay green). Replication ≥5 world-seeds; world-ceiling reference pinned before T0; activation-miss ⇒ WITHHELD.

The covenant + the deferred frontier

Only the discrete :scene/:depth indices cross the blanket; pixels, 64-dim Signals, prototype dictionaries, Hellinger distances, and disparity fusion stay cortex-side (mc.ex:253-258; UNI_SIGHT_PLAN.md:88-89). Frontier (deferred, covenant-breaking): ingesting the CONTINUOUS 64-dim Signal into Elixir would replace categorical A_scene with a growing Hellinger-prototype cell (cell.py InferenceCell), forfeiting fixed cardinality + Dirichlet-A learning + byte-identity. Higher fidelity; a separate later rung. Flag, don't build.

Target code (gated, additive; ONLY after formal MERGED VERDICT + owner go-ahead)

genome.ex (:depth/:binocular_cortex, @depth_states, init_a: :diagonal), mc_codec.ex (outcome(:depth)), mc.ex (name-indexed motor_config), bridge.ex (:depth slot + decoder-order test), the Python two-eye + unsupervised-disparity-cluster stereo cortex. Tests: extend decider_byte_identity, action_clone_invariance, vision_test, + the motor_config-by-name regression + decoder-order test. Launcher runs/binocular_vision_lineage.exs + probes + the world-ceiling reference.

sha256 267678186317231e — of the original file, so what was ingested stays checkable.

Written for this website — not the document. This is a plain-language retelling, written to help you meet the document. It is not the source, and it is not evidence. It has not yet been checked by a person. Read the document itself → (or choose Precise in the reading-level control above)

This is part two of a paired design: replacing hand-crafted vision categories with a full two-eyed pipeline that delivers true signals through an existing port. It is design only, and it runs only after the other half has a verdict.

The rule it works under is narrow. Only the likelihood is swapped; the categorical engine itself is untouched. Pixels and everything continuous stay on the far side of the boundary, and only discrete indices cross.

The most valuable passage is a self-criticism. Learning what a bin means is not the same as learning where the bin edges are. If the depth boundaries were designer thresholds, they would be the same forbidden hand-authored categories laundered through a learned likelihood. So the edges must be clustered without supervision, or the central honesty claim is unproven.

A further fix concerns an assumption in existing code that particular factors sit last, which the page shows is already untrue.

Plain · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is 267678186317231e

Written for this website — not the document. This is a clearer retelling, written to help you meet the document. It is not the source, and it is not evidence. It has not yet been checked by a person. Read the document itself → (or choose Precise in the reading-level control above)

This is part two of a paired design, read with a backbone document and its sibling. It is design-only, and it states that its experiment runs only after the other half has reached a verdict, because only one cure moves at a time.

The goal is to replace hand-crafted vision categories, which the project's own guidance forbids, with a full two-eyed pipeline delivering true signals through a port that already exists. The working rule is narrow and repeated: swap only the likelihood; the categorical engine is untouched.

The apparatus is described per eye as a chain of stages, each said to correspond to a real biological mechanism and to be arithmetic only, covering contrast gain control, fixation, aperture integration and an active rotation stage. One design choice is recorded honestly: a passive alternative was measured as negative and is kept switched off. A second section adds two eyes and places the fusion of their difference on the far side of the boundary where pixels legally live.

The brain then gains two discrete factors, one for what is seen and one for depth, with a third deferred to a later step. Here comes a load-bearing correction. Existing documentation notes that a factor of one particular shape with a flat likelihood is not identifiable, so its belief stays uniform and its counts smear. The earlier version of this specification declared both new factors that way. The consequence is spelled out precisely: an inert factor would make a key ablation refute the claim for the wrong reason, namely non-identifiability rather than the stereo signal being decorative. The fix is a weak seeded prior that only breaks the symmetry, plus an activation probe registered in advance requiring the likelihoods to leave uniform on frames kept back from training before any behavioural measurement.

A channels section requires the cardinality contract to raise an error rather than clamp, because the current code silently folds an out-of-range index onto the top bin and corrupts the counts with no error at all.

The central honesty fix follows, and it is the passage that gives the page its integrity. Learning the meaning of a bin by co-occurrence is not the same as learning where the bin edges lie. If the depth boundaries were fixed designer thresholds, they would be categorically identical to the forbidden hand-crafted categories, merely laundered through a learned likelihood. So the edges must be clustered without supervision from the statistics themselves, which is exactly why the other factor escapes the same criticism. An alternative is allowed only with a registered ablation showing the boundary placement does not carry the result. Until this is stated, the page says, the central honesty claim is unproven and the ablation tests the wrong object.

A seams section fixes a second latent defect. Existing code takes the last few factors by position and asserts they are the final ones. The page shows that assumption is already untrue for one combination, and would break with a third addition. The fix is to index by name, with a regression test.

The experiment is paired and deliberately un-bundled, because the treatment makes two changes at once, with an intermediate arm registered so each effect is separately attributable. An offline gate must be green with numbers pinned before any live start, since no stereo code exists yet. Every ablation is a registered number rather than a word like collapses.

The closing section restates what crosses the boundary and flags a higher-fidelity frontier that would break the covenant, with the instruction to flag it rather than build it.

Clear · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is 267678186317231e

Spec — Binocular true-signal vision, the perceptual layer

How to read this page

Precise — the source document