UNI Universal Natural Intelligence

Wiki · Evidence & Verdicts

Kill the fragile WebGL window-capture — move the cameras to a real media stream (scope, 2026-07-16)

Evidence & Verdicts · docs/handoffs/CAM_ROBUST_MEDIA_SOURCE_2026-07-16.md @ 44baf03d5041 (gen2-runtime) — opens the published snapshot ac338733bbba
1 value was removed from this page. Each one is marked in place as [redacted: category] — 1 internal hostname. Nothing else was altered. The document is otherwise exactly as it is written in the repository, and the sha256 below is of the original, so what was ingested stays checkable.

How to read this page

Three ways to read this page. Precise is the document itself, exactly as it is written in the repository. Plain and Clear were written for this website to help you meet that document — they are about it. They are not it, and they are not evidence.

Eighty-seven dated pages: receipts, pre-registrations, handoffs, validation records and review verdicts. A receipt is written at the moment a piece of work was checked. It names what was claimed, the commit and the seed, what was actually run, and the outcome in one of a small set of controlled words. Then it names what the work did not achieve. That last part is what makes it a receipt rather than an announcement. A pre-registration is the same discipline run in advance: the conditions that would count as a pass and the conditions that would falsify the claim are written down before the run, so neither can be adjusted once the numbers arrive.

That is why so many small dated stubs are an audit trail rather than noise. No one of them is meant to be a good read. The value is in the sequence and in the dates, because you can watch a prediction be registered, then the run happen, then the verdict land — sometimes against the prediction. Pages here record a falsified result, a rejected fix, a retracted overclaim, and a green receipt that turned out not to be reproducible from the commit that carried it. A record that carried only successes would be worth a good deal less than this one.

A gentle way in is to read a pre-registration first, so the shape becomes familiar, then a result page, then one of the corrections. This section sits off the main navigation on purpose: it is the record you check the rest of the site against, not the place to begin.

What it is not: documentation, and not a summary. Nothing here has been tidied in hindsight. Every entry reads as of its date, a later entry may overturn an earlier one, and the presence of a page is not a claim that its result stood.

Your browser cannot switch reading levels, so the document itself is shown.

Precise — the source document

This is the document. Rendered from the repository at the commit above, with nothing rewritten for the web. A gate re-renders it on every deploy and fails the build if a single byte differs.

The problem (proven, not theorized)

The colony camera (:3020 prismarine-viewer, a Chrome WebGL page) and the producer's OVERLOOK page (:4200/stream, DOM cards + a :3020 WebGL iframe) are put on the OBS program by window_capture (WGC) of off-screen Chrome windows (cap_colony, cap_overlook; built by viewer/studio_stage.cjs, windows launched by viewer/studio_channels.ps1).

This capture path is fundamentally unreliable for WebGL. Verified exhaustively on 2026-07-16:

  • The Chrome windows render correctly — a raw Win32 CopyFromScreen of the on-screen window shows the live world; CDP Page.captureScreenshot + gl.readPixels (centerMax 255) confirm the canvas paints.
  • Yet OBS's WGC capture of the same window returns a 4307-byte near-black frame — even with the window visible and foreground, even with --disable-direct-composition, even after forcing a re-acquire (SetInputSettings window:"" → back).
  • A full studio_stage.cjs rebuild (RemoveInput/CreateInput) makes a fresh source, and the fresh source randomly either captures live or sticks at 4307-byte black. On the 2026-07-16 rebuild it made cap_colony live and cap_overlook stuck; before the rebuild it was the reverse. Releasing one source did not free the other — so it is not a resource ceiling; individual fresh WGC sources just stick at black, unpredictably, and nothing but another dice-roll rebuild changes it.

Conclusion: no amount of settings-fiddling makes WGC window-capture of these WebGL windows reliable. The --disable-features=CalculateNativeWinOcclusion off-screen trick keeps them rendering, but WGC still fails to pull the GPU surface for a chunk of the time. This has cost the live window twice.

The fix — an actual video stream OBS ingests as a Media Source

Stop window-capturing WebGL. Turn each camera into a real H264/MJPEG stream fed by the proven CDP render path, and point OBS at it via ffmpeg_source (rock-solid, the same input class as RemoteCam*).

Pieces

  1. viewer/cam_bridge.cjs (NEW, supervised). For each WebGL channel (colony :9220, overlook :9221, glass :9222):
    • Connect to the channel's Chrome via CDP (the --remote-debugging-port already on each window).
    • Page.startScreencast({format:"jpeg", everyNthFrame:1, maxWidth:1280, maxHeight:720}) — CDP screencast forces the compositor to deliver frames (this is exactly why captureScreenshot always worked where WGC failed). ~15–30 fps.
    • Pipe the JPEG frames into ffmpeg (-f mjpeg -i - -c:v libx264 -preset veryfast -tune zerolatency) and publish to the LOCAL MediaMTX already running on THINKER: rtmp://127.0.0.1:1935/colony_cam, /overlook_cam, /glass_cam.
    • Auto-reconnect on Chrome/CDP drop; expose /health for the watchdog.
  2. OBS sources. In studio_stage.cjs, change cap_colony / cap_overlook / cap_glass from window_capture to ffmpeg_source with input = rtsp://127.0.0.1:8554/<path> (MediaMTX serves the same path over RTSP for OBS's ffmpeg reader; is_local_file:false, reconnect:true, low buffering). The off-screen Chrome windows stay (they're the render), but OBS never window-captures them again.
  3. Supervision + boot. Add cam_bridge.cjs to systray_watchdog.ps1 (restart-if-dead, same as the other node services) and to studio_up.ps1 bring-up ordering (after channels, before stage). Gate: restream.ps1-style — path=colony_cam ready=True on :9997 + a non-black frame proven by the command center's honest pixel classifier.

Why this is durable

  • ffmpeg media-source ingest is the same reliable path the remote cams use — no WGC, no DirectComposition, no off-screen-presentation lottery.
  • The CDP screencast is the one render path that has been 100% reliable in every test this session.
  • It also unlocks: one canonical camera stream that the command-center preview can sample cheaply, and that could be recorded / restreamed without a second capture.

Scope / cost

  • New: viewer/cam_bridge.cjs (~150 lines) + a ws/CDP screencast client + ffmpeg spawn per channel.
  • Edit: studio_stage.cjs INPUTS block (3 sources window_capture → ffmpeg_source); systray_watchdog.ps1 (+1 service); studio_up.ps1 (+1 launch step); mediamtx_local.yml (3 publish paths, if not all_others).
  • Test: bring up cold, kill Chrome mid-run (bridge reconnects), confirm both COLONY and OVERLOOK live simultaneously (the thing WGC could never do), confirm honest classifier reads them LIVE, confirm boot-persistence.
  • Do NOT rush this in a thrash. It is a clean, self-contained change; land it deliberately with a cold bring-up to verify, not on top of a live feed.

Durable camera-topology changes already made on the chip (2026-07-16)

These landed and are boot-persistent (verified systemctl --user enabled/disabled state on the chip):

  • uni-cam (the legacy standalone camera bot) RETIRED — stopped + container-uni-cam.service disabled. The half-finished migration is now finished: uni-producer is the sole camera+show-runner.
  • uni-viewer-cam-fwd re-pointed host :3020 → uni-producer:3020 (was uni-cam:3020), recreated + started via its persistent systemd unit (enabled).
  • Director "Director"-login kick-fight resolved — the frozen camera was two bots (uni-cam + uni-producer) fighting for the MC login. With uni-cam gone + a fresh producer, the director holds its connection (0 disconnects) and flies the camera; the world is live at the source (:3020/:4200).
  • Producer VIEWER_URL = http://[redacted: internal-hostname]:3020 (name, not a dead IP) — the OVERLOOK page's camera iframe resolves correctly. The chip/colony/producer/director are all HEALTHY; the only remaining problem is the OBS WGC capture path documented above.

Interim (until the above lands)

Only ONE of the WebGL window-captures is reliably live at a time, and which one is a dice-roll per rebuild. As of this writing COLONY (raw camera) is LIVE; OVERLOOK's capture is stuck black. A working live broadcast is available NOW by putting COLONY on program — it carries the live world plus the command-center's own lower-third/ticker/on-air overlays. The producer's UNI-insight cards (the OVERLOOK composition) are unavailable until the media-source fix lands. The colony, producer, director, and camera render are all healthy — this is purely the OBS capture path.

sha256 47ac03253ce5adf7 — of the original file, so what was ingested stays checkable.

Plain — written for this website, not the source document

Written for this website — not the document. This is a plain-language retelling, written to help you meet the document. It is not the source, and it is not evidence. It has not yet been checked by a person. (or choose Precise in the reading-level control above)

A scoping note proposing to replace a fragile way of getting two camera views into a broadcast. The evidence that the current path is unreliable is unusually thorough. The pages render, and other methods can grab their pixels, yet the capture used for broadcast returns a near-black frame. Which of the two views works is effectively a coin toss per rebuild. The proposal is to stop capturing windows and turn each view into a real video stream through the one path that has been reliable in every test. None of it is built yet, so nothing here could be called a result.

Plain · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is 47ac03253ce5adf7

Clear — written for this website, not the source document

Written for this website — not the document. This is a clearer retelling, written to help you meet the document. It is not the source, and it is not evidence. It has not yet been checked by a person. (or choose Precise in the reading-level control above)

A scope document rather than a receipt recording a run that already happened. It proposes work that has not been done, and it opens by establishing that the problem is real rather than suspected.

The evidence is layered deliberately. The pages themselves render, shown both by grabbing the window directly and by reading pixels from the drawing surface. Yet the capture method used to put them on air returns a tiny near-black frame from that same window, even when it is visible and in front, and even after being forced to re-acquire. A rebuild makes a fresh source that either works or sticks black unpredictably, and releasing one does not free the other, which rules out a resource ceiling. The conclusion is that no amount of settings-fiddling makes this path reliable, and that it has already cost the live window twice.

The proposal is to stop capturing windows and instead turn each view into an actual video stream, using the render path that has been reliable in every test this session. That stream then feeds into the broadcast software through the same input class the remote cameras already use. A new supervised bridge would connect to each page, request a screencast, pipe the frames into an encoder, publish locally, reconnect when the page drops, and expose a health endpoint.

The remaining pieces are listed as edits rather than inventions: switch three sources from window capture to a media source, add the bridge to the supervisor and to the bring-up ordering, and add the publish paths. A gate is named, requiring both a ready path and a non-black frame judged by an honest pixel classifier.

Why this would be durable is argued in three lines, and a scope-and-cost section gives the size of the change. One firm instruction comes with it: do not rush this in a thrash, and land it deliberately with a cold bring-up rather than on top of a live feed.

The last two sections are about the present. A set of camera-topology changes has already landed and is described as boot-persistent, including retiring a legacy camera bot, re-pointing a forwarder, and resolving a login fight between two bots that had frozen the camera. And the interim advice is practical: only one of the two captures is reliably live at a time, so put the working one on air and accept that the other composition is unavailable until the fix lands.

Clear · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is 47ac03253ce5adf7