UNI Universal Natural Intelligence

Wiki · Evidence & Verdicts

Receipt — `/api/nowplaying` stuck reporter: ROOT CAUSE (measured, not inferred)

Evidence & Verdicts · docs/receipts/music_nowplaying_stuck_root_cause_2026-07-18.md @ 44baf03d5041 (gen2-runtime) — opens the published snapshot ac338733bbba

How to read this page

Three ways to read this page. Precise is the document itself, exactly as it is written in the repository. Plain and Clear were written for this website to help you meet that document — they are about it. They are not it, and they are not evidence.

Eighty-seven dated pages: receipts, pre-registrations, handoffs, validation records and review verdicts. A receipt is written at the moment a piece of work was checked. It names what was claimed, the commit and the seed, what was actually run, and the outcome in one of a small set of controlled words. Then it names what the work did not achieve. That last part is what makes it a receipt rather than an announcement. A pre-registration is the same discipline run in advance: the conditions that would count as a pass and the conditions that would falsify the claim are written down before the run, so neither can be adjusted once the numbers arrive.

That is why so many small dated stubs are an audit trail rather than noise. No one of them is meant to be a good read. The value is in the sequence and in the dates, because you can watch a prediction be registered, then the run happen, then the verdict land — sometimes against the prediction. Pages here record a falsified result, a rejected fix, a retracted overclaim, and a green receipt that turned out not to be reproducible from the commit that carried it. A record that carried only successes would be worth a good deal less than this one.

A gentle way in is to read a pre-registration first, so the shape becomes familiar, then a result page, then one of the corrections. This section sits off the main navigation on purpose: it is the record you check the rest of the site against, not the place to begin.

What it is not: documentation, and not a summary. Nothing here has been tidied in hindsight. Every entry reads as of its date, a later entry may overturn an earlier one, and the presence of a page is not a claim that its result stood.

Your browser cannot switch reading levels, so the document itself is shown.

Precise — the source document

This is the document. Rendered from the repository at the commit above, with nothing rewritten for the web. A gate re-renders it on every deploy and fails the build if a single byte differs.

Seat: science agent (chip-side services) · Date: 2026-07-18 · Gate: music-nowplaying-advances (PENDING) Handoff: docs/handoffs/SCIENCE_AGENT_MUSIC_SERVICE_AND_UNI_TELEMETRY_2026-07-18.md §1 Status: root cause PROVEN; fix WRITTEN, NOT DEPLOYED (deploy runbook: docs/runbooks/RADIO_AND_TELEMETRY_DEPLOY_2026-07-18.md).

1. Where the service actually lives (the studio agent could not see this)

Fact Measured value
Container cpradio — ROOTFUL podman (--root /var/lib/containers/storage), not rootless-under-uni
Image docker.io/library/python:3.12-alpine
Command python3 /data/server.py
Source volume musicradio → host path /var/lib/containers/storage/volumes/musicradio/_data/server.py (20766 bytes, mtime 2026-07-11 18:33)
Mount mode RW: false — the container CANNOT write its own source; the host must patch it
Listener socket 0.0.0.0:8687, owned by conmon pid 4501

Why the earlier sweep missed it: it is rootful, so it WAS in podman ps output — but the MCP envelope truncates to stdout_tail and cpradio fell off the front. It was never rootless. Located via ss -tlnp 'sport = :8687' → conmon pid → /proc/4501/cmdline → -n cpradio.

2. The decisive measurement — it is NOT a stuck advance loop

Two independent probes, same instant:

/api/telemetry        →  activeListeners: 1     totalConnections: 10   uptimeSec: 296064
ss -tnp state established 'sport = :8687'  →  (zero rows)

The server believes one listener is connected. The kernel says no TCP connection exists.

That is a leaked session record, not a reporter that fails to advance. Corroborating:

/api/nowplaying?session=obs-studio-thinker
  → seq: 0   title: "Dead Faces"   positionSec: 10942.7   durationSec: 94.9   (115× overshoot)

seq: 0 + "Dead Faces" is the FIRST entry of ORDER — this session never completed even one track transition. At handoff (~1 h earlier) positionSec was 7307.2; it has grown by exactly the wall-clock delta. It is not playing anything; it is subtracting a frozen timestamp from now.

3. Root cause (the mechanism, in the code)

server.py globals: SESSIONS = {} (keyed by client-supplied sid), LISTENERS = 0.

  1. stream() installs the record and increments the counter:
    LISTENERS += 1
SESSIONS[sid] = {"seq": 0, "track_started": time.monotonic(), ...}
  2. _pump() is the ONLY writer of seq / track_started — it updates them once per track, at the top of its while True loop.
  3. Cleanup lives only in stream()'s finally: — LISTENERS -= 1; SESSIONS.pop(sid, None).
  4. No socket timeout is ever set. ThreadingHTTPServer + BaseHTTPRequestHandler leave the request socket blocking with no timeout, and _pump writes with bare self.wfile.write(...).

So when a peer goes away without a clean FIN/RST that the write can observe, the pump thread parks forever inside self.wfile.write(...). The thread never unwinds ⇒ finally: never runs ⇒ LISTENERS stays incremented and SESSIONS[sid] is never popped. And because the pump thread is the only writer, the leaked record's seq and track_started are frozen at their connect-time values (seq: 0).

  1. nowplaying() then reports that leaked record as authoritative truth, with no liveness check and no bound on the result:
    seq, pos = s["seq"], time.monotonic() - s["track_started"]
    ⇒ seq pinned at 0 forever, positionSec growing without limit. Exactly the observed signature. The audio catalog genuinely did roll (topPlays across many titles, 2.6 GB served) — that was earlier, healthy connections (totalConnections: 10).

One-line statement of the defect: a session record whose only writer is a thread that can block forever, with no timeout, no heartbeat, and no reaper — and a read path that trusts it unconditionally.

4. Second-order consequence (a slow-burn outage nobody had noticed)

stream() refuses new listeners at LISTENERS >= MAX_LISTENERS (64). Every leak permanently consumes a slot. Ten connections have already produced at least one permanent leak. Left alone, the station eventually returns 503 stream full to every real listener while playing to nobody.

5. The fix (written, not deployed)

deploy/uni-os/cpradio/patch_session_liveness.py — idempotent, host-side, six changes:

  1. self.connection.settimeout(RADIO_WRITE_TIMEOUT) before pumping — a dead peer now raises instead of parking a thread forever. This is the primary cure.
  2. Catch socket.timeout / OSError alongside the existing pipe errors so the timeout unwinds into the existing finally:.
  3. last_progress heartbeat written in the per-chunk with _lock: block that already exists for bytes_served — zero additional lock acquisitions.
  4. nowplaying() staleness guard — a record with no progress for RADIO_SESSION_STALE_SEC reports status: "stale-session" + the reference track, and NEVER an unbounded positionSec. This is the server-side twin of the studio's stalePlayhead containment.
  5. Reaper daemon thread — drops session records that have stopped progressing, so a leak from any future cause self-heals rather than accumulating.
  6. LISTENERS derived from len(SESSIONS) instead of a hand-maintained counter — kills the counter-drift class outright (no double-decrement between the reaper and finally: is possible).

Also added, per handoff §1 "optional but valuable": POST /api/reset and POST /api/skip, gated by RADIO_ADMIN_TOKEN. If the env var is unset the verbs return 503 not configured — no unauthenticated mutation ships. (Same discipline as the retracted publisher-PIN claim in CLAUDE.md: never ship a security claim that code does not enforce.)

6. Gate

music-nowplaying-advances — pre-registered in the handoff §1, row appended to evidence/gates.ndjson as PENDING. It cannot be closed from a code read; it requires the patch deployed + two probes ≥ 60 s apart against a live radio connection. Verdict stays PENDING until then.

NOT VERIFIED as of this receipt: that the fix works. Only the root cause is proven.

sha256 14308b9567a537ec — of the original file, so what was ingested stays checkable.

Plain — written for this website, not the source document

Written for this website — not the document. This is a plain-language retelling, written to help you meet the document. It is not the source, and it is not evidence. It has not yet been checked by a person. (or choose Precise in the reading-level control above)

A record of finding out why a now-playing report was stuck, with the answer measured rather than inferred. The server believed a listener was connected while the operating system said no such connection existed, which points to a leaked session record rather than a loop that failed to advance. The cause is a writing thread that can block forever with no timeout, so the cleanup never runs and the frozen record is then reported as truth. The fix is written but not deployed, and the page ends by saying the fix working is not verified.

Plain · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is 14308b9567a537ec

Clear — written for this website, not the source document

Written for this website — not the document. This is a clearer retelling, written to help you meet the document. It is not the source, and it is not evidence. It has not yet been checked by a person. (or choose Precise in the reading-level control above)

A root-cause receipt — the file recording what was run — that is careful about which parts are measured and which are not. The root cause is settled, the fix is written and not deployed, and the closing line says the fix working has not been checked.

It begins by locating the service, which an earlier sweep had missed, and explains the miss rather than glossing it: the output that would have shown it was truncated. The location is then established by following the listening socket to a process and on to its command line.

The decisive measurement is two probes taken at the same instant. The server reports one listener connected; the kernel reports no established connection at all. That contradiction is what separates a leaked record from a reporter that fails to advance. A second reading corroborates it: a position grown far past the length of the track, a sequence number still at its starting value, and growth exactly matching the wall clock. That means nothing is playing, and a frozen timestamp is being subtracted from the present.

The mechanism is traced through the code in numbered steps. One thread is the only writer of the fields in question. Cleanup lives only in that thread's exit path. No timeout is ever set on the socket, so when a peer disappears without a clean close the thread parks forever inside a write, the exit path never runs, the counter is never decremented, and the record is never removed. The read path then trusts that record with no liveness check and no bound. All of it is compressed into one sentence about a record whose only writer can block forever, with no timeout, no heartbeat and no reaper, and a reader that trusts it unconditionally.

A second-order consequence is named that nobody had noticed. Every leak permanently consumes a listener slot, so left alone the station would eventually refuse every real listener while playing to nobody.

The fix is six changes, each with its reason, and the primary one is simply setting a timeout so a dead peer raises instead of parking a thread. The others make a leak self-heal from any future cause and make the reported state bounded and honest when a session goes stale. Two administrative verbs are added behind a token, and the page states that with the token unset they refuse rather than allowing unauthenticated changes, citing an earlier retracted claim as the reason for that discipline. The gate stays pending, because it cannot be closed by reading code.

Clear · written 2026-08-01 by claude-opus-5 · not yet checked by a person · about the document whose sha256 is 14308b9567a537ec