← LearnWorkbench ▸·Path:

Clockwork Active Inference — POMDP Machine

A mechanical-feeling teaching game for the equations sτπ = softmax(vτπ), v̇ = ε( messages ), and π = softmax(−G − F). The hidden machine mode is never observed directly. You only get noisy gauge readings, transition rules for actions, and preferences over future outcomes.
State space: 2
Observations: 2
Actions: 2
Policies: 4
Horizon: 3 slices

1) Pick a clockwork world

Each preset defines the four ingredients: likelihood A, transitions B, preferences C, and start-state belief D.

2) Message passing

One tick applies the negative free-energy gradient update to every policy-specific belief trajectory. Settling repeats it until the beliefs stabilize.
Ticks
0
Time slice
1
Free energy
—
Policy mass
—

3) Policy selection

Equation 4.14 is evaluated after beliefs are updated: the posterior over plans is proportional to exp(−G − F).
Or override the machine and test your own move:

4) Explore the equations physically

Use the lever to change how strongly preferred the quiet outcome is. The policy posterior will re-weight accordingly.
Future preference for Quiet0.85

What this game is doing

The hidden state at each time slice is a categorical distribution over Calm vs Faulty. A noisy gauge emits Quiet or Rattle observations using A. Actions push state beliefs through B. Preferences C score future predicted observations. D is the initial belief before any data arrive.
Time slice
1
τ index
Ticks
0
message passes
Best policy
—
π posterior
F(best)
—
variational
G(best)
—
expected
Quiet streak
0
preference wins
GoalLearn when Repair beats Wait. The agent infers a hidden Calm/Faulty state and picks policies by expected free energy.
WatchThe π column on policy cards and the policy-posterior ledger. They compute softmax(−G − F).
First moveClick One message tick (below) a few times, then Score policies. Then Auto-act to execute the best plan.

Machine floor

The left window is the hidden chamber, the middle window is the sensory gauge, and the right window shows the current belief mass.

Hidden chamber

Hidden until revealed
True state evolves according to the selected action and the transition matrix B.

Observed gauge

Only this reading is directly observed now.
Quiet
Rattle
—

Current posterior q(s1)

Beliefs shown here are mixed across policies using the policy posterior.
Calm0.50
Faulty0.50
These are not the hidden state itself. They are the agent's beliefs after the variational updates.

Policy workshop

Each policy is a two-action plan over the next two transitions. Step the beliefs, then compute F, G, and π.
Actions: Wait = hold position, Repair = push toward Calm.

Generative model matrices

Columns are source states. Rows are outcomes or destination states.

Episode log

You can compare the machine's chosen action with your manual override and watch the hidden process unfold.

Equation 4.13 ledger

For the selected policy, the machine updates the auxiliary variable v with likelihood messages, transition messages from the past, transition messages from the future, and an entropy-like correction term −ln s.

Equation 4.14 ledger

After computing policy free energy F and expected free energy G, the posterior over policies is formed by a softmax over −G−F.

Teaching notes

Why there are multiple state beliefs at once: each policy carries its own predicted hidden-state trajectory. The agent then compares these policies using F and G.

Why future slices can update current slices: the backward message ln(BTsτ+1) lets a future consequence constrain an earlier hidden-state belief.

Why preferences matter only in policy selection here: observations at the current slice are actual data; future slices are not observed yet, so predicted outcomes are evaluated through G using C.

Layered Equation Panel

Same F/G/π math, four depths. Switch anytime.

Beat Script

Step one hero quantity at a time; every sim control stays usable.