Generative models cluster, sketch

Encoding a Generative Model in Elixir: A Sketch

A generative model is not a picture the organism holds of the world. It is the set of expectations the organism runs the world through. In the UNI workbench that set lives in Elixir, as a small collection of tensors and priors sitting inside a supervised process. This post is a sketch of how we lay it out, at a level a reader who has not seen our internals can follow.

Frame. UNI is a working hypothesis on an attainable path toward General Natural Intelligence: a natural, active-inference approach whose evidence is growing, evidence-classed, and tested in the open. Do not take the claim on faith. Test the build, inspect the gates, and help us find where it fails.

The starting shape

Active inference under a discrete POMDP formulation asks for four things the model must carry: a prior over hidden states, an observation likelihood mapping states to sensations, a transition model over how states evolve under action, and prior preferences over the outcomes the organism expects to occupy Class E (Parr, Pezzulo, Friston, 2022, MIT Press). Everything else in a lab episode, precision, planning depth, policy temperature, is a knob attached to one of those four.

In Elixir we sit those four inside a GenServer. The state field is a struct, the arithmetic is done through Nx tensors, and the process supervises itself under a plain Supervisor tree. This is not exotic. Most of what makes it feel calm is that the language already thinks in isolated processes, messages, and immutable state, which lines up cleanly with the Markov-blanket separation the theory asks for Class E.

The struct

The generative model itself is a struct, held as the process state Class C. A stripped, illustrative sketch (not our production shape):

defmodule UNI.GenerativeModel do
  defstruct [
    :qs,          # posterior over hidden states, Nx tensor
    :a,           # observation likelihood P(o | s)
    :b,           # transition model P(s' | s, a)
    :c,           # log prior preferences over outcomes
    :d,           # prior over initial hidden states
    :precision,   # sensory and transition precision knobs
    :depth        # planning horizon
  ]
end

The letters follow the convention in the Parr, Pezzulo, Friston textbook, so a reader who has been through Chapter 7 can map straight in without a glossary Class E. The important thing to notice is that this struct is not the organism. It is the organism's expectations. What the organism sees at any tick is a separate observation term, delivered by message, and the update to qs is what perception is in this frame.

Perception as a message

When an observation arrives, the process folds it into the posterior by running a short variational update: multiply the likelihood column for the observed outcome, multiply the transition-driven prior, normalize. In pseudo-Elixir:

def handle_cast({:observe, o}, state) do
  qs_prior = Nx.dot(state.b_prev, state.qs)
  qs_lik   = Nx.take(state.a, o, axis: 1)
  qs_new   = qs_prior |> Nx.multiply(qs_lik) |> normalize()
  {:noreply, %{state | qs: qs_new}}
end

Two things are worth naming. First, this is a Bayesian update, not a scoring function: the posterior is what we carry forward, and free-energy minimization is what the update is, not a metric we compute after the fact Class E. Second, the message boundary is doing real work. The observation crosses into the model through a single, typed cast; nothing else in the system reaches into qs. That single door is the practical face of a Markov blanket in code.

Action as expected free energy

Action selection asks the model to look forward a few ticks, roll out each candidate policy under the transition tensor, and rank them by expected free energy: a sum of a pragmatic term (how well the outcomes match c) and an epistemic term (how much the rollout is expected to reduce uncertainty about hidden state). We keep the depth small, usually 2 or 3, because the tensor shapes grow fast and, in practice, most of the observable behavior we care about in the Precision Lab and the Cell Lab shows up inside a short horizon Class E.

The specific inner arithmetic that turns a policy into a scalar, the way we shape precision across sensory and transition channels, and the smoothing we apply when a policy would otherwise collapse to a spike, are the parts of the codebase we do not publish. They are the internals under review.

What this sketch is, and is not

This is a Class B and Class E sketch. Class B because it describes real code shape you can compare against the labs. Class E because the framing follows a cited textbook rather than an in-house derivation. It is not a validation of the theory, and it is not a working implementation you can drop in. It is what we can say publicly about how we lay the model out, so that when we ask you to inspect the gates and help us find where it fails, you know what shape the thing under the gates has.

If you want the theoretical background, the companion post Generative Models: The Organism's Model of Its World is the pair to this one. If you want the honesty posture around it, our transparency page lays out which claims carry which evidence class, and which ones we hold as unverified.

Where to go next

Generative models, the theory ›
The pair to this post: what the four tensors are trying to be, before we write any Elixir.
Gates and falsifiers ›
How we know when we are wrong. The falsification posture that makes the sketch above worth reading.
Transparency ›
Which claims carry which evidence class. Where we mark our own work as unverified.
The Science ›
The preprint, the labs, the pre-registered Cell Lab benchmark, and the public MCP server.