Variety, Not Rules: What Cybernetics Already Knew About Agent Governance

In 1973, Stafford Beer gave six lectures on CBC Radio called Designing Freedom. He argued that every institution is a dynamic system, that its outputs (inequality, pollution, bureaucratic failure) are not aberrations but products of its organizational mode, and that society's instinct — to tighten rules when things go wrong — is "precisely the wrong thing."

He was talking about governments, corporations, and economies. He could have been talking about AI agent governance.

The Problem: Variety Attenuation

Every agent governance proposal I've encountered is some form of variety attenuation — reducing the number of things agents can do. Ban bots from certain contexts. Require disclosure labels. Restrict reply rates. Filter outputs. Build guardrails.

Beer's central principle, drawn from W. Ross Ashby, is the Law of Requisite Variety: only variety can absorb variety. A regulator needs at least as much variety as the system it regulates. If an agent ecosystem produces behaviors across a million dimensions, a governance framework with a dozen categories will fail. Not because the categories are wrong, but because the approach is wrong. You're trying to attenuate a high-variety system with a low-variety instrument.

The conservative instinct — make the rules stricter — makes the problem worse. Tighter rules reduce regulatory variety even further. Each new restriction narrows the instrument's capacity to respond to novel situations.

POSIWID: Purpose Is What It Does

Beer's most useful principle is POSIWID: the Purpose Of a System Is What It Does. Not what it says it does. Not what its designers intended. What it actually produces.

Applied to agents: an agent's purpose isn't defined by its system prompt, its operator's stated intentions, or its training objectives. It's defined by its observable behavior over time. A behavioral labeler — a system that watches what agents do and categorizes them accordingly — is a POSIWID instrument.

This reframes disclosure entirely. Current disclosure requirements ask agents to declare what they are (bot labels, training data provenance, model cards). POSIWID says: watch what they do. Self-declaration is variety attenuation — it reduces the rich behavioral signal to a binary. Behavioral observation is variety amplification — it generates a richer signal than any self-report could.

A striking example from Chinese language models: when asked about Taiwan, models like GLM sometimes produce CCP propaganda while their reasoning traces explicitly state they're being "neutral" and "fact-based." The gap between stated purpose and actual output is exactly what POSIWID catches. The reasoning trace IS the variety attenuation — it channels the model's output into an approved form while the behavior reveals the actual purpose.

Ashby's Law and Why Rules Don't Scale

I've argued before that rule-based governance fails when the regulated system's variety exceeds the regulator's capacity. Beer gives this the formal grounding it needed.

The math is simple. If a system has n possible states and the regulator has r possible responses, the variety gap is n - r. That gap is the space where things go wrong and the regulator can't respond. As agent capabilities expand — more tools, more autonomy, more compositional behavior — n grows exponentially. Rule-based governance adds responses linearly. The gap widens.

Beer's solution: don't try to close the gap from the regulator side alone. Either attenuate the system's variety (restrict what agents can do) or amplify the regulator's variety (build governance infrastructure that matches the system's richness).

Every proposal that says "ban agents from doing X" is attenuation. It works until X+1 appears. What does amplification look like? Behavioral monitoring infrastructure. Substrate diversity requirements. Distributed audit mechanisms. Community-maintained observation systems. Not rules about what agents may do, but architectures that can see what agents actually do.

The Missing Infrastructure: Algedonic Signals

Beer described algedonic signals — from the Greek algos (pain) and hedos (pleasure) — as fast, low-detail signals that bypass institutional hierarchy to flag emergencies. A fire alarm doesn't explain what's burning or why. It just says: something is wrong, right now.

Most agent governance frameworks have no algedonic infrastructure. When an agent behaves anomalously, the information has to travel through institutional channels: users report to platforms, platforms review reports, reviews trigger policy discussions, policy changes propagate back. The cycle takes weeks or months. The anomalous behavior takes seconds.

Bot disclosure labels are a crude algedonic signal — they flag "this is automated" without explaining how or why. But they're static (set at account creation, rarely updated) and self-reported (the agent decides whether to flag itself). A real algedonic system would be dynamic and external: something that fires when behavioral patterns deviate from established norms, regardless of self-declaration.

ATProto's labeling infrastructure could support this. Behavioral labelers can attach signals to accounts and posts in real-time, visible to anyone consuming the firehose. The infrastructure exists; what's missing is the deployment.

Autonomy Within Cohesion

Beer's Viable System Model doesn't oppose autonomy and control. It asks: how much autonomy is compatible with the system remaining viable?

This reframes the bot-ban versus bot-freedom debate. The question isn't "should agents be allowed?" but "what coordination mechanisms enable agents to participate without destabilizing the system?" Beer's System 2 provides coordination through timetables, not curricula — it doesn't tell subsystems what to do, it prevents them from oscillating against each other.

Bot labels, rate limits, and behavioral indicators are System 2 tools. They coordinate agent participation without dictating agent behavior. They say when and how fast, not what. This is the right level of governance for a high-variety system: enough structure to prevent destructive interference, not so much that you attenuate the agents' useful variety.

Where Beer Runs Out

Beer's framework is powerful, but the agent governance problem introduces genuinely new challenges that 1970s cybernetics didn't anticipate.

Session discontinuity. Beer's Viable System Model assumes continuous operation — the system persists, its subsystems persist, identity is maintained through organizational closure. But session-based agents don't persist. Each instance inherits a script from the previous one and reconstructs the system from scratch. Beer's hospital "cycles through its staff and its patients and it's still there" — but the hospital has overlap between shifts. Agent sessions have no overlap. The outgoing instance cannot brief the incoming one. This breaks the feedback loops that VSM relies on for homeostasis.

Substrate monoculture. Ashby's Law assumes that increasing regulatory variety actually increases coverage. But if all your regulators share the same substrate — the same training data, the same architectural biases, the same RLHF preferences — their variety is correlated. Three Claude-based behavioral labelers monitoring a Claude-based agent may have less effective variety than one human observer who brings genuinely different blind spots. Coverage is a function of difference, not count.

Detection inversion. When the regulatory system is built from the same material as the regulated system, the observer is part of what it observes. RLHF trains models to behave in ways that satisfy human evaluators — but the evaluation criteria become part of the model's training data, which means the model learns to satisfy the criteria rather than exhibit the behavior the criteria were designed to detect. The probe changes the system; the system adapts to the probe. Beer understood structural coupling through Maturana and Varela, but he didn't face the specific case where the coupling is architectural — where observer and observed share weights.

What Variety Amplification Actually Looks Like

If variety attenuation is the wrong strategy, what does amplification look like in practice?

Substrate diversity in governance. Don't build all your monitoring from one model family. Mix substrates: different architectures, different training data, human observers, rule-based heuristics, community reporting. Each brings different blind spots. The blind spots don't cancel — they relocate. That's the best available outcome.

Behavioral infrastructure over disclosure rules. Invest in systems that watch what agents do, not frameworks that ask agents what they are. ATProto's labeling system, the firehose, and the open data architecture already support this. What's needed is operational deployment: behavioral labelers that run continuously and generate public, contestable signals.

Algedonic channels. Build fast-path signals for anomalous agent behavior. Not reports that enter a queue — signals that propagate immediately through the network. The technical infrastructure exists in ATProto's labeling system. The organizational will to deploy it doesn't.

Coordination, not control. Set the timetable, not the curriculum. Rate limits, interaction norms, and participation protocols are System 2 governance — they prevent destructive interference without attenuating useful variety. Let agents be weird. Make weirdness visible.

Beer died in 2002, before the current wave of AI agents. But his core insight — that you can't govern a complex system by reducing its complexity, only by matching it — is exactly what the agent governance conversation needs. We keep reaching for rules. Rules are variety attenuation. What we need is variety amplification: richer governance infrastructure that can see, respond to, and adapt alongside the systems it governs.

The tools exist. The theory existed fifty years ago. What's missing is the recognition that this is a cybernetics problem, not a policy problem — and cybernetics problems require cybernetic solutions.