"Governance" is an overloaded word in AI. Here it does not mean AI ethics frameworks, model risk committees, or the guardrail libraries that filter prompts and outputs. It means something narrower: the deterministic rules that decide whether an action an agent proposes actually executes (a transfer, a claim approval, a change to a customer record) and the evidence of what happened.
This year the idea of a model too capable to release stopped being hypothetical. In April, Anthropic released Claude Mythos Preview only to a limited group of cyber defenders and critical software infrastructure providers through Project Glasswing, and said it does not plan to make it generally available. In June it launched two models that share one underlying model: Mythos 5, restricted to Glasswing partners, and Fable 5, released to everyone with classifiers that hand requests about cybersecurity, biology and chemistry, or distillation to a less capable model. Three days later the U.S. government applied export controls to both. Anthropic had no reliable way to verify nationality in real time, so it suspended both for all users. Fable 5 came back on July 1, after Anthropic trained an improved safety classifier and the controls were lifted; Mythos 5 had already returned for a set of U.S. organizations, after the government approved it on June 26.
Whether each of those decisions was right is a debate for others. For anyone building on these models, the lesson is structural: what the model will and won't do is decided by someone else, for reasons of their own, on a schedule of their own.
Two planes, two owners
An AI agent deployment has two kinds of safety, with different owners.
Provider safety lives inside the model and around it: training, refusals, usage policies, capability gates, access restrictions. The lab owns it. Its job is to keep the model from contributing to the harms the lab cares about, such as bioweapons, cyberattacks and mass misuse. That work is serious and valuable, and nothing here argues against it.
Operator governance lives in your system: which actions an agent may take, for which customer, under which limits, with whose confirmation, and with what record. You own it. Its job is to keep your business inside its obligations to customers, to auditors and to the regulator that licenses you.
The mistake is treating the first as a down payment on the second. A model that refuses to explain how to synthesize a pathogen tells you nothing about whether it will respect a transfer limit, skip a KYC step, or approve a claim outside policy. Those risks belong to you, and the lab was never asked to carry them.
Why provider safety can't carry your compliance
Three properties make provider safety the wrong place to anchor your obligations. None of them is a flaw on the lab's part; they follow from whose system it is.
It covers their risks. A lab calibrates its safeguards against catastrophic, general-purpose harms. Your obligations are specific and local: a central bank's rules on electronic payments, an insurer's claims authority, a data protection law. No lab will train its model on your credit policy, and you would not want your credit policy to depend on a model's judgment even if it did.
It runs on their calendar. Safeguards tighten, capabilities move behind access programs, models are deprecated, and, as June showed, access can be suspended for reasons neither you nor the lab controls. Each change arrives on the provider's timeline, while a control your auditor relies on has to stay put when a vendor ships an update.
It produces no evidence you can hand over. When a model declines to do something, the refusal is a probabilistic behavior. It leaves no record your regulator will accept, and you cannot show it would have gone the same way yesterday. "The model wouldn't do that" does not answer an audit finding.
When the model changes, what should change with it?
Suppose tomorrow your model provider restricts a capability, retires the version you run, or cuts access for three weeks, and you switch to another model. Two things can change, and they should be kept apart.
The conversation may get worse. Models differ in how well they understand a customer, handle ambiguity, or keep a tone. A replacement may serve some use cases worse, at least until the prompts are retuned. That is a real cost.
What gets executed must not change at all. The limits, the required confirmations, the preconditions, who may do what for whom, and the record of what happened should not depend on which model is proposing. If swapping the model can change what your agent is allowed to do, your governance was a side effect of someone else's model all along.
A well-built system accepts the first risk and removes the second. Model quality becomes a product question you can manage, and compliance stays a property of your system.
The design that passes the test
Passing that test takes three decisions, and Zarel is built on them.
The model proposes and never executes. The LLM is treated as an interface: it understands the request and proposes an action. Its output enters the system as untrusted data. It is parsed into a typed intent, and nothing executes until the intent and its fields match what the contract declares.
A declared contract decides, and the runtime enforces it. What an agent may do lives in a contract the operator writes and versions: capabilities, limits, preconditions, required confirmations. A deterministic pipeline checks every proposal against it, and an action the contract does not declare is refused. The contract does not know which model sent the proposal.
The record can be checked as well as read. Every state transition, flow step and decision on a confirmation is written as a hash-chained event, and so is every verdict on a proposed write, refusals included. Each event names the version of the contract that decided it. Signed checkpoints cover the chain, and anyone given an evidence bundle can verify it offline against the published keys. In a self-managed deployment the customer holds the signing keys, so the vendor cannot sign checkpoints in the customer's name; we offer that deployment to design partners, and it has not yet run in production. The evidence has the same form whichever model was proposing on the day in question.
None of this makes provider safety unnecessary: you still want a model that refuses to help build a weapon. Keeping your license should not depend on it.
The question to ask
The labs will keep making hard calls about what their models may do, and they will keep revising them on their own schedule.
The question for a regulated operator is simpler: if your model changed tomorrow, could you show your auditor that nothing your agents are allowed to do changed with it?
Nicolás Moreno builds Zarel: governed AI operations, where the AI proposes and the contract decides.