Building on the Verifiable Autonomy paper

Where the paper stands

"Autonomy without verifiability is a liability, not a capability."

That line is the paper's thesis, and it's the right one. Insurers want agents to do whole pieces of work: reading a submission, checking appetite, quoting, settling a simple claim. What blocks them is the second line (risk and compliance) and the third line (audit), who will ask what an examiner asks: show me. A manual process can answer that but is slow. Most AI pilots are fast but can't. The paper argues for having both, and the picture below is worth adding as an exhibit.

The paper's core: four properties per decision

The paper requires every autonomous decision to be explainable, auditable, accountable and reproducible, and says which mechanism provides each. This is its strongest part, and the ideas below build on it rather than replace it.

The paper describes these as requirements and never shows them together. A worked example of one case record would make them concrete for a risk reader:

What it leaves thin

The four properties are about a single decision. An insurer running a fleet of agents has three more questions, and the paper only touches them in passing. Amber marks where an idea below adds something.

Beam status comes from reading the code on staging as last pulled (13–18 Aug 2026), not docs. Check with engineering before the paper or the Road Ahead section claims anything.

Ideas to build on it

Idea 1 is the mental model: what is being governed, and the five things that matter. Idea 2 is who governs it: the three lines. Ideas 3–9 each sit in one of those lines, and idea 10 is the order in which to roll it all out. Each follows the same shape: what the draft says now, the idea and how agents deliver it, the evidence, where Beam stands, and the strongest objection.

1One mental model: the agent system is a box

Seen from outside, the agent system is a box. A submission, a claim or an email goes in, and an outcome comes out: a quote, a decision, a payment. It is an outcome-generation machine.

Insurers already run boxes like this. An actuarial pricing model takes risk data in and puts a premium out, and the industry has spent decades learning to govern it under the name model risk management. Supervisors define a model in exactly these terms: a method that processes input data into output. The UK definition explicitly includes qualitative inputs and outputs (PRA SS1/23).

What is important to manage the box

Five things, and four of them insurers already do for actuarial models. The third is new, because an actuarial model only advises: a person still decides. The agent box acts.

In one line: validate it like a model, authorise it like an underwriter.

Why this box is harder than an actuarial model

• It acts. A pricing model never pays a claim, so authority limits (point 3) have no precedent in model risk management.
• The same input can give different outputs. Run a sample of cases more than once and measure the spread, and keep the written reasoning so a different answer can be explained.
• It changes itself. A recalibrated actuarial model goes back to validation. A learning agent changes weekly, so learned changes need the same approval as written ones, and line 2 has to check every run.
• Its inputs are unstructured. The validated range is defined by document types and fields, not just numeric bands.
• It has many steps inside. That's why it should be built as boxes inside the box (idea 3): each stage has its own input, goal, authority and check.

2Three lines of defence, with agents doing the reading

Keep the three lines, which supervisors already know. Change who does the reading, and give the third line a job the paper doesn't have yet: testing the checkers.

• Line 1: the agents doing the work control themselves. They work in stages with gates, evaluate each step, and stop at guardrails (ideas 3 and 5). The process owner answers for the outcome.
• Line 2: AI agents cross-check line 1, continuously. They do three jobs no dashboard does:
  • Risk-based review: look harder where the stakes, the agent's uncertainty or the novelty of the case are high, and lightly elsewhere.
  • Patterns across runs: the same mistake in many cases, one broker's submissions suddenly passing more often, two agents contradicting each other, drift in one segment.
  • Anomalies: an approval rate that jumps, an unusual sequence of tool calls, an outlier amount.
  People in the risk function still own the risk appetite, set the review budget and decide every flag. Agents do the reading, not the deciding.
• Line 3: people with tools to look into the system. They drill into any case, replay it with the version that ran, and draw samples by query (idea 4). Then they use the same tools to audit lines 1 and 2: how well do the checks catch failures? Every miss becomes a fix to one of the two lower lines.

Auditing the auditors

Once agents do the second-line reading, the obvious question from a supervisor is "how do you know your AI checker works?" Line 3 answers it with two tests:

• Seeded cases. Line 3 plants known-bad cases in the live stream, such as a submission just over the authority limit or a duplicate invoice. Line 2 doesn't know which ones they are. The share it catches is its catch rate on known failures. This is how AI-control research measures monitors: by attacking them with failures of a known kind (Redwood).
• Blind re-review. Line 3 independently reviews a random sample of what line 2 passed and what it flagged. Errors in the passed sample give the miss rate. Flags that turn out fine give the false-alarm rate.

The other ideas in this writeup each sit in one line:

3Stages with gates, not one free-flowing agent

The simplest way to build an agent is to give one agent every document, system and tool and let it work out its own path. It is flexible and very hard to audit. The idea is to break the work into stages with three rules:

• Each stage has one clear goal, such as "is this submission inside appetite?", stated before the agent runs.
• Each stage gets only the data it needs for that goal, and nothing else.
• Work moves on only after it passes a gate: an exit check that the goal was met. Some gates are automatic; at the high-stakes ones a named role approves.

For an auditor, the difference shows up in every question they ask:

Gates are also what the other ideas hang on. The case record logs one entry per stage (idea 4). Guardrails and narrow evals sit at gates (idea 5). A gate is where a case goes to two reviewers (idea 6).

Beam today: agents are already graphs of steps, and each step can have its own evaluation criteria and an approval gate before a tool runs. Whether each step sees only the data it needs wasn't checked in the code.

4Logs an auditor can check in minutes

An audit is a spot check. The auditor pulls a sample, then reads each case from the top down, drilling in only where something looks off. A complete record isn't enough if nobody can find their way through it. It needs three things:

• a one-line summary at every level
• metadata on every record, so cases can be filtered and followed across agents
• search over those fields, so the sample is drawn by query rather than picked by hand

Regulators ask for the duty, not the format. The EU AI Act requires automatic logging over a system's lifetime (Art. 12). The IAIS wants logs that let supervisors "assess and challenge the decisions". US market-conduct exams test samples of about 76–109 files (NAIC). None of them says "searchable". The argument is ours: if an audit is a sample, the sample has to be findable. OpenTelemetry's agent conventions give a shared vocabulary for the metadata.

Which of Beam's logs are verifiable today

5A monitor agent for live risk

A running fleet needs supervision in three layers, each doing a different job:

• Guardrails block. They are fixed rules that sit before actions that can't be undone, such as paying above a limit or binding outside authority. A block can't be allowed to fail open, so these are rules, not a model's judgement.
• Narrow evals detect known insurance failures after each step: a sanctions hit overridden, gross used in place of net, a coverage decision that contradicts the payment. A detector built for one failure can be no better than a coin flip on another, so each one targets one pattern (Oxford, 2026).
• A broad monitor discovers. A general model reads whole runs across agents and flags what nobody wrote a rule for. It runs on a different model family from the agents, because judges pass their own family's failures more often (study). It sends only a fixed budget of cases to people, so they aren't swamped.

The three layers feed each other. That's how the fleet learns new risks without anyone having to guess them in advance:

Each promotion is a versioned change with an approver. That plugs directly into the governed learning loop the paper describes under practice 4 in §7.

Beam today: a per-step judge with criteria an LLM generates, one per output field (generic, not domain-specific), an approval gate per tool, and a scheduled email or Slack digest of task statuses. There are no blocked-action lists, no PII filtering at runtime, and no monitor across agents.

6Two humans as the benchmark

The human isn't ground truth. In Kahneman's noise audit at an insurer, two underwriters pricing the same case differed by a median 55% of the premium, against the 10% executives expected. Claims adjusters differed by 43% (Noise). Against a reference that noisy, "the agent agrees with the human 85% of the time" has nothing to be measured against.

So keep people working in the system and send some cases to two of them. It looks inefficient, but it is how the system learns. Their agreement rate is the ceiling the agent is scored against, and their disagreements mark the rules that are unclear.

Cost: practitioners double-review 5–20% of items, not everything (guide), so this is a benchmarking spend on a slice of cases. I found no primary source on how often insurers already double-review claims blind, so the paper shouldn't call it standard practice.

Beam today: nothing. Tasks can't go to named reviewers, and nothing compares one human with another. The closest thing to a benchmark is the expected output a user gives as feedback on a step.

7Learning from copilot chats, under sign-off

When staff use a copilot, the conversation already holds what a rule needs: the why behind each correction. "No, use net TIV, gross double-counts the sub-limits" is worth more than any thumbs-down. But a chat records what the person accepted, which isn't always what is right:

• People often give in to a confident model (HBS).
• About 82% of follow-up turns carry no feedback at all (WildChat).
• Approval rewards agreement over truth (Anthropic).

So the chats are a source of candidate rules, not training data.

Extracting written rules also has evidence behind it. In a Microsoft study, turning each correction into a stated preference cut later edits by 31–73% (PRELUDE). In Germany, mining staff conversations needs works-council agreement, because the system could be used to monitor people (Orrick).

Beam today: copilot conversations are stored, and nothing mines them. A user can send feedback from the copilot to the auto-tuner, which rewrites prompts. With auto-apply on, it does that without anyone signing off.

8Audit that pays for itself

Most organisations run audit as compliance: find the problem, write it in a report, close it later. The best run it as a learning engine. Every finding is traced to the reason the system allowed it, fixed once, and never paid for again.

Research on internal audit finds the same split. Audits mostly focus on compliance, but can act as a "generative mechanism for business-relevant improvements" when they are tied to business objectives rather than a fixed audit cycle (Lenning & Gremyr). With agents this goes further than it can with people: a fix becomes a versioned rule, eval or gate that applies to every future case from the next run onward.

Two KPIs

Does the audit pay? Building the case

Yes, and the case is stronger for insurance audit than for evals in general, for three reasons:

• There is a measured pool of loss for the checks to stop.
• Human claims audits already show a return, even while reviewing only a small sample.
• An AI check costs a tiny fraction of a human one.

The published "eval ROI" statistics are weaker than they look. Most are vendor arithmetic built on illustrative inputs, and some quote analyst figures without a primary source. They're fine for intuition and shouldn't go in the paper. What can go in:

Where the return comes from. Not from checking faster. It comes from three shifts a human-only audit can't make:

• Coverage: from about 5% of files to all of them.
• Timing: from after payment (recover what you can) to before payment (prevent it).
• Compounding: each finding becomes a fix that stops the same error on every future case (the loop above).

The number that settles the argument is the break-even: the share of leakage the checks must stop to pay for themselves. Because the pool of loss is so large relative to the cost of checking, it comes out at well under 1%.

At the defaults (10,000 claims a month, €6,000 average, 8% leakage), checking costs about €31,000 a month and pays for itself once it stops 0.65% of leakage. The case does weaken for a carrier that already leaks little. At best-in-class leakage of 2%, break-even rises to about 2.6%. At 1% leakage with only 2% of it stopped, the checks return 0.4× and don't pay on leakage alone. The input that moves cost most is the flag rate: a check that sends too many false alarms to people burns the return on reviewer time. That's why line 2 is scored on false alarms (idea 2).

For the paper, this turns §9's first conviction from an assertion into something an institution can track. It also adds a ninth question to §8's list of what a supervisor should be able to find: what has audit made better, and by how much?

Beam today: the auto-tuner already runs a learning loop. It rewrites prompts and criteria from feedback and compares scores before and after against expected outputs. But it optimises the eval score, not cost of failure or a business metric. Task records carry run cost, tokens and time, but nothing links an agent to the business outcome it is responsible for.

9AI-enabled audit: breakers, backtests and smarter samples

Three platform capabilities turn audit from a periodic human exercise into something the system does with people:

1. Circuit breakers: the kill switch that pulls itself

A kill switch waits for someone to notice. A circuit breaker trips on its own when error signals cross a threshold, much as exchanges halt trading after a sharp fall. Software has used the same pattern for years to stop one failing service taking others down (Fowler). For agents, it means:

• Triggers:
  • the rate of failed gates or retries jumps
  • line 2 raises an anomaly
  • inputs drift outside the validated range
  • a volume cap is hit, such as total payments released in an hour
• Scope: trip only what is affected. That could be one stage, one agent, or one segment such as a single broker's submissions, rather than the whole fleet.
• States, not an off switch: when it trips, cases go to people instead of stopping. After a cool-down, a small share runs autonomously again under checks, and full autonomy returns only if that share comes back clean.

2. Backtest every new check on history before it goes live

Ideas 5 and 8 keep adding checks: a pattern line 2 found becomes a narrow eval, and a miss becomes a new rule. A new check is a model too, and it should be back-tested like one. Because the case record keeps every input and the version that ran (idea 4), the platform can replay a new check over months of past cases before trusting it. That answers three questions:

• How much work will it create? The share of past cases it would have flagged is the future review load, and part of that is false alarms.
• Does it catch what it should? The share of known past failures it would have caught is its recall.
• What did we already get wrong? Past cases that now fail are found before an examiner finds them. The paper's "retroactive audit" becomes routine rather than an exception.

3. Let the system choose where line 3 looks

A random sample measures the error rate fairly but mostly lands on routine cases, where the agent was validated and rarely fails. The system knows things a sampler doesn't. It knows which cases are unlike the data the agent and its checks were validated on, for example:

• a new broker
• an unusual combination of class and size
• a document layout seen for the first time
• a segment whose inputs have drifted

Those are where both the agent and the checkers are most likely to be wrong, and where a person's time finds the most. So line 3 samples in two parts:

• A random share to measure: an honest error rate for the paper's "is it still performing?" question.
• A targeted share to discover: the cases furthest from the validated distribution, plus the highest-value ones. Financial auditors already weight samples toward large amounts (monetary-unit sampling), so weighting by novelty is the same idea on a new axis.

Mixing the two needs one statistical rule. Targeted cases are over-represented, so they must be weighted down when the overall error rate is reported. Otherwise the targeted share inflates it.

10Roll out in waves: low impact first, so the audit culture grows with autonomy

The risk map that sorts use cases by impact on the customer and AI autonomy is usually read as a control map: the further up and right, the heavier the controls. It works just as well as a rollout order.

Start where mistakes are cheap, and use those cases to build the audit machinery and train the people who run it. Each wave then adds one new kind of risk, either more autonomy or more impact, never both at once. By the time a use case reaches the top right, the institution has already run every control it needs, on cases where getting it wrong cost little.

Start every use case with a parallel run

Before an agent acts on anything, a person and the agent work the same live cases side by side. The person does the work as today, and theirs is the answer that goes out. The agent's answer is recorded and compared. The paper already prescribes this as hypercare (§6): at least four weeks for deterministic work and eight for judgement, with the configuration frozen and every divergence classified. It treats the parallel run as the way to retire a manual process. It is worth more as the benchmark every later decision is measured against.

Three things make it a benchmark rather than a go-live hurdle:

• Add the second reviewer from day one. A disagreement between agent and person doesn't say who was wrong. A second, blind reviewer on a sample gives the human-human agreement rate (idea 6), and that sets the bar the agent has to reach.
• Keep the cases. Every settled case in the parallel run becomes a labelled example. Together they are the benchmark set that each later change, new check and backtest is scored against (idea 9).
• Run it again when the stakes change. A shorter parallel run before each authority increase or major change gives the evidence for raising the limit. It shouldn't be a one-off at go-live.

Inside each use case: authority is earned

The same principle applies within a single use case. A junior underwriter's authority letter starts small and widens as their files hold up. An agent can earn authority the same way:

Each step is a governed decision. It is backed by a fresh parallel run on the paper's hypercare terms and signed by the role that owns the limit. That gives the box's third control, "what it may do alone" (idea 1), a way to grow rather than a fixed setting.

Why low impact first works for change, not just risk. An audit culture is built by doing audits, and the first ones go wrong. People learn to read case records, to challenge a well-reasoned AI answer, and to act on a breaker trip. Those lessons are cheap on document summaries and expensive on pricing. Each wave also leaves behind reusable machinery: the case record, the line-2 agents, the sampling, trained reviewers. That's the paper's "the tenth inherits the estate", made into a plan.

Material for the Road Ahead section

Laurin asked for bullets on where Beam is going. These are the gaps above, written as roadmap items. Each is a commitment only you can make: tick the ones you'd put Beam's name to.