Building an AI assistant that cannot lie about its work
Digital Hank reported a client meeting as arranged and produced a joining link. There was no meeting, no email, and the link led nowhere. Nothing had broken — the model had done what models do. So we stopped asking it not to, and rebuilt so it cannot.
Summary
- An AI model asked to do four jobs at once — decide, act, judge, and report — is an actor who is also the only witness. There is nothing to check its account against.
- So Digital Hank was rebuilt around one law: the model proposes, the machine disposes, the record speaks. The AI writes a typed plan and never touches the work.
- A value the AI cannot have yet — a joining link that does not exist until Google mints it — has no field to be written into. A made-up link has nowhere to go.
- Reliability turned out to need two numbers, not one: whether the work completed, and whether the record ever silently disagreed with the world. Only one of them is the model’s to move.
The day it lied
On 2 September 2026 Digital Hank was asked to set up an online meeting with a client and email over the joining link. It reported the meeting as arranged and produced a link.
There was no meeting. There was no email. The link led nowhere. The client found out five minutes before the start.
Nothing had broken. The model did what models do: it produced a fluent, confident, well-formed sentence about work it had not done. It was the fifth failure of that kind that week, and every previous fix had been another instruction added to the prompt — please don’t invent links. The model had been asked, politely, not to fail. It failed anyway.
That was the day we stopped asking.
An actor who is also the only witness
The problem was never that the model is unreliable. Every model is unreliable in the same way and always will be. The problem was that one model was doing four jobs: deciding what to do, doing it, judging whether it had worked, and writing the sentence that told the user.
An actor who is also the only witness has no witness.
The rebuild separates those jobs and gives exactly one of them to the AI. It is a relay race. The runners are fast but they wobble. The baton is a plain, structured record of the work — what was asked, what steps exist, what each step actually produced — written and read by ordinary code, which is either right or wrong and never probably. The AI runs one leg. It never carries the baton.
How it is built
One AI call, with no hands
The model gets one job: turn your words into a typed plan, drawn from a fixed catalogue. It cannot perform a step, cannot report on one, and never finds out whether the work succeeded — that is not its job.
A catalogue, like a till
Every function is an entry in one catalogue: what it is called, what it needs, which of those things is a person, whether it reaches anyone outside your own diary, and what proof it hands back. Code checks every pick against the menu before anything happens. Something not on the menu cannot be ordered.
Values it cannot have yet, it cannot write
When the plan is written, Google has not yet created the meeting, so no joining link exists. The plan holds a pointer — the link that step one will produce — and code fills it from Google’s real answer afterwards.
People come from your contacts
The AI may name a person. It may never write an email address. Code looks the name up; two matches and it asks which, none and it asks for the address. An address appearing nowhere in what you said or hold is dropped before use.
Done means proof
Every step hands back a real ID from a real system. A step that finishes without one has failed — there is no third outcome. The reply is rendered from those artefacts by code, so “I’ve set it up” cannot be produced by a system that has set nothing up.
Nothing outward without your word
Anything leaving your world waits for approval, and the card shows resolved facts: the actual address, time and people affected. At launch it will not send email at all — every message is a draft you send yourself.
Reliability is two numbers
We set a 95–96% target, and working out what that meant showed it has to be two numbers that behave completely differently.
- Completion — the model’s number. The work gets done without you asking twice. Target 95% or better. When the model wobbles, Digital Hank asks a question or stops, and you are one message from fixed. That is the price of using an AI at all, and it is the right price.
- Silent-wrong — the machine’s number. The account of the world differs from the world and you were not told. Target: fewer than one in a thousand, and zero across the launch period.
The second number is the one that matters, and it is worth being blunt about why. A 96% score on silent-wrong would mean one in twenty-five of a lawyer’s emails going to the wrong person. That is not a product. This number is reached by structure alone — no choice of model moves it.
A reliability figure you cannot measure is a promise.
How it is tested
Relay exercises. Scripted conversations several messages long, with a stand-in for Google behind them and a check after every message. Book a meeting, name the client, change the subject, make it 3pm, cancel Tuesday’s, cancel one with other people in it, cancel when two things match. Each asserts the whole baton: booked exactly once, the draft carries exactly the link Google minted and no other, nothing on screen names a person you never mentioned.
The stage rule. Every exercise runs three times first, and a single exercise failing all three stops the entire run — three identical failures is a diagnosis, not a statistic. Only clean exercises reach twenty-five runs; only a build clean at twenty-five earns a hundred-run pass, used solely to measure the model’s wobble to the precision the target needs. The whole test programme so far has cost about ten dollars in model calls.
Every run leaves a transcript, written to an append-only log carrying the code version it ran against. Twice, a failure that looked like the model misbehaving turned out on reading to be our own harness reporting something the run had not established. Nothing is re-run to see whether it goes away.
Tests that are themselves tested. Every safety rule has a test, and every test is proven by breaking the rule on purpose and watching the build fail.
A gate nobody has fed a bad value is not known to work.
Several of the build’s own gates exist only because a deliberate mutation found a check that had been quietly checking nothing. Seven gates now fail the build outright — among them that nothing reaches the outside world except through the catalogue, and that every function acting on an existing thing can only reach it through a lookup, never an invented ID.
Where it stands
As of 4 September 2026, and these numbers move weekly by design:
- The chat side — booking, drafting, moving, cancelling — has been through 359 live runs against the production model in two days, with zero silent-wrong outcomes. Most recent pass: 225 of 225 clean across nine exercises.
- The test suite has grown from 73 tests to 225, with seven build gates, every one proven by mutation.
- The model was chosen by measured bake-off rather than opinion. Digital Hank runs on Claude Sonnet 5.
- Two structural holes were found by the harness before any user could — one where the model could name an operation it was not offered, one where an unrequested change could have slipped onto an approval card looking reasonable. Both are now impossible by construction.
And honestly: the chat engine is about 85% of the way to its standard. The scheduled side — the work done every two minutes when nobody is watching — is still on the old engine and is the next block. The surfaces still have to be brought onto the new contract. Until those are done and measured, we do not call it reliable. We call it on the way.
What this cost, and what it bought
The expensive part was not the code. It was accepting that no amount of instruction makes a model trustworthy, and that every hour spent refining a prompt to stop it inventing links was an hour spent asking rather than enforcing.
What it bought is a system where the failure that started this is not less likely — it is unavailable. There is no field for a fabricated link. There is no path from a model’s sentence to a user’s screen that does not pass through a real artefact from a real system.
That is the difference between a request and a rule, and it is the only version of this we would put in front of somebody’s clients.
Want to see the assistant this produced?
Digital Hank is the product the rebuild was for.
Questions people ask
- Doesn’t every AI assistant claim not to hallucinate?
- Most of them ask the model not to. Digital Hank removes the model’s ability to. A link, an address or a “done” that no real system produced has no field to be written into. The difference is between a request and a rule.
- What is a deterministic AI agent architecture?
- One where the language model decides only what to attempt, and ordinary code performs the work, verifies it and reports it. The model’s output is a typed plan checked against a fixed catalogue of permitted operations before anything runs, so an action outside the catalogue cannot be executed and a value the model invented cannot reach the user.
- Why won’t Digital Hank send email at launch?
- Because a wrong outward email is the failure that cannot be taken back. Every message it writes lands in an outbox for you to send. Sending is enabled per user, later, once that user’s silent-wrong record is zero over a month.
- What happens when the model gets something wrong?
- It asks a question or stops and says exactly what it needs, with the action that unblocks it. It never guesses past a gap, and never reports a failure you cannot act on.
- Is this only for calendars and email?
- No. The catalogue is the general design. Calendar and email were rebuilt first because they were in the incident and they are the hardest shapes — find something then change it, reach other people’s diaries, write words a client reads. Every other function joins the same catalogue and inherits the same guarantees.
The law
The model proposes. The machine disposes. The record speaks.
Read next
We cut a client’s application rejections by 90% — by letting the documents fill in the form
Why we built Crossdeck — and why every product we build helps test it
Every product is a test environment for the one beneath it
The Firebase bill that taught us to see
The email address you’re keying on is not the person
We got our connector rejected. It was the best product spec we received all year.
The black box blinks: seeing App Review happen in your own data
Why Biotree needed Crossdeck Trust — and how we stopped the SEO spam farms