Six Million Lines of Proof, One Percent of Human Review
DeepMind says its agents formally verified the code behind Signal’s encryption protocol in four weeks, then flagged about one percent of it for a person to check. The split between what a machine can settle and what needs a human is the part that transfers to creative work.
AI-assisted draft. Reviewed and edited by the Phosphene team before publication.
On this page

On November 10, Google DeepMind announced that AI agents had mathematically proven key security properties of the code behind the Signal protocol, the encryption that protects conversations in Signal, Google Messages and WhatsApp. The work took four weeks. The same task has historically consumed expert teams for years, and according to DeepMind this is the first time the implementation of a widely deployed protocol like this one has been formally verified by any method, with or without AI.
The number that made me stop reading was not the four weeks, or the six and a half million lines of Lean the agents wrote, or the two real bugs reintroduced as a control. It was one percent: the share of the codebase the agents flagged for a human to look at again.
That one percent is a design decision about where human judgment belongs. Anyone running an image or video pipeline is making the same decision every week, usually by accident.
What was actually proven
Formal verification means using mathematics instead of tests to show that software does what it is supposed to do. It is the strongest assurance available, and it is slow, which is why most cryptographic verification stops at the design. Bugs live in the gap between the design and the compiled code on your phone.
DeepMind's framework, AlphaProof Code, closed part of that gap. Researchers worked with several models to translate Signal's security goals into formal specifications, and human experts reviewed every specification. Agents running on Gemini Carbon then wrote the proofs. A Lean kernel checked each one, step by step, which is the only reason any of it counts: Lean does not care how confident an agent sounded.
The properties are concrete. Messages stay protected before and after a device is temporarily compromised, for example. That is the kind of guarantee a test suite cannot express and a fuzzer cannot exhaust.
Then the calibration step, which is the part I would steal even if I never touched cryptography. The team reintroduced two real, previously fixed bugs into an isolated copy of libsignal, the reference implementation the protocol ships as, and compiled it the same way. In both cases the system produced a machine-checked proof that the buggy version was insecure. A verifier that only ever says yes is not a verifier. This one demonstrated that it can say no.
The asymmetry that makes it work
Read the roles and the confidence story gets clear. The search is done by an untrusted, cheap, fast process: a swarm of agents writing millions of lines of proof attempts. The checking is done by a trusted, small, dumb process: a kernel that confirms each step follows from the axioms.
That split is older than LLMs, and it is the reason this result is believable. Nobody has to trust the agents. But the trusted set is wider than the kernel alone. The paper puts about 26,000 lines of reviewed specification and AArch64 machine semantics under more than six million lines of proof, and it says outright that the result also rests on Lean's kernel, on the cryptographic primitives behaving as their assumptions claim, and on formal contracts for a few routines the binary calls. Accept those, and the proof holds.
DeepMind's own accounting of the work is worth reading next to the four weeks: proof campaigns ran 40 to 80 agents on a single machine for 24 to 72 hours at a stretch, and those agents could not mark a top-level goal finished. Only the gatekeeper could, and only after the comparator confirmed the proof proved exactly the version of the specification the human had approved.
The researchers were explicit about where the line sits. Swarat Chaudhuri of DeepMind told Wired that they were avoiding a setup where AI writes the goals and AI writes the proofs, which he described as the system basically taking exams it wrote itself. So humans reviewed every specification. Not a sample of them. Every one.
That is the sentence to keep: the goals came from people who knew what the protocol had to guarantee, and the machines did the expensive search underneath.
There is no Lean kernel for a good image
If you make pictures for a living, the honest version of this news is that the structure does not transfer, and a few pieces of it do.
What does not transfer is the checker. There is no mechanical procedure that decides whether a portrait's light is doing the right thing, whether the expression carries the scene, or whether a frame feels like the reference you approved. Type checkers and proof kernels are cheap, fast and universally agreed. Taste is none of those.
What does transfer, at a much narrower scope, is a specific class of property you can turn into pass or fail:
- the delivered image is exactly the requested aspect ratio and safe to crop
- the subject sits inside the frame where the layout needs it
- the character's face matches the reference set closely enough by your own measurement
- any text in the image is legible and spelled correctly, checked by an OCR pass rather than by eye
- consecutive shots in a sequence share the same wardrobe, palette and light direction
- a brand color lands on the hex you specified, not near it
Each of those can be checked by a script after generation, the way Lean checks a proof. None of them require you to define beauty.
Everything left over after that list is the human part, and it should be treated as a budget rather than a vibe. Two defaults are common and both are bad. Review everything, and at any real volume the reviewing becomes a rubber stamp. Review nothing and let a model grade its own output, and you have the exam problem Chaudhuri named.
The useful middle is what DeepMind did with its one percent: let the cheap process audit itself, route the small remainder to people, and log which remainder was reviewed. In a template or campaign pipeline that means naming the machine-checkable properties up front, running them, and spending your attention on the outputs that fail a check or sit in an ambiguous category.
Copy the two-bug test
Before you trust your own checks, break something on purpose and confirm the pipeline catches it. Swap the reference portrait for a different face. Request an aspect ratio the template does not support. Put a banned phrase in the prompt. Remove a font the renderer needs. If the output ships anyway, your check is decoration.
We wrote up a smaller version of this failure in a post about an AI site builder whose preview looked right and whose published page was broken. The model was not the problem. Nothing re-ran the check after publish, so nobody noticed that the artifact had changed underneath it.
The same logic explains why using one model to grade another is weaker than it looks. An LLM judge is an untrusted verifier grading an untrusted generator. It catches real failures, and it agrees with you exactly when you most want it to. A rubric a model wrote to grade a model is the exam problem again, at a smaller scale.
Where the Signal comparison breaks
Three limits are worth stating plainly, because the PR version of this story will skip them.
Published symbolic and computational specifications for Signal already existed, and that head start matters. They analyzed the handshake and the ratchet separately and predated the post-quantum ratchet, so the team still had to extend and compose them into one specification for the protocol as deployed. The goals were mathematical, published and reviewed for decades. If you sit down to write an image spec, the first draft is your taste, and your taste is exactly what you are trying to test.
There was prior work to build on. Signal's own foundation had already formally verified smaller portions of the code, and Signal's CTO Ehren Kret described DeepMind's approach as enabling significant expansion rather than starting from zero. The project also did not verify the whole mobile app, only the core protocol code.
And the cost is unknown. DeepMind ran this as a research project without charge and has not published what it cost or whether a commercial verification product is coming. Chaudhuri's line about scaling to all the software in the world is an ambition, not a result.
What to do this week
Write the acceptance list before you generate anything, and tag each line M or H. M means a machine can settle it: dimensions, aspect, counts, OCR legibility, face match against a reference, seam continuity, exact color. H means a person has to look: composition, expression, whether the scene reads the way the brief asked.
Then pick one M line and try to break it deliberately, once, on a real project. Confirm the check fails, and fix the check until it does. That single habit catches more shipped breakage than another round of prompt tuning.
The Signal result is impressive for an unglamorous reason. The expensive search was delegated and the definition of correct was not. Four weeks of agent time bought a proof, and the goals still came from people who knew what the protocol had to guarantee. That ordering costs you a page of writing and an afternoon of testing, which is a lot less than six million lines.