Lawmakers just learned that their laws have "bugs" AI can find and exploit. Your standard contract terms can have the same problem. Here's why, and how to fix it before somebody else finds it.
What did the researchers find?
In late October, Stanford Law School's Legal Innovation through Frontier Technology Lab published a report by Felix Ringe and Linus Folkerts called "Legal exploits may emerge at scale." Their starting point is simple: Written law is a lot like software, so it can contain vulnerabilities that someone can exploit.
They tested that idea on German regulations. Under the German constitution, every regulation has to cite the statute that gives the government the power to issue it, and any provision that the cited statute doesn't cover is void. The researchers used large language models to hunt for provisions that failed that test, and they say they found "a substantial number" that, to their knowledge, nobody had recorded before.
They published three of them, and (in what I think is a responsible practice) they only shared the ones whose exploitation wouldn't cause serious harm.
Why hadn't anyone noticed?
According to the researchers, courts absorb most of these defects through interpretation, so the bugs rarely come to light in the first place. Nobody looks closely at a rule until someone has a reason to fight about it.
Their best example is a rule added in December 2015 that required Malian nationals to get a visa just to change planes at a German airport. The amendment cited the wrong parts of the statute, so according to the report it's void. Germany amended that same list again in 2023 and again in 2026, and neither amendment fixed the original defect. So the bug has been sitting there for about 11 years, and the document was edited twice in the meantime without anyone catching it.
The exploit they describe is mundane. A traveler flying through Frankfurt could skip the transit visa and its cost. In another case, a captain assigned to a NATO command in Germany could claim time off for ten hours of ordered overtime that a void rule had taken away.
What does this have to do with my contracts?
Now to be fair, their study looked at German regulations, not contracts. But my decades of academic research into this and experience developing software for legal analysis has shown me just how many of these defects exist in practice. Chances are that your own standard terms have the same kind of defect, for the same reasons.
Think about how a template ends up in your office. Somebody wrote it years ago, and that person may not work there anymore. It's been reused on solicitation after solicitation. It's been amended a few times, usually by adding a clause or swapping a number. And nobody has read the whole thing with hostile eyes, because nothing ever went wrong.
That's exactly the profile of the German rules. I'll often hear people say "we've used it for years, it works" as an excuse of complacency. But history shows that long use is no guarantee of correctness, as software engineers have learned the hard way.
Software went through this recently. Google pointed AI-written tests at OpenSSL, which it calls "critical and well-tested," and found a vulnerability that it says had likely been present for two decades. Years of heavy use hadn't caught it, because nobody had looked in the right way. Just because it's been around forever, doesn't mean it's perfect.
Who's going to look first?
The Stanford researchers end on a warning for legislators. If cheap legal reasoning spreads, they write, lawmakers would need to "significantly harden" how they write laws, possibly by using AI themselves.
Nobody is sending that warning to procurement offices or contract managers. But the same tools are available to a vendor's counsel, to a bidder who lost, or to anyone with a reason to dispute your agreement. I'd rather you find the bug in your template than read about it in their protest letter.
How do you find the bugs in your own templates?
Here's where I'd start. You don't need a new system for any of this, just someone (or something) willing to read the document the way an opponent would.
-
Start with your oldest template. Pick the one nobody remembers writing, because it has gone the longest without a hard read.
-
Check every reference. Make sure each statute, section, exhibit and attachment a clause points to still exists and still says what the clause assumes. The German bugs were all citation failures, and a contract's version is a clause that points at something that moved.
-
Check your defined terms. If "Contractor" means one thing in Section 2 and something slightly different in an exhibit, the other side gets to pick the meaning it likes.
-
Re-check after every amendment. The Mali rule survived two later edits. Changing one clause doesn't mean anyone verified the rest.
-
Ask AI to attack it, then decide yourself. Give the model your template and ask how a vendor could use each clause against you. Treat what comes back as a list of places to read, not as a ruling. You make the call, and any change goes in as something you approved.
This is the kind of work I built SquarePact to do inside Word. You can save the checks your office already runs as plain-English steps and run them on any document. The report tells you how much of the document it covered, and nothing changes in the file until you click Apply.
FAQ
Can
AI really find errors in legal documents?
Yes. The Stanford report's authors used
large language models to find void provisions in German regulations that, as far as they know,
nobody had recorded before. They published three examples with the exploits.
Did
the study look at contracts?
No. It looked at German regulations. The step from
there to contracts is my view, based on the fact that templates are written, reused and amended
the same way.
What's the most common kind of hidden contract bug?
The study can't answer
that for contracts. In the German rules, every example was a document citing the wrong
authority. In a contract, my first check would be clauses that point to sections, exhibits or
statutes that have since changed.
Should I let AI fix my contract templates?
I wouldn't let it change anything
on its own. Use it to point you at the places worth a second read, and approve each change
yourself. SquarePact always suggests and previews changes rather than making them automatically,
so you remain in control.
If a
template has worked for years, isn't it safe?
No. One of the German rules went
about 11 years and two amendments without anyone catching its defect. Years of use only show
that nobody has fought over it yet.
Each time we add a check like this one to SquarePact, I explain on the SquarePact mailing list what it catches and how to use it. Join the list to get the next one.
To try it now, install SquarePact in Word.