The numbers
There is a lot of opinion about whether AI-written code is safe. There is also, now, a reasonable amount of measurement. Three findings do most of the work.
In the same Veracode study, models failed to defend against cross-site scripting in 86% of the tasks that called for it, and against log injection in 88%.
The same scan found sites where anyone with the public API key could delete rows, modify records, or read tables containing emails, hashed passwords, tokens and phone numbers. The cause in almost every case was the same: a database designed to be reached directly from the browser, protected by access rules that were never written.
Why it happens
Not because the models are bad at writing code. They are good at it, which is precisely the problem: the output works, and working code is the only signal most founders have.
- A model solves the problem it was given. “Let users edit their profile” produces an endpoint that edits a profile. It does not produce the check that the profile belongs to the person asking, because nobody mentioned an attacker.
- Security is invisible in the demo. A missing authorization check has no symptom. The feature works, the tests pass, the app looks right. You only discover it when someone who is not you changes an id.
- The generated stack pushes work to the client. Modern AI-built apps talk to the database from the browser. That architecture is safe only when row-level rules exist. Generators reliably produce the queries and unreliably produce the rules.
- Speed compresses the review that used to happen. Code that took a team three weeks got looked at by three people. Code that took one person an afternoon gets looked at by one person, who was not looking for this.
The five failure patterns we see most
- Broken object-level access control. An id in a URL or a request body is trusted. Changing it returns someone else's record. This is the single most common serious finding, and it is usually one server-side line away from being fixed.
- A database with no access rules. The public key was never meant to be a secret, and the policies that were meant to do the protecting do not exist. Every table is readable, and often writable, by anyone who opens the network tab.
- Secrets in the browser bundle. A provider key, an admin token, or a service credential that was “temporarily” put in a client-side environment variable and shipped.
- Endpoints with no cost control. Anything that sends an email, generates a file or calls a paid model, reachable without a session and without a rate limit. This one does not leak data; it just arrives as an invoice.
- Production still wearing development defaults. Debug mode on, stack traces in responses, a permissive CORS policy, no security headers, and a preview deployment from six weeks ago still serving an admin panel.
What it costs when it goes wrong
Those averages are dominated by large organisations, and an early-stage product will not lose five million euros. The relevant costs at your scale are different and still severe: notifying every affected user, the regulatory exposure if any of them are in the EU, the enterprise deal that quietly stops replying, and the fact that a small team usually loses weeks — not hours — to an incident, because nobody has done one before.
The asymmetry is the whole argument. Finding the problem costs an afternoon and less than a dinner. Finding out the other way costs whatever the person who found it decides it costs.
What to actually do
In rough order of value per hour spent:
- Check your data-layer rules today. If your frontend talks to your database directly, open the network tab, take the public key, and try to read a table you should not be able to read. This is a ten-minute exercise with the highest expected value of anything on this list.
- Test one id. Create two accounts. Sign in as the first, take a record id belonging to the second, and request it. If it comes back, you have found the most common serious flaw in AI-built software.
- Search your client bundle for keys. Anything named secret, token, key or password that ships to the browser is public.
- Ask your coding agent to review it. Genuinely useful, and free. Just know what it cannot see.
- Then have the running application tested independently. That is what we do, from €59, and it is deliberately the last item rather than the first: the four above cost nothing and you should do them regardless.
If you want the structured version of steps one to four, the pre-launch security checklist is twenty items you can work through yourself, free and without an account.