AI Built the Car. Nobody Checked the Brakes.

A regional auto parts supplier called me last year — about 22 locations, 300 or so employees. The ask sounded routine. They wanted help standing up a development domain to test a new tool their team had built: a parts-fitment lookup, the thing that tells a counter clerk whether a given brake rotor fits a 2018 Silverado. They just needed a safe place to try it before going live.

They sent over the spec sheet so I could size the environment. I started reading. Something felt off.

The document was too clean. Too complete. Every section perfectly parallel, every edge case enumerated, the kind of exhaustive symmetry no tired developer produces at 4 p.m. on a Thursday. It read like it had been generated. Because it had.

So I asked the obvious question. Who wrote the app?

Claude Code wrote the app. Their developer had prompted his way through the whole thing over a few weeks. No security review during the build. No framework decisions made up front. No guardrails of any kind — just prompt, generate, run, prompt again. The result was roughly 35,000 lines of code. And here’s the thing that made everyone comfortable: it worked. The lookup returned the right parts. The demo was smooth. Everybody was ready to test.

Here’s the part that matters. “It works” was doing an enormous amount of load-bearing in that sentence, and nobody on their side could see how much.

I hit pause on the dev domain request. Setting up a test environment was the least of their problems, and spinning one up would have moved a set of risks nobody had counted one step closer to production. Before I’d help them test the thing, I wanted to look inside it.

They sent me a copy. What I found is the reason I’m writing this — not because AI wrote bad code, but because of what 35,000 unread lines actually is when you’re the one who has to answer for them.

The Manual Nobody Read

The first thing I did was run the code through Snyk, a security scanner that flags known vulnerability patterns. I wasn’t expecting a clean report. I wasn’t expecting what came back either.

SQL injection — the classic hole where a malformed search string can rewrite the database query underneath it. Authentication bypass. Cross-site request forgery. Cross-site scripting. The full catalog, spread across the codebase like weeds. Not one careless mistake in one file. A pattern, repeated everywhere the app touched user input or checked who was allowed to do what.

That alone wasn’t a surprise. When you generate code by prompt without anyone owning security as the code is written, this is roughly what you get. A formal-verification study out of a group of researchers, testing multiple large language models, found that 55.8% of AI-generated code samples contained at least one provable security vulnerability — and that telling the model to write secure code only dropped the rate by about four points. The same study found that six commercial scanning tools, combined, missed nearly 98% of the flaws a formal prover caught. So the scan I ran was the floor, not the ceiling.

Over half of AI-generated code samples contain at least one provable security vulnerability — and the commercial scanners miss most of them. The scan is the floor, not the ceiling.

But the vulnerabilities weren’t the part that made me sit back.

The part that made me sit back was how much code there was in the first place. The AI had reinvented the wheel — over and over. Thousands of lines of custom database-handling code. A hand-rolled authorization system deciding who could see what. Custom routing, custom session logic, custom input handling. All of it built from scratch.

Every one of those things is a solved problem. Mature frameworks handle database access, authorization, and request routing with code that thousands of developers have hardened over a decade. The AI ignored all of it and wrote its own version, badly, in the middle of an app about brake rotors.

That’s the trap. Each custom class the model wrote was another place a vulnerability could live, another few hundred lines nobody would ever audit, another wheel that was rounder and safer the moment it was invented but was now going to be square forever.

The Engine Powering the Demo

Now, if you’re a business owner reading a list of acronyms, your eyes glaze. SQL injection, CSRF — so what? It’s an internal parts-lookup tool. It has its own domain. It has access controls. Only employees touch it. Where’s the fire?

Here’s where. The tool shared a database with the live e-commerce store.

That single architectural decision changed everything. The parts-lookup app was framed as low-stakes — internal, boring, a convenience for the counter staff. But it reached into the same database that held customer orders, account records, and payment-adjacent data for the public-facing store. The authentication bypass I found in the “low-stakes” tool wasn’t a door into a parts catalog. It was a door into customer data, propped open by an app nobody thought was worth securing.

An app’s blast radius isn’t defined by what it’s for — it’s defined by what it can reach.

This is the gap between a reassuring signal and real proof. “It’s internal” is a signal. “It has its own login” is a signal. Neither one is proof of anything, because the app’s blast radius isn’t defined by what it’s for — it’s defined by what it can reach. And this one could reach the store.

Put a dollar figure on the wrong side of that door. IBM’s 2025 breach research, summarized by the security firm Vectra AI, puts the average U.S. data breach at $10.22 million — an all-time high. I’m not saying this parts supplier was staring at a ten-million-dollar loss. I’m saying nobody on their side had done the math at all, because everyone was looking at the tool and nobody was looking at the database behind it.

The demo worked. That was never the question. The question was what the working demo was quietly connected to.

Nobody Behind the Wheel

Strip away the acronyms and the dollar figures and you’re left with the real problem, the one that has nothing to do with AI specifically.

Thirty-five thousand lines of code existed, and not one named person understood them.

The developer had directed the AI, but “directed” is not “understood.” He could tell you what the app did. He could not tell you how the authorization logic actually decided who got in, because he didn’t write it and had never read it line by line. The AI wrote it. The AI moved on. The knowledge of how those 35,000 lines actually behaved lived nowhere — not in a person, not in documentation, not in a design anyone had reviewed.

That’s the accountability gap, and it’s worth naming plainly. The failure here was never that AI writes insecure code. The failure was that no single person owned understanding what got shipped. Responsibility was diffused across a developer, a tool, and a spec that read like it wrote itself.

Every line of code is not an asset. Every line is surface area — something to secure, something to maintain, something that has to fit in somebody’s head when it breaks at 2 a.m.

This is where AI quietly flips on you. It generates volume, and volume looks like progress. Thirty-five thousand lines feels like a lot got done. But every line of code is not an asset. Every line is surface area — something to secure, something to maintain, something that has to fit in somebody’s head when it breaks at 2 a.m.

A human developer writing this app by hand would have produced a fraction of the code, because humans are lazy in the useful way. They reach for the framework. They don’t reinvent the database layer at 4 p.m. on a Thursday. The AI has no such fatigue. It will happily generate a custom solution to a solved problem, and it will do it fast, and the speed feels like a gift right up until you realize you now own thirty-five thousand lines nobody can vouch for.

Volume disguised as output. Liability disguised as progress. And one linchpin developer expected to hold it all.

I Left My Own Engine Running

I’d like to tell you I only ever watch other people fall into this. I don’t. I did it to myself a few months back.

I had a simple job. I wanted to pull server data and log files together into one place so I could look at them side by side. Small task. A consolidation script. I wrote a prompt describing what I wanted, kicked it off, and walked away to do something else. Fire and forget.

I came back two hours later. It was still running.

Not stuck — running. Working. In the time I’d been gone, the tool had decided my modest consolidation request deserved an entire log-correlation engine. It had built handling for log formats I never mentioned, normalization for timestamp variations across systems I didn’t ask about, parsers for edge cases that would never occur in my environment. It was thorough. It was impressive. It was enormous. And roughly ninety percent of it solved problems I did not have.

Same root cause as the parts supplier, on a smaller stage. I gave a loose instruction and walked off, and the tool sprinted far past the finish line I’d drawn — because I never actually drew one.

That’s the lesson I took, and I’m not scolding anyone from a hilltop when I say it. These tools build well past what you ask. Left unsupervised, they don’t do the small thing you requested; they do the biggest thing your request could possibly imply. You cannot fire and forget. You have to set the fence before you start, or you come back to a machine that built a factory when you wanted a shed.

Factory Parts Beat Custom Fabrication

So what did I actually recommend to the parts supplier? Not “stop using AI.” The fix was more specific than that.

The app was written in PHP, so I recommended rebuilding it on Symfony — a mature, widely used PHP framework. Not a rewrite for the sake of rewriting. A rewrite that let us delete the reinvented wheels and stand on hardened ones instead.

Roughly 35,000 lines became roughly 6,000. Six thousand lines a person can actually read, review, and own.

The results told the story better than I can. The rewrite closed nearly all of the security findings, because the framework handles database access, authorization, and input validation with code that’s already been vetted. And the size collapsed. Roughly 35,000 lines became roughly 6,000. Six thousand lines a person can actually read, review, and own.

Here’s why frameworks win, and it’s the whole blueprint in one idea. A framework’s security code has been audited by a large community, maintained across years, and patched fast when someone finds a hole. That’s the opposite of custom code written once by a tool and never read again. OWASP’s Proactive Controls guidance is explicit about this: use well-supported security frameworks rather than writing your own, because they embed protections against common vulnerabilities and get patched by the crowd. Microsoft’s secure-development training says the same thing.

And the maintenance advantage is measurable, not just philosophical. An empirical study of the Maven package ecosystem — over 14 million library versions analyzed — found that widely used, frequently released libraries fix their vulnerabilities faster than obscure ones. More users, more eyes, faster patches. Your hand-rolled authorization class has exactly one user and zero eyes.

The blueprint move is simple to say and easy to skip: don’t let AI reinvent solved problems. When the tool starts writing its own database layer, that’s not productivity. That’s your cue to point it at the framework instead.

Always Test Drive Your Code

After I flagged all this, the company’s CTO — they did have one, which is more than a lot of firms this size can say — asked me the right question. Not “was AI the mistake?” He asked how they should be using it.

My answer surprised him a little. Use AI anywhere you want. I mean that. I’m not the advisor who tells you to ban the tool that just cut your development time in half. Use it in the parts lookup, use it in the store, use it wherever it helps — as long as someone manages it.

The trap he’d nearly fallen into was reasoning from an AI standpoint: what can the AI build, how fast, how much. Wrong starting point. The right starting point has nothing to do with AI at all.

Before any code — AI-written or not — goes anywhere near production, ask two questions:

1. What breaks the business if this code is compromised? Not “is it internal,” not “does it have a login.” What can it actually reach, and what happens to the company if someone walks through it? For that parts tool, the honest answer was “our customer database,” and that answer should have set the security bar on day one.

2. Who owns that answer? Not who wrote the code — who understands it well enough to be accountable for it when it fails. If you can’t name the person, you don’t have a guardrail. You have a hope.

Everything else — the framework choice, the security scan, the code review — flows from those two questions. Get them backwards and no amount of tooling saves you. Get them right and AI becomes what it should be: a fast worker on a short leash held by someone who’s paying attention.

It Was Never About Who’s Driving

The debate everyone wants to have is “AI or human?” It’s the wrong debate. It was always the wrong debate.

The question was never who — or what — writes the code. It’s who understands it, and what breaks the business if it’s compromised. That question predates AI by decades. Outsourced dev shops raised it. Offshore contractors raised it. The lone developer who quit and took the whole system in his head with him raised it. AI didn’t create the accountability gap. It just generated enough volume that you can no longer pretend the gap isn’t there.

And that’s almost good news. Because AI removes the last excuse for not owning your code. It’s cheap — the scanners are free or nearly so, the frameworks are free, the review is a few hours of someone’s attention. Cost was never the obstacle. The obstacle is deciding it matters enough to put a name next to it.

Here’s the part that should change how you think about this. An outsourced dev team gets more expensive and more distant over time. An AI-built codebase does the opposite — it gets easier to maintain as the models improve, as the tools get better at reading and refactoring what they wrote. But only if someone owns the guardrails. Without an owner, better models just mean you generate liability faster.

So the parts supplier didn’t have an AI problem. They had six thousand lines of clarity buried under twenty-nine thousand lines of nobody’s-job.

Which raises the only question worth sitting with: if I asked you today who owns understanding the code your business runs on — could you say a name?