Is AI-Generated Code Secure? Risks and How to Gate It

No, not by default. In Veracode's 2025 GenAI Code Security Report, AI-generated code introduced security vulnerabilities in 45% of the tasks tested, and code models regularly invent package names. Treat AI output as untrusted input: human review plus failing CI gates for SAST, secret scanning, dependency (SCA) and IaC checks before anything merges.

AI coding assistants now write or influence a large share of the code shipping to production. They are fast, tireless, and genuinely useful. They are also confidently wrong about security in ways that a junior engineer usually is not, because they will produce a plausible, well-formatted, insecure implementation and never flag their own uncertainty.

The most-cited number comes from Veracode, which ran 80 coding tasks against more than 100 models and found insecure code in 45% of cases; Java failed more than 70% of the time, and models failed to defend against cross-site scripting in 86% of relevant samples. Treat any single figure as directional. The exact rate depends on the model, the language, and the prompt. The point stands regardless. A meaningful fraction of what these tools emit is exploitable, and none of it arrives labeled.

Why AI-generated code is often insecure

Three structural reasons explain most of the risk.

Training data: Models learn from vast public code, and public code is full of insecure patterns. Tutorials concatenate SQL strings because it reads cleanly, and short answers skip authorization checks to stay on topic. The model absorbs the average of what it saw, and the average is not secure.

Missing context: The assistant does not know your threat model, your trust boundaries, or which function runs behind authentication. It optimizes for code that looks correct in isolation.

Confident plausibility: The failure mode that matters most. AI produces security-relevant code (crypto, token handling, access control) with the same fluent confidence it uses for a loop. Plausible-but-wrong security code is worse than obviously broken code, because it survives a casual glance and reaches review looking finished.

The common flaw classes

Across languages, the same categories show up again and again:

• Injection: SQL, command, and template injection from string concatenation instead of parameterized queries.
• Missing authorization: Endpoints that authenticate the user but never check whether that user may act on the requested resource.
• Hardcoded and weak secrets: API keys inlined into source, default passwords, tokens with no rotation.
• Insecure deserialization: Reaching for pickle, yaml.load, or native Java serialization on untrusted input. Python's own docs say unpickling malicious data can "execute arbitrary code during unpickling."
• Vulnerable dependencies: Suggesting an old package version with known CVEs, or a library abandoned years ago.
• Weak crypto: MD5 or SHA-1 for passwords, ECB mode, hardcoded IVs, homegrown token schemes.
• Missing input validation: Trusting request bodies, headers, and path parameters as-is.

Here is a representative AI-style snippet. It is idiomatic, it runs, and it is vulnerable to SQL injection.

# AI-generated: looks fine, is not
def get_user_orders(db, user_id):
    query = "SELECT * FROM orders WHERE user_id = " + user_id
    return db.execute(query).fetchall()

The fix parameterizes the query and does not trust the caller to have validated the input:

# Reviewed and hardened
def get_user_orders(db, user_id: int):
    query = "SELECT * FROM orders WHERE user_id = ?"
    return db.execute(query, (user_id,)).fetchall()

Same behavior, one is exploitable and one is not. The difference is exactly the kind of thing a reviewer or a scanner catches and an autocomplete does not.

The newer risk: package hallucination

Supply chain is where AI introduces a genuinely new attack surface. Models sometimes invent dependencies. Asked for a library to do X, an assistant may confidently import a package that does not exist, because a name that fits the pattern is statistically likely even when it is not real. A USENIX Security 2025 study of 16 code models and 576,000 generated samples found hallucinated packages in at least 5.2% of output from commercial models and 21.7% from open-source ones, with 205,474 unique invented names.

Attackers noticed. They watch for commonly hallucinated names, register those packages on public registries, and wait for developers to paste AI output and run install. The malicious package then executes in your build, often through an install script. This is slopsquatting, a variant of typosquatting driven by model hallucination rather than fat-fingered typing. Any unfamiliar dependency an assistant suggests deserves a look at the real registry, the download counts, and the maintainer before it enters your lockfile:

$ PKG=name-the-assistant-suggested
$ pip index versions "$PKG"                          # does it exist on PyPI at all?
$ npm view "$PKG" time.created maintainers repository  # age, owners, source link
$ npm install --ignore-scripts "$PKG"                # no install hooks on first contact

A package created last week with one maintainer and no repository link is not something an assistant "knew about"; it is something an attacker hoped you would install.

How to gate it

The goal is not to ban AI coding. It is to route AI output through controls that assume it is untrusted, because it is.

• Mandatory human review: No AI-generated change merges without a human owner who understands it. Review is where missing authorization and business-logic flaws get caught, because scanners cannot see intent.
• SAST and secret scanning in CI: Run static analysis and secret detection on every pull request and fail the build on high and critical findings. Do not leave these advisory.
• SCA on every dependency: Software composition analysis flags vulnerable and hallucinated packages before they land. See dependency and SCA scanning for wiring this into the pipeline.
• IaC scanning: AI writes plenty of Terraform and Kubernetes manifests, and it will happily open a security group to the world. Scan infrastructure as code with the same failing gates.
• Guardrails and policy: Codify standards (parameterized queries, approved crypto, no inline secrets) as lint rules and policy-as-code so the machine enforces them, not tribal memory.

Grounding these gates in secure coding best practices keeps reviewers and rules aligned on the same standard.

Metrics that reflect the shift

When AI writes a large slice of your code, "vulnerabilities found" stops being a useful north star. Volume goes up because output goes up, and review becomes the bottleneck, as we covered in why AI agents write more code but teams don't ship more. Shift the scoreboard toward leading indicators: guardrail coverage (the share of repositories whose pipelines enforce SAST, SCA, secret and IaC gates), the share of AI-introduced flaws caught pre-merge rather than in production, and time-to-remediate. These measure whether your gates work, not whether your assistants make mistakes. They always will.

How to adopt AI coding safely

Order matters. Turn the gates on first, failing on high and critical findings, before you scale up AI usage. Then set the ownership rule: the human who merges AI code owns it, and "the model wrote it" is not a defense. Feed real findings back into prompts and lint rules so the same flaw doesn't recur.

Questions people ask

Is code written by ChatGPT or Copilot safe to use in production?

Only after the same checks you apply to any contributor. Assistants produce code that compiles and looks idiomatic but often skips authorization, uses string-built SQL, or pins vulnerable dependencies. Veracode found flaws in 45% of its 2025 test tasks. Use it, but merge it only after human review and passing SAST, secret scanning and dependency checks.

What is slopsquatting?

Slopsquatting is registering a package name that AI coding assistants hallucinate, so developers who paste the suggestion and run pip install or npm install pull in malware. It is typosquatting driven by model output rather than typing mistakes. A 2025 USENIX Security study counted 205,474 unique hallucinated package names across 16 models. Check that every unfamiliar package exists, has history, and has a real maintainer.

Which tools catch vulnerabilities in AI-generated code?

The same ones that catch them in human code: a SAST scanner such as Semgrep or CodeQL for injection and unsafe APIs, secret scanning for inlined keys, SCA for vulnerable or nonexistent dependencies, and IaC scanners for open security groups. None of them sees missing authorization or broken business logic, which is why human review stays mandatory.

The call we'd make

AI-generated code is not secure by default, and it is not going away. Both things are true. Ban it and you lose the velocity; trust it and you ship the flaws. The durable answer is to let the assistants write, and let a hardened pipeline decide what merges. Assume every AI-produced line is untrusted until a human and your scanners have signed off. Teams that build those gates get most of the speed and very little of the risk. Teams that skip them are shipping Veracode's 45% straight to production.

For the broader program, start with our AI security guide and build the gates outward from there.

Sources

• Veracode: AI-generated code poses major security risks in nearly half of all development tasks (July 2025)
• Spracklen et al.: We Have a Package for You! Package hallucinations by code-generating LLMs (arXiv 2406.10279)
• Python docs: pickle security warning
• OWASP Top 10 for Large Language Model Applications