How to audit the security of AI-generated code

By the QuantumSec team

You have a repository full of code generated by GitHub Copilot, Cursor or ChatGPT. Or maybe you're not even sure which parts a human wrote and which parts the AI generated. Either way, you need to know whether that code is secure before it reaches real users. This guide explains how to do it systematically.

Step 1: Identify the highest-risk code

Not all code carries the same risk. Prioritise reviewing: code that handles authentication and sessions, code that accesses databases, code that processes user input, code that manages files or executes system commands, code that handles payments or financial data, and code that interacts with external APIs. Within these areas, pay special attention to code generated in bulk (it usually comes with a consistent style and no comments explaining design decisions).

Step 2: Automated static analysis (SAST)

SAST tools analyse the source code looking for known vulnerability patterns without executing it. The best options for AI-generated code: Semgrep (open source, highly configurable, excellent for detecting LLM-specific patterns), Snyk Code (integrated with GitHub/GitLab, good detection of injections and secrets), CodeQL (GitHub, powerful but more complex to configure), and Bandit (Python-specific, very effective at detecting common insecure patterns). None of these tools replace manual review, but they eliminate the obvious patterns.

Step 3: Manual review for LLM patterns

LLMs have specific vulnerability patterns that automated scanners sometimes miss. Manually review: database queries (does it use a parameterised ORM or string concatenation?), authentication implementations (do tokens expire? is the signature validated?), error handling (are implementation details exposed in error messages?), secrets management (are there variables named "key", "secret", "password" hardcoded?), and authorisation logic (is the user's permission verified before every action, not just at login?).

Step 4: Dynamic testing (DAST)

Static analysis finds what it can see in the code. Dynamic testing uncovers how the system behaves while it's running. Key tools: OWASP ZAP (open source, automated web application scanning), Burp Suite (the industry standard for manual web application testing), and Nuclei (templates for known vulnerabilities). For web applications, always manually test the functionality that handles sensitive data.

FAQ

How long does an audit of AI-generated code take?

It depends on the size of the repository and the complexity of the application. For a typical SaaS MVP (20-50k lines of code), a full audit including static analysis, manual review of critical areas and basic dynamic testing takes an expert between 2 and 5 days of work.

Can I audit the code myself or do I need to hire someone external?

Part of the audit (SAST, basic pattern review) you can do yourself with the right tools. For a full assessment that includes exploitation testing and business-logic analysis, you need an external perspective: someone who doesn't know the code and thinks like an attacker.