How Well Do Large Language Models Serve as End-to-End Secure Code Agents for Python?

The rapid advancement of large language models (LLMs) such as GPT-4 has revolutionized the landscape of software engineering, positioning these models at the core of modern development practices. To fully realize their potential in producing secure source code autonomously, LLMs must not only generate code but also identify and repair vulnerabilities in their outputs, thereby improving security iteratively. Despite growing prominence, LLMs’ effectiveness in performing such end-to-end tasks remains unexplored. This paper bridges this gap by systematically investigating the capability of LLMs to generate source code, evaluate their own outputs for vulnerabilities, and apply necessary repairs to improve the security of their self-generated code. Specifically, we studied the ability of GPT-3.5 and GPT-4 to identify and repair vulnerabilities in the code generated by four popular LLMs including themselves (GPT-3.5, GPT-4, Code Llama, and CodeGeeX2). By manually or automatically reviewing 4,900 pieces of code, our study reveals that: (1) LLMs generate over 75% vulnerable Python code in given scenarios; (2) LLMs such as GPT-3.5 and GPT-4 are unable to precisely identify vulnerabilities in the code they generated; (3) GPT-3.5 and GPT-4 can achieve 33.2% ∼ 59.6% success rates in repairing the insecure code produced by the 4 LLMs, but they both perform poorly when repairing self-produced code, indicating self-repair “blind spots". To address the limitation of a single round of repair, we developed a lightweight tool using LLMs as agents to construct safer source code through an iterative repair procedure based on the insights gained from our study. Experiments show that, assisted by semantic analysis engines, our tool significantly improves the success rates of repair to 65.9% ∼ 85.5%.