Scholarlink

Code completion and generation tools are crucial in improving software development’s efficiency and overall quality by assisting programmers. On this basis, Amazon CodeWhisperer is a prominent Large Language Model (LLM) and code assistant tool trained on Amazon and open-source code. Amazon CodeWhisperer also stands out as the only AI coding companion with security scanning capabilities to identify and recommend solutions for vulnerabilities in code. However, given the open-source data CodeWhisperer was trained on and the probabilistic nature of LLMs, it is inevitable that part of the generated code will contain security vulnerabilities. In this work, we first evaluate CodeWhisperer’s generated code in regard to the Common Weakness Enumeration (CWE) list, where we found approximately 85% of the security-relevant scenarios to be vulnerable. Next, we tested the security scan tool, which was able to identify 62% of the demonstrably insecure code.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Protecting the Whisper: A Security Assessment of Amazon CodeWhisperer’s Generated Code

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Gabriel Ferreira Araujo,
  • Thaier Hayajneh

摘要

Code completion and generation tools are crucial in improving software development’s efficiency and overall quality by assisting programmers. On this basis, Amazon CodeWhisperer is a prominent Large Language Model (LLM) and code assistant tool trained on Amazon and open-source code. Amazon CodeWhisperer also stands out as the only AI coding companion with security scanning capabilities to identify and recommend solutions for vulnerabilities in code. However, given the open-source data CodeWhisperer was trained on and the probabilistic nature of LLMs, it is inevitable that part of the generated code will contain security vulnerabilities. In this work, we first evaluate CodeWhisperer’s generated code in regard to the Common Weakness Enumeration (CWE) list, where we found approximately 85% of the security-relevant scenarios to be vulnerable. Next, we tested the security scan tool, which was able to identify 62% of the demonstrably insecure code.