LLMs in Coding and Their Impact on the Commercial Software Engineering Landscape
摘要
Large language model coding tools are now mainstream in software engineering. But as these same tools move human effort up the development stack, they present fresh dangers: 10% of real-world prompts leak private data, 42% of generated snippets hide security flaws, and the models can even “agree” with wrong ideas, a trait called sycophancy. We argue that firms must tag and review every AI-generated line of code, keep prompts and outputs within private or on-premises deployments, obey emerging safety regulations, and add tests that catch sycophantic answers – so we can gain speed without losing security and accuracy.