The head of the UK’s national cybersecurity agency has called for more safeguards around vibe coding, describing a “fundamental issue with the quality of technology we use”.
Richard Horne, chief executive at the National Cyber Security Centre (NCSC) said the international security community needs to balance using AI to reduce collective vulnerability against the risks the technology can introduce to software.
“The attractions of vibe coding are clear, and disrupting the status quo of manually produced software that is consistently vulnerable is a huge opportunity, but not without risk of its own,” he said.
“The AI tools we use to develop code must be designed and trained from the outset so that they do not introduce or propagate unintended vulnerabilities.”
Vulnerability management approaches are maturing, he said, but not quickly enough. Meanwhile, NCSC research shows that the rate of defects per line of code has remained broadly static – but that the software source code in systems doubles on average every 42 months, with zero day vulnerabilities regularly weaponized before organizations have been able to patch for them.
“The attractions of vibe coding are clear, and disrupting the status quo of manually produced software that is consistently vulnerable is a huge opportunity, but not without risk of its own,” he said.
Horne’s speech, at the RSAC Conference in San Francisco, coincided with the publication of a blog post warning that AI-generated code currently presents “intolerable risks” for many organizations.
Developers with little or no technical experience often produce code that’s unreliable, hard to maintain, or has critical issues.
Experienced coders, meanwhile, tend to find AI outputs far below the standard they require, while managers responsible for development teams’ efforts also report concerns about the quality and maintainability of AI-generated code.
Obvious safeguards, said the NCSC, include models that write code that is secure by default, and that haven’t been developed in such a way as to maliciously introduce issues in code it produces. The way AI is used to review code, both existing human-written code and that which will be written by AI, needs to be examined.
But there are also more nuanced considerations.
“How do we use a deterministic architecture (that is, known controls implemented in rules and code, rather than expecting an AI to limit another AI) to limit what code can do even if it is malicious, compromised, or unsafe?” the NCSC wrote.
“What platforms for hosting AI-generated services (and even human-generated ones) can we design to implement the controls above and protect the organisation and its data even if the code running is of poor quality?”
There are growing calls for more consideration of how AI can be used to carry out security hygiene such as documentation, test cases, fuzzing and permanently updating threat models for all software.
Late last year, a survey from Aikido found that one-in-five CISOs had suffered major incidents because of AI-generated code, with 69% of security leaders, security engineers, and developers across Europe and the US revealing they’d found serious vulnerabilities in AI-written code.
