In a new post, the UK government agency urged caution when building integrations with LLMs into services or businesses. The NCSC said AI chatbots occupy a “blind spot” in our understanding, and the global tech community “doesn’t yet fully understand LLM’s capabilities, weaknesses and (crucially) vulnerabilities. ”The NCSC noted that while LLMs are fundamentally machine learning technologies, they are showing signs of general AI capabilities – something academia and industry are still trying to understand.A major risk highlighted in the blog was prompt injection attacks, in which attackers manipulate the output of LLMs to launch scams or other cyber-attacks. This is because research suggests that LLMs inherently cannot distinguish between an instruction and data provided to help complete the instruction, said the NCSC.This can lead to reputational risk to an organization, such as chatbots being subverted to say upsetting or embarrassing things….

Be Cautious of Latest AI Trends The NCSC also highlighted the risks of incorporating LLMs in the rapidly evolving AI market. Therefore, organizations that build services that user LLM APIs “need to account for the fact that models might change behind the API you’re using (breaking existing prompts), or that a key part of your integrations might cease to exist.”