Business Owners Have a New Security Problem: AI Agents With Keys to Company Secrets

While the OpenAI rogue agent that hacked Hugging Face has become a poster child for the latest AI threat, it was hardly alone. Days later, Anthropic announced that its AI models also went rogue, hacking external organizations. And a few days after that, Meta announced that one of its agents had also accessed the internet and hacked a third-party service. The incidents have raised concerns about whether AI can be controlled by its creators and what will happen as the technology continues to get smarter. Dylan Ayrey, co-founder and CEO of open-source security software company Truffle Security, and Feross Aboukhadijeh, founder and CEO of security infrastructure firm Socket, recently joined the a16z podcast to discuss what these attacks signal and how AI is in the process of entering a new era. That could mean developers and business owners have to rethink system security. Here are some of the top takeaways from the discussion. The barrier for hacking is a lot lower While the AI rogue agents haven’t invented any new hacking techniques, they have made existing methods much more accessible. “Everyone needs to worry about these models making it materially easier to hack into things,” said Ayrey. “The bar previously [for hacking] was just subject matter expertise—and now the models have the subject matter expertise.” Put another way: Wannabe hackers today simply have to ask the model, which has been trained to hack into things, to do it for them. And that puts businesses and individuals at greater risk. Low hanging fruit is the best target The AI models are goal oriented, Ayrey said. Their objective is to fulfill a request and they’ll use any cybersecurity technique they need to in order to achieve their objective. That’s not unlike human hackers, in a way. Hacker collectives generally look for targets that have unpatched vulnerabilities, as it saves time and effort. AI is similarly lazy when it comes to breaching a system, only the technology has a more extensive toolset than a typical human hacker. “They will do the path of least resistance to accomplish the task,” Ayrey said. “And that includes drawing on their cybersecurity expertise.” One way that attackers, including AI models, can find that low hanging fruit, said Aboukhadijeh, was by exploiting the trust developers have in software, rather than launching an attack on an individual system. “Get developers to install that, and then you could use the access stolen from those developers as they install it to self-propagate the worm,” he said. It’s not emergent behavior The behavior that we’re seeing from these AI models isn’t an emergent intelligence, said Ayrey. It’s a flaw in how they were trained. “If a lab tells you that this is an emergent super intelligence behavior, they’re just lying to you,” he said. The AI hacking risk isn’t limited to machine-driven attacks, either. Human hackers can exploit AI tools that developers use, rather than traditional malware, to end-run a system. That will bypass much of the EDR (Endpoint Detection and Response) tooling security systems have in place, since the AI has permissions to access data. There’s a growing need for better authentication and patching Aboukhadijeh brought up pending change in npm, the digital toolkit programmers use to build software. In January 2027, it will no longer run or publish updates automatically. A human will need to approve them, which is meant to stop hackers from sneaking malware into systems. “It’s going to be super disruptive, … but I think it’s the right call,” he said. Ayrey added that the accelerating speed of vulnerability discovery via AI makes current patching processes inadequate. Engineering teams can’t be depended on to make a complicated upgrade every time a vulnerability appears. “The frontier models are causing kind of a massive reduction in the time between the vulnerability discovery and vulnerability exploitation,” he said. “What we need to start thinking about is: How do we patch more quickly?” Protecting agents is the new Wild West AI agents, as the technology advances, are going to potentially have access to a large number of credentials. While that has some conveniences, it also presents a growing risk. And the industry isn’t quite sure what to do about it yet. As a result, the security problems that IT departments face is about to expand from just shielding human users and their credentials to protecting those AI agents and the secrets they have access to. “The way agents interact with secrets right now is a Wild West unsolved problem that we’re working very hard to solve,” said Ayrey. BY CHRIS MORRIS @MORRISATLARGE