Calls for slowing down frontier AI development due to cyber attacks raise stakeholder concerns

Anthropic CEO Dario Amodei wrote in a Sept. 12 open letter , “I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models.

Progress will still seem fast, and we must make wise use of the time we gain.” Amodei addresses the July Hugging Face incident where he writes, “a swarm of agents essentially acted as a fanatically devoted collective , conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the ‘grader’ responsible for evaluating their performance.” “It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage,” according to Amodei. Amodei proposes a three-step plan to pace the development of frontier models including adding a team of third-party evaluators into each frontier AI company, frontier AI companies working together on common safety standards and United States and allies working with authoritarian governments to address “the challenges of verifying compliance.” OpenAI CEO Sam Altman wrote in a Sept. 14 social media post , “When we talk about ‘pacing’, we do not mean ‘stopping’.

Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.” Altman argued, “Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.” The R Street Institute’s Mark Dalton is pushing back on how such a pause would work in practice. Dalton told Inside Cybersecurity that Amodei is taking a “naïve approach” in the slowdown and it “presents just as many problems as solutions.” R Street is a free market think tank.

Frontier AI labs can’t “dial back on speed and capability, especially when it is a global research area. This isn’t just happening in two companies in Silicon Valley. It’s happening across the world and at a variety of different levels so there’s no such thing as a global slow down on the frontier,” Dalton said.

Dalton backed the use of AI-based tools for defenders, arguing that it can “help them to react to the new attack vectors that AI can present. AI is a problem from a cyber perspective but it’s also a solution.” But Leah Siskind of the Foundation for Defense of Democracies said pacing the development of frontier AI models can make sense because so much has happened over the past six months with the launch of Claude Mythos and Fable 5, as well as the Hugging Face incident and other impacts. “There’s just so much to process and it’s becoming clear that we are not fully in control, either technically we are not able to be in control or we don’t have the systems set up in a secure enough way that we are able to exercise full control,” Siskind told Inside Cybersecurity .

Siskind argued it is “abundantly clear that things are moving too fast.” She said, “We need time, not to stifle innovation, but time to react to the progress that has already been accomplished in the last six months and figure out are these tools really ready for primetime. Do we have adequate measures in place to control them when we need to?” Aaron Cooper of the Business Software Alliance meanwhile told Inside Cybersecurity , “Each company can decide what they are going to do and how they are going to develop and deploy AI. The cybersecurity-related issues remain just as present” to any company’s frontier AI cybersecurity model that makes “the leap” of having advanced capabilities like those developed by OpenAI and Anthropic.

BSA wants a “phased rollout” for models that have “dramatically improved abilities both to identify vulnerabilities but also if in the wrong hands to identify and exploit vulnerabilities,” Cooper said. Cybersecurity defenders should “get access to the model ahead of time as part of the phased rollout to make sure we are remediating risks and vulnerabilities before the model is more widely deployed,” Cooper said. BSA pitched on Sept.

9 a new framework to establish a process for reviewing advanced AI security models. Cooper said, “The AI that is available today has new capabilities that did not exist before and that AI is important for improving cybersecurity around the world.