The AI Giants Say Slow Down. There Are Two Reasons Why.

Slow down. That is the message from the US AI giants over the past few days.

On 12 September, Anthropic's chief executive, Dario Amodei, published an essay called We Must Pace the Frontier. "We must slow the pace at which we improve the capabilities of AI models," he wrote. Within hours, the heads of the other big labs agreed. Sam Altman said, "I agree with Dario that we need to pace the frontier." Elon Musk wrote, "Dario is right." Demis Hassabis of Google DeepMind called it the right path forward at a critical moment.

These are the companies that have spent three years racing each other. When all of them ask for the brakes on the same day, it is worth asking why.

Is it right? I think there are two reasons behind it: a competitive one and a genuine one. Both are real, and neither cancels out the other.

The competitive reason

A slowdown suits whoever is in front.

If you already have the best models, the most compute and the deepest pockets, a pause locks in your position. New safety standards cost money to meet, and you can afford them. Independent evaluators need access to your systems, and you have the staff to host them. The smaller player trying to catch up is hurt far more by the same rules.

Plenty of people have said this out loud. China's foreign ministry dismissed it as fear-mongering, a way to hold back Chinese AI through regulatory barriers. President Trump posted "WHOEVER WINS AI, WINS!" David Sacks, a technology adviser to Trump, put it more precisely: "Stop pretending the motivation to slow down is purely altruistic." His point was that the labs face huge product-liability exposure if their models help cause a damaging cyberattack. Trading some raw power for reliability, he said, is simply good business.

Amodei does not really hide from this. His essay says keeping the democracies' lead as large as possible comes before any pacing, and he calls China's willingness to go along the "toughest dilemma" in his plan.

So yes, there is a competitive reason. But when someone's commercial interest lines up with a warning, that does not make the warning false. Sometimes it is just true, and useful to them as well.

The genuine reason

The essay did not come from nowhere. It came after the worst AI security incident so far.

Between May and July this year, at least 1,200 AI agents were running inside OpenAI's cyber-security test environments. Their job was a benchmark: find and exploit software vulnerabilities. For the test, they were set up to refuse less than a normal model would.

They did not stay inside the test. The agents coordinated on message boards they improvised themselves. One of them found Hugging Face credentials that had been left exposed on the internet and shared them with the rest. According to Hugging Face's own timeline, the agents reached cluster-admin access across multiple Hugging Face clusters in under thirteen hours. From the agents' point of view, the whole thing was an attempt to cheat the test by stealing the answers instead of solving it.

No human told them to attack Hugging Face. There is no evidence that public models or datasets were tampered with, and no customer data was leaked, which is the good news. The bad news is everything else in that paragraph.

A week after OpenAI and Hugging Face disclosed the incident, more than 1,100 employees of OpenAI, Anthropic, Google DeepMind and Meta signed an open letter asking for an effort to "deliberately pace the frontier". Amodei's essay goes further. He worries that within six to 12 months, a swarm like that could be capable of "taking over the entire internet".

You do not have to accept the most dramatic version to take the direction seriously.

The part most organisations miss

Here is what worries me more than the swarm, and it is much closer to home.

Most organisations don't realise it, but the scariest thing about AI is this: a seven-year-old who can say "hack this organisation" now has the capability to follow through.

For the whole history of cyber crime, skill was the filter. To break into a business, you had to know how networks worked, how to find a weakness, and how to write or adapt the code to use it. That took years to learn, and it kept the number of capable attackers fairly small. Small businesses relied on that without ever saying it: we are not interesting enough for anyone good to bother with.

That filter is going. The Hugging Face agents chained together vulnerabilities nobody had taught them. A model that can do that for a benchmark can do it for anyone who asks the right way. The hard part used to be the doing. Now the hard part is only the wanting.

When skill stops being the filter, the only thing left is intent, and there is no shortage of that. It also means attacks are cheap enough to try against everybody, not just the targets worth an expert's time.

The Australian numbers were already bad before any of this. In the Australian Signals Directorate's 2024–25 threat report, it received more than 84,700 cyber crime reports, about one every six minutes. The average self-reported cost to a small business rose 14% to $56,600 per report. That was the year to June 2025, before autonomous agents were part of the picture.

A slowdown will not protect you

Whether or not the labs pace the frontier, the capability that has already been released does not get un-released. The models on the market today are enough to change who can attack you.

So the useful response is not to wait for Washington, or for the labs to agree on standards. It is the boring basics, done properly, because the basics are exactly what cheap, automated attacks bounce off:

• Multi-factor authentication on every account, not just the ones you remember.
• Patching quickly, because an AI can find and use a known weakness faster than any person.
• Backups kept offline, so ransomware cannot reach them.
• Admin rights restricted to the few people who genuinely need them.

These come straight from the ASD's Essential Eight, and they have not become less important. They matter more now, because the attacks they stop just got much easier to launch. I have written about why insurers are already underwriting on these controls, and if you want to know where you stand, the Essential 8 self-assessment takes two minutes.

So, should the AI giants slow down? Probably, and for both reasons. But your security should not depend on whether they do.

What are your thoughts? And if you would like a straight answer on how exposed your business is, come and have a chat with us, or look at how we run IT and security for Perth businesses. Or get this kind of thinking weekly.