Australian boards have spent the past two years pouring money and enthusiasm into artificial intelligence, racing to bolt chatbots onto customer service, feed sensitive data into copilots and let models make decisions that once sat with people. A pointed op-ed has now put a number on how little of that rush has been matched by preparation for when it all goes wrong. According to a piece published by Cyber Daily, just 4 per cent of Australian organisations regularly test their response to an AI-related cyber incident.
The claim lands at an awkward moment. Artificial intelligence has widened the attack surface of almost every business that has adopted it, creating fresh categories of trouble that did not exist a few years ago: prompt injection that tricks a model into leaking data, training sets quietly poisoned by an attacker, deepfake audio used to authorise fraudulent payments, and the simple but corrosive risk of a model confidently producing wrong answers that staff act on. Each of those failure modes behaves differently from the ransomware and phishing campaigns that incident response plans were written to handle, and rehearsing them is the only reliable way to find out whether a plan survives contact with reality.
Why testing matters more than plans
Security professionals have long drawn a hard line between having an incident response plan and knowing it works. A document that lives in a shared drive and is dusted off once a year for an audit is not the same as a tabletop exercise where executives, lawyers, communications staff and engineers are forced to make decisions under pressure, against the clock, with incomplete information. The op-ed’s core argument is that AI has not changed that principle so much as raised the stakes, because the speed and opacity of AI systems leave less room to improvise once something breaks.
The 4 per cent figure is meant to be uncomfortable. It suggests that the overwhelming majority of organisations deploying AI have never sat down and war-gamed a scenario in which the technology itself is the problem. For a model that makes thousands of automated decisions an hour, the gap between an incident starting and a human noticing can be enormous, and the absence of rehearsal means nobody has practised the awkward questions: who has the authority to switch the model off, how do you explain an AI-driven breach to a regulator, and how do you reconstruct what the system actually did.
Two ways to read the number
There is a straightforward reading of the statistic, which is that Australian organisations are dangerously complacent and need to treat AI incident rehearsal as a baseline discipline rather than a nice-to-have. On this view, the solution is cultural and procedural: run the exercises, involve the board, and stop pretending that a plan nobody has tested offers any real protection.
There is also a more sympathetic reading. Many Australian organisations, particularly small and medium businesses, are stretched thin on security talent and budget, and AI governance is a genuinely new discipline that few teams have the in-house expertise to exercise properly. Regular testing of AI-specific scenarios demands people who understand both the technology and the threat landscape, and that combination is scarce and expensive. Seen this way, the low figure is less a story of negligence than of a capability gap that the market and the training pipeline have not yet closed. Both readings point in the same direction, but they imply very different fixes, one about will and one about resources.
What it means for Australia
The warning does not arrive in a vacuum. The Australian Signals Directorate has repeatedly cautioned that AI is no substitute for the fundamentals of cyber defence, and the Cyber Security Act 2024 has begun to formalise expectations around incident reporting, including rules touching ransomware payments. Operators of critical infrastructure already sit under the Security of Critical Infrastructure Act, which carries risk-management obligations that a serious regulator would expect to see exercised rather than merely documented. Against that backdrop, an organisation that cannot demonstrate it has tested its response to an AI incident is exposed not only to the attack itself but to the regulatory and reputational fallout that follows.
The economics make the point sharper still. Australia has been building data centres and signing AI infrastructure deals at pace, and local firms from the ASX down to early-stage startups are weaving models into products that handle health records, financial data and government services. Every one of those deployments is a new dependency, and resilience has not kept pace with adoption. The Office of the Australian Information Commissioner has made clear it expects organisations to secure personal information regardless of the technology involved, which means an AI breach that spills customer data will be judged by the same privacy standards as any other, with the added complication that the organisation may struggle to explain how its own model behaved.
There is a competitive dimension too. Boards that can show customers and partners they rehearse AI incident scenarios will increasingly have an edge in procurement, especially as large enterprises and government buyers push assurance requirements down their supply chains. In that sense the 4 per cent figure is not only a risk indicator but a market signal, flagging where the laggards sit and where disciplined operators can differentiate themselves.
What’s next
The practical path forward is not mysterious. Security teams have argued for tabletop exercises built around realistic AI scenarios, clear kill-switch authority for models in production, logging detailed enough to reconstruct an AI system’s decisions after the fact, and board-level involvement so that the people accountable for the organisation have actually sat in the room when the scenario was run. None of that requires new technology so much as the discipline to schedule it and the honesty to learn from where the rehearsal falls apart.
Whether the figure shifts over the coming year will be a useful barometer of how seriously Australian organisations are taking the gap between deploying AI and defending it. For now, the message from the security community is blunt: adoption has sprinted ahead, and preparedness has been left standing at the start line. Closing that distance is cheaper and far less painful before an incident than during one.
Sources: Cyber Daily.
