There is a particular kind of silence that falls over a Senate estimates hearing when an official is asked a direct question and cannot answer it. That silence settled over a committee room this week when senior health officials were pressed on the workings of an algorithm now embedded in Australia’s aged care system, and struggled to explain how it actually reaches its conclusions.
As The Canberra Times reported, officials appeared caught out when asked to detail the logic behind an algorithm used in aged care decisions, a moment that will feel uncomfortably familiar to anyone who has watched the slow creep of automated tools into Commonwealth service delivery. The exchange was brief, but the implications are not. If the people charged with overseeing a system cannot describe how its machinery works, the obvious follow-up question is who can.
Why this keeps happening
Algorithms have quietly become load-bearing infrastructure across Australian government. They help sort who gets assessed and when, how funding is weighted, and how limited resources are triaged across a population that is ageing faster than almost any cohort of services can keep pace with. Aged care is a prime example. The sector has moved through the recommendations of the Royal Commission into Aged Care Quality and Safety, a new Aged Care Act, and the rollout of the Support at Home program, all of which lean on assessment and classification tools to decide what care a person is entitled to.
The difficulty exposed at estimates is that these tools are often described in public as neutral calculators, when in practice they encode a long chain of judgement calls about which data matters, how it is weighted, and what thresholds trigger a different outcome. When an official cannot walk a committee through that chain, it suggests the knowledge sits either with a vendor, with a small number of technical staff, or in documentation that has not been made plain to the decision-makers themselves. None of those is a comfortable place for public accountability to rest.
Australians have been here before. The Robodebt scheme, which used an automated income-averaging method to raise unlawful debts against welfare recipients, remains the defining cautionary tale. A royal commission found that automation had been deployed without adequate legal foundation and that officials had failed to stop it even as harm mounted. The lesson most observers drew was not that automation is inherently dangerous, but that automation without comprehension is. An algorithm nobody in the room can fully explain is an algorithm nobody can confidently defend.
Two ways to read the stumble
There are competing readings of what happened in that hearing room. The more generous interpretation, and the one the department will lean on, is that estimates is a gruelling format in which officials are asked hundreds of granular questions across sprawling portfolios, and that not having the precise mechanics of one algorithm at your fingertips is a lapse of preparation rather than a sign of systemic blindness. On this view, the information exists, it is governed properly, and it can be provided on notice. Governments routinely take questions on notice precisely because no witness can hold an entire portfolio in their head.
The less generous reading, and the one aged care advocates and digital-rights groups will press, is that the stumble is symptomatic. If an algorithm genuinely shapes who receives care and how much, then the inability to explain it is not a trivia-quiz failure. It is evidence that the humans nominally in charge have outsourced their understanding to a system they treat as authoritative. Advocates have argued for years that automated decisions affecting vulnerable people must come with a right to a meaningful explanation and a clear route to human review. A health official hesitating under questioning is, in that framing, exactly the scenario those safeguards are meant to prevent.
Both readings can be partly true at once. Estimates does reward rote preparation over genuine insight, and it is also true that the growth of algorithmic tooling across government has outrun the institutional habit of being able to explain it.
What it means for Australia
For Australians, this is not an abstract governance puzzle. More than a million older people rely on aged care services, and the number is climbing as the population ages. The decisions these systems influence are intimate and consequential: whether someone can stay in their own home, how many hours of support they receive, how quickly they are assessed after a fall or a diagnosis. When those decisions are shaped by a tool that officials cannot readily explain, the people most affected are precisely those least equipped to challenge an outcome they do not understand.
It also lands at a moment when Canberra is actively trying to write the rules for automated decision-making. The Attorney-General’s Department has been consulting on a legislative framework for automated decisions in government, a direct response to Robodebt’s finding that the law had not kept pace with the technology. The Commonwealth has also published guidance on the responsible use of artificial intelligence in the public service. An estimates stumble over a live aged care algorithm is a useful, if awkward, stress test of whether that emerging framework is reaching the operational level, or whether it remains a layer of principles sitting above systems that keep running on their own logic.
What happens next
The immediate mechanics are predictable. The committee is likely to have placed questions on notice, and the department will be expected to return with a written account of how the algorithm functions, what it draws on, and who maintains it. Those answers, when they arrive, will be scrutinised by senators, by sector peak bodies and by the new Aged Care Quality and Safety regulator for signs that the tool is doing what it is meant to do and nothing more.
The larger question is cultural rather than procedural. Australia is rapidly adopting algorithmic and increasingly AI-driven tools across health, welfare and aged care, often for sound reasons of scale and consistency. The condition that keeps this safe is not that every system be simple, but that someone accountable can always explain it in plain language to the people it governs. This week’s hearing suggested that condition is not yet reliably met. Getting it met, before the next generation of more opaque AI systems arrives, is the real work ahead.
Sources: The Canberra Times.
