“60% of Australian professionals are already using AI at work, yet 78% have received no formal AI training from their employer.”
– Hays Salary Guide 2026 – The Australian
Most AI governance discussions focus on policies, frameworks and technology controls. Yet one of the biggest risks emerging inside organisations today has very little to do with the technology itself…
AI Risk Is No Longer Technical
One of the biggest misconceptions about AI governance is that the primary risk sits within the technology itself.
In reality, many of the most significant risks emerge after the AI has generated its response and its what happens next that makes all the difference.
A single AI-generated response can influence customer communications, participant records, tenancy management, employee interactions, compliance documentation and operational decision-making.
When inaccurate, incomplete or unverified information enters those processes, the that is where the risk can begin.
For organisations operating in sectors such as community housing, NDIS, healthcare and aged care, these risks are not theoretical. They are increasingly becoming part of everyday operations.
Where Organisations Fall Down
Recent Australian research suggests that 60% of professionals are now using AI at work, yet 78% have received no formal AI training from their employer.
The issue is not a lack of policies. Many organisations already have acceptable use policies, governance frameworks and risk management processes in place.
The challenge is ensuring those controls translate into day-to-day behaviour.
Increasingly, leaders are observing staff copying and pasting AI-generated content directly into workflows without sufficient review. its understandable they can’t resist the convenience and the time saved but just because it appears professional and well-written, it must also be accurate.
But AI systems generate incorrect information with resounding confidence ALL THE TIME and omit critical context or misinterpret organisational policies.
Without human review, those errors can quickly become operational risks.
New Privacy Act Requirements Are Raising The Stakes
This issue is becoming increasingly important from a regulatory perspective.
New Privacy Act requirements taking effect from December 2026 will require many organisations to identify and disclose where AI or other automated systems are used to make, or substantially contribute to, decisions that significantly affect individuals.
Importantly, the reforms recognise that simply having a human “rubber stamp” an AI recommendation is not enough.
Organisations will need clear governance, transparency and review processes that demonstrate meaningful human involvement in decision-making.
For organisations operating in community housing, NDIS, healthcare, aged care and other people-centric sectors, this is particularly significant. Decisions involving service delivery, tenancy matters, participant outcomes, patient care or employee welfare may increasingly require organisations to demonstrate not only what decision was made, but how that decision was reached and what role AI played in the process.
Source: Office of the Australian Information Commissioner (OAIC) Automated Decision-Making Transparency Consultation and Privacy Act amendments
Risk Is Created In Execution
The most important insight is that risk is rarely created in the governance document, but in execution.
Having an AI policy is important but having staff consistently apply that policy is what matters.
Human-in-the-Loop practices are often applied inconsistently and the result is a growing gap between governance intent and operational reality.
Just as organisations cannot assume employees understand every workplace policy, they cannot assume employees understand when and how AI-generated outputs should be challenged, validated and verified.
Human-In-The-Loop Is Not Optional
One of the core principles of responsible AI adoption is Human-in-the-Loop (HITL).
The concept is straightforward. AI can assist but humans remain accountable.
Any AI-generated content that directly or indirectly affects a customer, participant, tenant, employee or stakeholder should be reviewed by an appropriately qualified person before action is taken.
That review should consider:
• Accuracy
• Completeness
• Policy alignment
• Regulatory implications
• Organisational context
The individual approving the outcome remains responsible for the outcome. Not the AI model, software vendor or technology platform.
What Good Looks Like
There is no single AI governance framework that fits every organisation but organisations that are successfully adopting AI tend to demonstrate common characteristics.
They:
• Understand where AI is being used
• Define acceptable and unacceptable use cases
• Provide practical employee training
• Embed Human-in-the-Loop controls
• Reinforce accountability at every level
• Monitor outcomes and continuously improve governance practices
Most importantly, they can demonstrate that these controls are being actively applied. Not simply documented.
The Donnabrook Perspective
An example of a low risk use of AI and embedding a human in the loop in the context of communication platforms may be as follows. An inbound phone conversations if transcribed summarised by AI, and then inserted into a CRM against a client/tenant/participant record. To ensure the call summary is accurate. the agent is forced to tick a box that says that the call summary accurately reflects the conversation. That doesn’t ensure they read it but it can be backed up by listening to the recording.
Across the sectors we work with, the challenge is rarely awareness.
Most organisations understand that AI introduces new risks alongside new opportunities. First, understand where and how the AI is being applied. There is a big difference in risk between AI using to summarise calls within a compliant software vendor communication platform and another agentic AI that is triggering a process through integration into your systems.
The second challenge is ensuring governance extends beyond policy documents and into operational practice.
Technology adoption does not automatically create organisational capability. People create capability.
As AI becomes embedded into everyday workflows, organisations need confidence that employees understand when to trust AI, when to question it and when to escalate concerns.
And this is where leadership, training and operational visibility become critical. Because AI governance is not ultimately a technology problem.
It is a people challenge that can be solved through education, accountability, communication and culture.
Further Reading and Reference Sources
The principles discussed in this article are reflected in a range of Australian and international frameworks relating to AI governance, accountability, risk management and human oversight.
Australia’s Voluntary AI Safety Standard
Guidance from the Australian Government outlining ten guardrails for the safe and responsible development and deployment of AI systems.
https://www.industry.gov.au/publications/voluntary-ai-safety-standard
National AI Centre Guidance
Resources and implementation guidance supporting responsible AI adoption across Australian organisations.
Office of the Australian Information Commissioner (OAIC)
Guidance on privacy obligations, automated decision-making transparency requirements and the Privacy Act reforms affecting AI-enabled decision processes.
ISO/IEC 42001 – Artificial Intelligence Management Systems
The first international management system standard specifically designed for AI governance and oversight.
https://www.iso.org/standard/81230.html
NIST AI Risk Management Framework (AI RMF)
A globally recognised framework for identifying, assessing and managing AI-related risks.
https://www.nist.gov/itl/ai-risk-management-framework
Australian Government Policy for the Responsible Use of AI in Government
Guidance for the safe, accountable and transparent use of AI within Australian Government agencies.
Automated Decision-Making Transparency Consultation
Consultation material relating to the new Privacy Act requirements for organisations using automated decision-making systems.
