Most organisations have moved past asking whether to use AI. The question now is whether you can trust what it touches. This is a practical strategy for Australian businesses adopting generative and agentic AI, mapped to the ASD Essential Eight, the NIST AI Risk Management Framework and the Australian Privacy Principles.
AI risk is no longer a technology question for the IT team alone. Models now read your documents, answer your customers and, increasingly, take actions on your behalf. Security has to move at the same pace as adoption.
Individual tools, enthusiastic users, few rules. Value arrives fast, and so does shadow AI.
An AI register, approved tools, data rules and least-privilege access for anything that can act.
Continuous monitoring, regular red-teaming and governance that lets you adopt new AI quickly and safely.
Three priorities carry most of the risk reduction: know what AI you have (and what data it touches), limit what AI can do (especially agents with tools and credentials), and assume untrusted content will try to steer your models (prompt injection is the defining AI security problem of this era). The good news: the Essential Eight basics — application control, patching, MFA, restricted admin rights, backups — still do much of the work.
The AI-specific threats Australian organisations face now, how you'd notice them, and what reduces the risk. References point to the OWASP Top 10 for LLM Applications, the Essential Eight and the APPs.
| Threat vector | Potential impact | Detection method | Mitigation strategy |
|---|---|---|---|
| Data & privacySensitive data leakageOWASP LLM02 · APP 6, 8, 11 | Personal information, client data or IP typed into prompts, stored by a vendor or repeated in outputs — a likely APP 6/11 problem and possibly a notifiable data breach. | Data-loss prevention on AI traffic; prompt logging with PII detection; regular review of what approved tools retain. | Enterprise AI tools with no-training and data-residency terms; mask or redact identifiers before prompting; clear "never paste" rules. |
| Data & privacyShadow AIEssential Eight: application control | Staff use unvetted AI apps, browser extensions and plugins, so data leaves the business with no contract, log or owner. | DNS, proxy and SaaS discovery logs for AI domains; expense and card audits; browser-extension inventory; a no-blame staff survey. | Offer approved alternatives first, then block the rest; an AI acceptable-use policy; a fast path to request new tools. |
| Data & privacyContext pollution & cross-user leakageOWASP LLM02 | One user's documents, memory or chat history surface in another user's session. | Tenant-isolation tests with canary strings; review of shared memory and session features. | Scope memory, caches and sessions to a single user or tenant; expire sessions; never share context windows across customers. |
| Data & privacyRAG permission bypassOWASP LLM08 · APP 11 | An internal assistant retrieves and summarises documents the person asking is not allowed to see. | Permission-aware test sets (ask as a low-privilege user); compare retrieval logs against document access lists. | Enforce each document's permissions at retrieval time, using the asker's identity; don't index what you can't permission. |
| Data & privacyKnowledge-base & vector poisoningOWASP LLM04, LLM08 | Planted or altered documents skew answers or carry hidden instructions into every response that retrieves them. | Scan content at ingestion; track the source of every chunk; canary documents; watch for sudden shifts in answer quality. | A controlled ingestion pipeline from approved sources; review for high-impact corpora; re-index and roll back cleanly. |
| Agents & promptsDirect prompt injection & jailbreaksOWASP LLM01, LLM07 | Users talk the model out of its rules — bypassing policy, extracting the system prompt or producing harmful content. | Input classifiers; alerts on refusal spikes and known jailbreak patterns; a standing red-team prompt suite. | Assume the system prompt will leak, so keep secrets out of it; layered guardrails on input and output; least-privilege tools behind the model. |
| Agents & promptsIndirect prompt injectionOWASP LLM01 | Instructions hidden in a web page, email, PDF or tool result hijack an assistant that reads it — for example to leak data through a link. | Tag every piece of untrusted content; watch for tool calls that follow external content; monitor outbound requests. | Keep trusted instructions and untrusted data separate; no automatic link or image rendering; human confirmation for sensitive actions; allow-listed destinations. |
| Agents & promptsExcessive agency (unauthorised function calling)OWASP LLM06 · Essential Eight: restrict admin privileges | An agent sends emails, moves money, deletes records or changes settings nobody intended. | Complete audit log of every tool call with its inputs; policy-engine alerts on out-of-scope actions. | Allow-listed tools only; scoped, short-lived credentials; human approval for irreversible or external actions; dry-run modes. |
| Agents & promptsRunaway agents (loops & cost blow-outs)OWASP LLM10 | An agent loops on a task, burning budget, hitting rate limits and loading internal systems. | Alerts on step count, token spend, latency and cost per task. | Hard step, time, token and dollar budgets; circuit breakers; a kill switch that works mid-run. |
| Agents & promptsLateral movement through agentsEssential Eight: restrict admin privileges, MFA | An attacker steers an agent's credentials into internal systems it was never meant to reach. | Identity monitoring for non-human (service) accounts; alerts on new API destinations and unusual hours. | A dedicated identity per agent; network segmentation; short-lived tokens; no shared or standing admin rights. |
| Agents & promptsInsecure output handlingOWASP LLM05 | Model output used directly as HTML, SQL, shell commands or code leads to injection, XSS or remote code execution. | Static and dynamic application testing; validation-failure logs. | Treat all model output as untrusted input: validate, encode, parameterise; run generated code only in a sandbox. |
| Agents & promptsAI supply chain (models, plugins, MCP servers)OWASP LLM03 · Essential Eight: application control | A compromised or careless third-party model, plugin or connector exposes data or runs unwanted actions. | An AI bill of materials; vendor security monitoring; review of every new connector before it is enabled. | Vendor risk assessment; pin and review versions; the least-privilege scopes a connector needs, nothing more. |
| AdversarialAI-powered phishing, voice clones & deepfakesEssential Eight: MFA | Convincing emails, calls or video from a "CEO" or supplier trigger payment redirection or credential theft. | Email security; staff reporting; call-back anomalies on payment changes. | Out-of-band verification (a known phone number) for every payment or bank-detail change; phishing-resistant MFA; regular drills. |
| AdversarialFaster, automated exploitationEssential Eight: patching, application control | Attackers use AI to find and exploit unpatched systems faster and at larger scale. | Endpoint detection and response; threat intelligence; vulnerability scanning. | Shorten patch windows; application control; hardened configurations — the Essential Eight basics matter more, not less. |
| AdversarialModel inversion & data extractionNIST AI 600-1 · APP 11 | Repeated querying reconstructs training data or clones a fine-tuned model — leaking personal information or IP. | Monitor query volume and patterns per user or key; rate-limit anomalies. | Don't fine-tune on raw personal information; rate limits and authentication on model endpoints; output filtering. |
| GovernanceConfident errors in decisionsOWASP LLM09 · AI Ethics Principles | Hallucinated facts or citations feed advice, decisions or public statements — legal, financial and reputational exposure. | Evaluation sets for key tasks; citation checks; complaint and correction monitoring. | Ground answers in sources with citations; human review for consequential decisions; disclose AI use where it affects people. |
Each control mapped to the ASD Essential Eight, the NIST AI RMF functions (Govern, Map, Measure, Manage) and the Australian Privacy Principles. The same controls support ISO/IEC 42001 and Australia's AI Ethics Principles.
| Control | Essential Eight | NIST AI RMF | Privacy & other |
|---|---|---|---|
| AI register & data-flow maps | Foundation for all eight | Map | APP 1 (open and transparent management) |
| Approved-tools list; block unvetted AI apps and extensions | Application control; user application hardening | Govern · Manage | APP 11 (security) |
| Least-privilege, short-lived credentials for agents | Restrict administrative privileges | Manage | APP 11 |
| MFA on AI platforms, consoles and API keys | Multi-factor authentication | Manage | APP 11 |
| Patch AI apps, SDKs, model servers and plugins | Patch applications; patch operating systems | Manage | — |
| Control Office add-ins and macros that call AI services | Configure Microsoft Office macro settings | Manage | — |
| Back up prompts, configs, vector stores and eval sets | Regular backups | Manage | NDB scheme readiness |
| Redact or mask personal information before prompting | — | Map · Manage | APP 3, 6, 11 |
| Know where each model and vendor processes data | — | Govern | APP 8 (cross-border disclosure) |
| Explain substantially automated decisions | — | Govern | Privacy Act ADM transparency; AI Ethics Principles |
| Continuous monitoring and red-teaming | — | Measure | ISO/IEC 42001 (monitoring, improvement) |
| AI incident response, including breach assessment | Regular backups (recovery) | Manage | Notifiable Data Breaches scheme |
Practical actions for each audience. Tick them off as you go — your progress is saved in this browser only.
Phases overlap on purpose: start containing the worst risks while discovery is still finishing. Timelines suit a small-to-mid-sized Australian organisation; larger or regulated organisations should allow more time.
Find out what AI is really in use, what data it touches and where the biggest risks sit.
Put boundaries around the risks you found — without stopping people from getting value from AI.
See problems early, test defences before attackers do, and know exactly what to do when something goes wrong.
Make secure AI the normal way of working as adoption grows.
Pick one phase-one milestone, then build the skills to close the gaps.
Last reviewed September 2026. General guidance only — not legal advice. Frameworks and regulation change; check the current versions from the ASD (cyber.gov.au), the OAIC (oaic.gov.au), NIST and the Department of Industry, Science and Resources before relying on specific requirements.