"AI security" is one phrase covering four unrelated problems. Vendors sell solutions to one of them while customers are worrying about another, and the mismatch is expensive because both parties think the conversation went well.
| What people mean | The actual problem | Who owns it |
|---|---|---|
| 1. Securing AI you deploy | Your models and agents are a new attack surface | Whoever owns the system. This page. |
| 2. Data leaking through AI tools | Staff paste confidential material into third-party tools | Policy and training, not engineering |
| 3. Attackers using AI | Better phishing, faster reconnaissance, synthetic media | Existing security function |
| 4. AI used for defence | Detection and triage tooling that happens to use models | Security operations — a procurement question |
Problems 3 and 4 are your existing security programme meeting a new capability. They matter, and they are not what makes AI a distinct security topic. Problem 2 is a governance and training question — a written policy about what may be pasted where does more than any technical control. Problem 1 is the one with genuinely new failure modes, and the rest of this page is about it.
Which AI you run decides your threat model
This is the distinction almost all AI security writing skips, and skipping it sends people to the wrong controls.
NIST's Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (AI 100-2e2025, March 2025) does not present one list of AI attacks. It maintains two separate taxonomies — one for predictive AI, one for generative AI — because the attack classes are genuinely different rather than variations on a theme.
That structural choice by the standards body is the most useful thing in the document, and it has a blunt practical consequence: if you run classifiers and scoring models, prompt injection cannot touch you, and prompt-injection tooling will not protect you. Conversely, a team deploying a chat assistant is not usually worried about someone perturbing an input image to flip a classification.
If you run predictive models
Fraud detection, credit scoring, risk models, recommendation, image classification, forecasting. NIST groups the attacks into three classes.
| Attack class | What the attacker does | What it costs you |
|---|---|---|
| Evasion | Crafts an input that the model classifies wrongly, at inference time | Fraud passes as legitimate; the filter misses what it exists to catch |
| Poisoning | Corrupts training data or the model itself. NIST separates availability poisoning, targeted poisoning, backdoor poisoning and model poisoning | Degraded accuracy, or a hidden trigger that behaves normally until used |
| Privacy | Extracts what should not be recoverable — data reconstruction, membership inference, property inference, model extraction | Training data leaks; an attacker rebuilds a functional copy of your model by querying it |
Two of these deserve emphasis because they are frequently absent from security reviews.
Backdoor poisoning means a model that passes every test you run. The trigger is chosen by the attacker and is not in your evaluation set, so normal accuracy metrics are silent on it. This is a supply-chain problem as much as a modelling one: it is a live question for any model or dataset you did not build yourself.
Membership inference means determining whether a specific record was in the training data. If your model was trained on medical, financial or HR records, the fact of someone's inclusion can itself be the sensitive information — regardless of whether any field is recoverable.
If you run generative models or agents
Chat assistants, retrieval systems, copilots, anything calling a language model API. NIST's generative taxonomy uses different classes again.
| Attack class | What the attacker does |
|---|---|
| Supply chain | Poisons data or a model before you ever touch it — the pre-trained weights, the fine-tuning set, a dependency |
| Direct prompting | Interacts with the model adversarially to extract information or push it outside its intended role |
| Indirect prompt injection | Plants instructions in content the system retrieves, so the person asking and the person instructing are different people |
NIST treats the security of agents as its own concern within this, which matches where the risk has moved. A model that only writes text produces a wrong answer. An agent produces a wrong action.
Indirect prompt injection is the class with no clean fix, and it has its own article: why prompt injection happens and what actually stops it. The short version is that it is architectural rather than a bug, so the controls limit consequences rather than prevent occurrence.
If you run both
Most organisations of any size do, usually without having said so out loud — a fraud model in one department and a chat assistant in another, procured years apart by people who never met. Both taxonomies apply, to different systems, and the security review has to be per-system rather than per-organisation.
The authoritative sources are free
Worth stating plainly, because this is a market with expensive answers to questions that already have published ones.
- NIST AI 100-2e2025 — the taxonomy above, plus mitigations and a glossary. Free.
- OWASP Top 10 for LLM Applications — practitioner-facing, ranked, aimed at people shipping. Free.
- Joint CISA/NSA guidance on agentic AI — a Cybersecurity Information Sheet issued with international partners in April 2026, covering agent identity, short-lived credentials and human sign-off for high-impact actions. Free.
NIST is explicit that its document is voluntary and does not carry the force of law. That is worth knowing in both directions: nobody can require you to follow it, and nobody can sell you compliance with it either.
Where to start
The order matters more than the individual steps, because most of the expensive mistakes come from buying a control before knowing what it protects.
- Inventory what you run. You cannot secure a system nobody has written down. Record for each one whether it is predictive or generative, what it can do, and what data it touches — see building an AI system inventory.
- Classify each entry against the right taxonomy. Predictive systems get the evasion/poisoning/privacy questions. Generative systems get the supply-chain/prompting/injection questions. Asking a system the wrong set produces a clean review that proves nothing.
- Ask what each system is permitted to do. Consequences scale with capability, not with model quality. A read-only summariser and an agent with write access to production are different risks running the same model.
- Check what you did not build. Pre-trained weights, fine-tuning datasets, model hubs and third-party APIs are all supply chain. The vendor risk assessment covers the questions worth asking.
- Name an owner. Not a team — a person. See who owns AI risk.
Why this is not only a security problem
The decisions that determine your exposure are not technical ones. What an agent may do without a person, which actions stop for confirmation, how much accuracy you will trade for robustness, whether a model trained on personal data should exist at all — these are judgements about acceptable risk.
A security team can describe the options and cannot make those calls alone, which is why AI security fails most often at the point where it meets governance. If nobody can say who decided an agent could send external email, the control gap is organisational rather than technical.
For the review mechanics, see human oversight that is more than a rubber stamp. For which framework to organise all of this under, see NIST AI RMF vs ISO/IEC 42001.
The bottom line
Before buying anything labelled AI security, answer two questions. Which of the four problems is this? And does it address predictive systems, generative systems, or is the vendor unclear which?
An answer that treats those as interchangeable is describing a product rather than a threat model.