Safety, security and ethics

A risk map: the OWASP Top 10 for LLM applications

The OWASP GenAI Security Project maintains a widely used list of the most critical risks in applications built on language models. The 2025 edition lists:

ID	Risk	In one sentence
LLM01	Prompt injection	Attacker-controlled text changes the model's behavior
LLM02	Sensitive information disclosure	The model reveals data it should not
LLM03	Supply chain	Compromised models, adapters, datasets or dependencies
LLM04	Data and model poisoning	Tampered training or fine-tuning data
LLM05	Improper output handling	Model output used unsafely by downstream code
LLM06	Excessive agency	The model can take damaging actions in connected systems
LLM07	System prompt leakage	Hidden instructions are extracted
LLM08	Vector and embedding weaknesses	Attacks on retrieval stores and embeddings (RAG)
LLM09	Misinformation	Confident, false or misleading output
LLM10	Unbounded consumption	Runaway usage, denial of service and cost attacks

Use it as a checklist for design reviews. Prompt injection is first for a reason.

Prompt injection

An LLM can't reliably separate instructions from data. Two forms matter:

• Direct: the user tells the model to ignore its rules.
• Indirect: instructions hidden in content the assistant reads, such as a web page, an email, a PDF, a retrieved passage or a tool description.

There is no complete fix, so defend in layers:

1. Limit capability. Give tools the least privilege and the narrowest scope. A hijacked model with no powerful tools can do little.
2. Confirm consequential actions with a human, showing exactly what will happen.
3. Treat external content as data. Tell the model to ignore instructions inside sources, and mark where untrusted text begins and ends. This helps, but don't rely on it alone.
4. Separate duties. The component that reads untrusted content shouldn't also hold sensitive data and the ability to send it elsewhere. When one system has private data, untrusted input and an outbound channel, assume an attacker can combine them.
5. Validate outputs. Check tool arguments, links and formats in code before acting.
6. Monitor and test. Red-team with injection attempts through every input channel, and log suspicious patterns.

Excessive agency

The more an assistant can do, the more damage a mistake or attack can cause.

• Grant only the tools a task needs, with read-only access where possible.
• Put spending, deletion and messaging behind confirmation.
• Cap steps, time and cost per conversation.
• Use per-user credentials, so the assistant can only touch what that user could.
• Keep audit logs of every action.

Privacy and data

• Collect and send the minimum. Don't put full records into prompts when a few fields will do.
• Redact personal data before logging and before sending it to third parties where possible.
• Know the data flow: where prompts go, how long they are kept, whether they are used for training, which region processes them.
• Memory needs consent. Let people see and delete what is remembered.
• Retention limits for transcripts, and access controls on who can read them.

Honesty, disclosure and accountability

• Say it's an AI. Don't present a bot as a person.
• Offer a human, especially for complaints, money and distress.
• Don't promise what you won't honor. Assistants that invent discounts or policies create real obligations.
• Answer from authority. Policy questions should come from approved sources (module 6), with a clear not-found path.
• Own the outcome. In Moffatt v. Air Canada (2024), a British Columbia tribunal found the airline responsible for incorrect advice from its chatbot and rejected the argument that the chatbot was a separate entity. Assume you will be held accountable for what your assistant says.

Fairness, vulnerable users and accessibility

Test with different dialects, languages and writing styles. Provide accessible alternatives and plain language. Decide in advance how the assistant responds to people in crisis: acknowledge, avoid giving risky advice, and route to appropriate human or emergency resources. Medical, legal and financial topics need conservative scopes and clear handoffs.

Governance

• Name an owner for the assistant's behavior and a process for incidents.
• Keep a change log of prompts, models, tools and knowledge.
• Review before launch with design, engineering, legal and support.
• Check the rules that apply to you. Several jurisdictions have disclosure, consumer-protection and data-protection requirements for automated systems. Ask qualified counsel rather than assuming.