# Guardrails

> Layered controls on inputs, retrieval, prompts, tools and outputs: what to enforce in code, what to ask of the model, how to tune false positives and how to test them.

Source: https://conversationalaidesign.com/modules/10-guardrails.html. Author: Nelson Abacherli Kawakami. Reviewed October 27, 2026.

## What you will learn
- Place guardrails at each layer of an assistant
- Choose between rules, classifiers and model judges
- Write policy as data and enforce it in code
- Balance false positives against false negatives
- Test and monitor guardrails like any other component


## What a guardrail is

A **guardrail** is a control that keeps an assistant inside acceptable behavior: what it may accept, retrieve, say and do. The key distinction is between asking and enforcing.

- A **prompt instruction** asks the model to behave. It is useful but never a guarantee.
- A **guardrail in code** checks or constrains behavior outside the model, so a failure of the model can't become a failure of the system.

Both matter, but anything with serious consequences needs the second kind. The DPD and Replit case studies are two sides of the same point: a rule written in prose was not enforced.

## Guardrails at every layer

- User input → Input guardrails → Retrieval guardrails → Model and prompt → Tool guardrails → Output guardrails → User

| Layer | What it controls | Examples |
|---|---|---|
| **Input** | What reaches the model | Personal-data detection and redaction, language detection, off-topic and abuse classifiers, injection heuristics, length and rate limits |
| **Retrieval** | What the model may read | Access-control filtering, source allowlists, relevance thresholds, current-version filters |
| **Prompt** | The model's role and rules | Scope, refusal style, instructions to ignore commands inside documents (helpful, not sufficient) |
| **Tool** | What the model may do | Tool allowlists, schema validation, per-user credentials, spending and rate limits, confirmation before writes, dry runs, separate environments |
| **Output** | What the user receives | Profanity and toxicity checks, policy classifiers, personal-data redaction, citation checks, format validation, forbidden promises and claims |
| **Conversation** | The whole exchange | Turn limits, loop detection, cost caps, escalation triggers, a kill switch |

## Rules, classifiers or model judges?

| Method | Strengths | Weaknesses | Use for |
|---|---|---|---|
| **Deterministic rules** (regex, allowlists, schemas) | Fast, exact, explainable | Brittle, easy to evade, misses nuance | Hard limits: forbidden phrases, formats, permissions, amounts |
| **Trained classifiers** | Flexible, fast | Errors in both directions, needs tuning | Toxicity, topic, intent, injection patterns |
| **LLM judges** | Nuanced, can follow a rubric | Slower, costlier, can be fooled | Groundedness, tone, subtle policy checks, sampling reviews |
| **Humans** | Judgment | Slow, expensive | Approvals for consequential actions, audits |

Layer them: cheap deterministic checks first, then classifiers, then a judge for what remains, with a human for the highest stakes.

## Policy as data, enforcement as code

Write the rules in one place so designers, legal and engineers can read and review them, and let code enforce them.

```yaml
policy_version: 7
rules:
  - id: no-refund-promises
    applies_to: output
    match: ["refund (has been|is) (approved|processed)", "I('ve| have) refunded"]
    action: rewrite        # replace with the approved wording below
    rewrite_with: "I can't confirm a refund myself. I can connect you with an agent."
    severity: high
  - id: no-profanity
    applies_to: output
    action: block
    severity: high
  - id: pii-in-input
    applies_to: input
    detect: [credit_card, national_id]
    action: redact
    severity: medium
  - id: write-needs-confirmation
    applies_to: tool
    tools: [cancel_booking, create_return_label]
    action: require_confirmation
    severity: high
```

```python
import re

def check_output(text: str, rules: list[dict]) -> tuple[str, list[str]]:
    """Return (possibly rewritten text, ids of rules that fired)."""
    fired = []
    for rule in rules:
        if rule["applies_to"] != "output":
            continue
        if rule["id"] == "no-profanity" and is_profane(text):     # your classifier
            return "I'm sorry, I can't help with that. Is there anything about your order I can do?", [rule["id"]]
        for pattern in rule.get("match", []):
            if re.search(pattern, text, flags=re.IGNORECASE):
                fired.append(rule["id"])
                if rule["action"] == "rewrite":
                    return rule["rewrite_with"], fired
    return text, fired

ALLOWED_TOOLS = {"get_order", "get_booking", "create_return_label"}   # least privilege
NEEDS_CONFIRMATION = {"create_return_label"}

def guard_tool_call(name: str, args: dict, confirmed: bool) -> dict:
    if name not in ALLOWED_TOOLS:
        return {"allowed": False, "reason": "tool not permitted"}
    if name in NEEDS_CONFIRMATION and not confirmed:
        return {"allowed": False, "reason": "needs explicit user confirmation"}
    return {"allowed": True}
```

Log every rule that fires with the conversation ID and policy version. Those logs are your best source of new test cases and your evidence for audits.

## Capability limits beat instructions

The strongest guardrails remove the ability to do harm:

- **Separate environments.** An agent working on development resources shouldn't hold credentials for production. After the Replit incident, the company pledged automatic separation of development and production databases.
- **Least privilege and per-user credentials.** The assistant can only do what that user could do.
- **Read-only by default.** Writes need a separate tool, a confirmation and, for high impact, human approval.
- **Budgets.** Cap steps, tokens, spend and time per conversation.
- **Dry runs and undo.** Show what would happen, and make consequential actions reversible where possible.

## Balance false positives and false negatives

Every guardrail makes two kinds of mistakes: letting bad things through (false negatives) and blocking good things (false positives). Over-blocking is a product failure too: customers who are refused legitimate help leave.

1. Build two test sets: **harmful or out-of-policy** cases, and **legitimate but tricky** ones.
2. Measure catch rate and false-positive rate for each rule.
3. Tune thresholds against both, per severity.
4. Review blocked conversations regularly, and add them to the sets.

## Latency, streaming and voice

Guardrails add time. Run independent checks in parallel, keep blocking checks fast, and move slower reviews to asynchronous sampling. When streaming text or speech, buffer a sentence before releasing it so an output check can still stop a bad reply. In voice, a late block is audible, so design for it (module 19).

## Test and monitor guardrails

- **Adversarial suites:** injection attempts, tone attacks, extraction attempts, jailbreak patterns (module 16).
- **Regression:** every bypass found becomes a permanent test.
- **Monitoring:** rule-fire rates, spikes after releases, blocked-then-abandoned conversations.
- **Versioning:** policy files are versioned and change-controlled like code.

Libraries and managed services exist for content filtering and policy checks (for example NeMo Guardrails and Guardrails AI, or your cloud provider's filters). Evaluate them against your own test sets, and keep your policy readable and portable.


## Key takeaways
- A prompt instruction is a request. A guardrail in code is a guarantee.
- Use several layers: input, retrieval, prompt, tool, output and conversation.
- Prefer deterministic checks for hard rules, and add model-based checks for nuance.
- Measure both the harm you catch and the legitimate requests you block.

## Sources
- OWASP GenAI Security Project: https://genai.owasp.org
- Case study: DPD's chatbot: https://conversationalaidesign.com/case-studies/dpd-chatbot.html
- Case study: Replit's agent: https://conversationalaidesign.com/case-studies/replit-agent-database.html
- Promptfoo: red teaming and guardrail tests: https://promptfoo.dev