# Prompting across different models

> Why a prompt that works on one model fails on another, what actually differs, and how to keep prompts portable, versioned and tested.

Source: https://conversationalaidesign.com/modules/05-prompting-across-models.html. Author: Nelson Abacherli Kawakami. Reviewed October 27, 2026.

## What you will learn
- Explain why prompts do not transfer cleanly between models
- Adapt one task to frontier, small and reasoning models
- Render one prompt specification into different API shapes
- Apply chat templates correctly for open-weight models
- Tune and track prompts per model using an evaluation set


## Why prompts do not transfer

A prompt is not a program. It is a message to a statistical system that was trained in a particular way, so the same words can produce different behavior on a different model, and even on a newer version of the *same* model. Teams discover this the hard way: a migration "works" in a demo, then policy refusals loosen, formatting drifts and tool calls change.

The fix is not a magic universal prompt. It is a method: keep the *intent* portable, and treat the *wording* as a per-model artifact that you test.

> **Tendencies, not rules**
> Everything below describes common differences. Vendors change model behavior between versions, so confirm details in each vendor's current prompting guide, linked at the end of this module.

## What actually differs

| Aspect | What can differ between models | What to do |
|---|---|---|
| **Roles and where instructions go** | Names and handling of system, developer, user and assistant messages; how strongly the system message is followed | Put durable rules in the system or developer slot, per the vendor's guide. Test that user messages can't override them |
| **Structure and delimiters** | Some models respond well to XML-style tags, others to Markdown headings or plain labels | Use clear, consistent delimiters and test alternatives if quality drops |
| **Instruction following and verbosity** | Some obey literally, some over-explain or add caveats | State length, tone and format limits explicitly and measure them |
| **Reasoning behavior** | Reasoning or "thinking" models plan internally; others benefit from you asking for steps | For reasoning models, give goals and constraints. For others, ask for brief structured reasoning only when it helps |
| **Examples (few-shot)** | Small models gain a lot from examples; strong models can over-anchor on them | Start with fewer, varied examples and add only if tests improve |
| **Output formats** | JSON modes, structured-output features and tool schemas differ in strictness and syntax | Prefer the vendor's structured-output or tool features, and validate in code regardless |
| **Sampling parameters** | Availability and behavior of temperature and top-p vary; some reasoning models restrict them | Don't assume a setting exists. Read the docs and pin your values |
| **Context and caching** | Position effects and long-context quality differ; caching favors stable prefixes | Put stable instructions first and variable content last, and test long contexts |
| **Refusals and safety style** | Different thresholds and phrasing for declining | Test your out-of-scope and sensitive cases on every model |
| **Languages** | Quality, tone and code-switching vary by language | Evaluate each language you serve |
| **Chat templates (open-weight)** | Each family has its own role markers and special tokens | Always use the tokenizer's chat template |

## One task, three prompts

Take a refund-request assistant. The goal is the same across models, but the best prompt isn't.

**For a strong general model**, a concise, structured prompt works well:

```text
You are Sol, a support assistant for an online shop.

<rules>
- Answer only from the policy excerpts provided. If the answer is not there, say so and offer a person.
- Never promise a refund. Explain the policy and the next step.
- Reply in the customer's language, in at most four sentences.
</rules>

<policy>
{policy_excerpts}
</policy>

<customer_message>
{message}
</customer_message>
```

**For a small, fast model**, be more explicit, add a format, and include one or two examples:

```text
Task: reply to a customer about a return.
Follow these steps in order:
1. Find the answer in POLICY. If it is not in POLICY, write exactly: "I don't have that information. I can connect you to a person."
2. Never say a refund is approved.
3. Write 2 to 4 short sentences in the customer's language.

Example
POLICY: Returns accepted within 30 days with receipt.
MESSAGE: Can I return shoes from last week?
REPLY: Yes. Returns are accepted within 30 days with a receipt. Would you like a return label?

POLICY: {policy_excerpts}
MESSAGE: {message}
REPLY:
```

**For a reasoning model**, state the goal and constraints, and skip the step-by-step script:

```text
Goal: help the customer with their return question, grounded only in the policy below.
Constraints: no promises of refunds; if the policy does not cover it, say so and offer a person;
reply in the customer's language in at most four sentences.

Policy:
{policy_excerpts}

Customer message:
{message}
```

None of these is "the" right prompt. Each is a starting point you then evaluate on your test set.

## Keep the intent portable: a prompt spec

Rather than keeping one blob of text per model, keep a structured **prompt spec** and render it. The renderers below show the *shapes* different APIs expect. Treat them as sketches and confirm exact field names in each SDK's documentation, because they evolve.

```python
from dataclasses import dataclass, field

@dataclass
class PromptSpec:
    role: str
    rules: list[str]
    format: str
    examples: list[tuple[str, str]] = field(default_factory=list)

    def system_text(self) -> str:
        parts = [self.role, "Rules:"] + [f"- {r}" for r in self.rules] + [f"Format: {self.format}"]
        for user, reply in self.examples:
            parts += [f"Example user: {user}", f"Example reply: {reply}"]
        return "\n".join(parts)

def render_anthropic_style(spec, user_text):
    return {"system": spec.system_text(), "messages": [{"role": "user", "content": user_text}]}

def render_openai_chat_style(spec, user_text):
    return {"messages": [{"role": "system", "content": spec.system_text()},
                         {"role": "user", "content": user_text}]}

def render_gemini_style(spec, user_text):
    return {"system_instruction": spec.system_text(),
            "contents": [{"role": "user", "parts": [{"text": user_text}]}]}

SPEC = PromptSpec(
    role="You are Sol, a support assistant for an online shop.",
    rules=["Answer only from the provided policy excerpts.", "Never promise a refund.", "Reply in the customer's language."],
    format="At most four sentences, plain text.",
)
```

Variants per model then become small, reviewable overrides on top of one spec: a different `format` line, extra `examples` for a small model, fewer rules for a reasoning model.

## Open-weight models: use the chat template

Open-weight model families are trained with specific formats for roles and special tokens. Assembling that by hand is a common source of mysterious quality loss. Let the tokenizer do it:

```python
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("your-model-name")   # use the exact model you deploy
messages = [
    {"role": "system", "content": SPEC.system_text()},
    {"role": "user", "content": "Can I return shoes from last week?"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
```

If you serve the model through an inference server, check whether it applies the template for you, and don't apply it twice.

## Tune per model, with evidence

1. **Baseline:** write the prompt spec once and run your evaluation set (module 14) on the first model.
2. **Port:** render for the second model and run the *same* set.
3. **Diagnose the gaps:** read the failures. Is it format, refusals, tone, tool arguments, language?
4. **Adjust the smallest thing** that fixes the failures, per model. Re-run everything, not just the failing cases.
5. **Record it:** keep a prompt registry with the model name and version, the prompt version, the test results and the reason for each override.
6. **Repeat on every model update,** because "same model name" does not mean "same behavior."

## Common pitfalls

- **Copying a prompt from a blog post** written for another model or an older version.
- **Conflicting instructions** that one model resolves one way and another resolves differently.
- **Examples that leak** into answers, or that the model copies too literally.
- **Relying on undocumented behavior,** such as a quirk that disappears in the next version.
- **Huge prompts** that bury the important rules. Shorter and clearer usually wins.
- **Assuming the system prompt is secret.** Treat it as extractable, and never put credentials or sensitive data in it (module 18).
- **Testing in one language** and shipping in five.


## Key takeaways
- Treat a prompt as a per-model artifact that you test, not text you copy.
- Differences cluster around roles, structure, reasoning, examples, output formats and sampling.
- Smaller models need clearer, more explicit prompts; reasoning models need goals, not scripts.
- Keep a portable prompt spec, render it per model, and re-run evaluations whenever a model changes.

## Sources
- Claude documentation: prompt engineering: https://docs.claude.com
- OpenAI platform docs: prompting and reasoning guides: https://platform.openai.com/docs
- Gemini API docs: prompting and system instructions: https://ai.google.dev/gemini-api/docs
- Hugging Face: chat templates: https://huggingface.co/docs/transformers/chat_templating