System Prompts vs. User Messages

Where the role, constraints, and per-turn task belong, and why mixing them up causes drift in multi-turn conversations.

Prompt Engineering·Lesson 1 of 6·7 min

The Claude API separates a request into two channels: a top-level system parameter and the messages array. They are not interchangeable, and the exam tests whether you know which content belongs in which.

What each channel is for

The system prompt sets identity and standing instructions that apply for the entire conversation: the assistant's role, tone, output format defaults, safety constraints, tool-use policy, and anything that should hold true across every turn. It is set once per request and effectively persists as long as the conversation keeps sending it back.

A user message carries the per-turn task: the specific question, the document to summarize, the code to review right now. It changes every turn. Content that is true for turn 3 but not turn 7 does not belong in the system prompt.

Why the distinction matters in practice

If you put per-turn specifics into the system prompt, you end up rebuilding the entire system prompt on every request just to swap one detail, which is wasteful and error-prone, and it also means that detail applies to the whole conversation history even though it was only relevant to one exchange. Conversely, if you put role and constraint information into a user message, later turns may drift away from it, because the model treats it as one data point in the conversation rather than a standing rule.

A useful test: if the instruction would still be true on turn 50 of the same conversation, it belongs in the system prompt. If it is only true for this one exchange, it belongs in the user message.

Exam trap A question describes a chatbot where the system prompt is rebuilt on every single turn to include that turn's user-specific data (e.g., "the user's account ID is 4471, answer their question about billing"). The correct fix is to move the account ID and the question into the user message and keep the system prompt static across turns. Expect a distractor answer that says "add more instructions to the system prompt" — that does not address the actual problem, which is channel misuse, not insufficient instruction.

A support tool sends a fresh system prompt on every API call containing: the assistant's role ("You are a support agent for Acme Cloud"), tone rules, escalation policy, and also the current ticket's subject line and the customer's last message. Each turn, the ticket subject and customer message change, so the system prompt is regenerated in full every time, and the message history grows without ever reusing a stable system prompt. The fix: keep the system prompt fixed to role, tone, and policy only; move ticket subject and customer message into the user message for that turn. This also allows prompt caching to work correctly, since the system prompt no longer changes between calls.

System prompts and prompt caching

Because the system prompt is meant to stay stable across a conversation, it is also the natural place to put large, reusable content you want cached: long role descriptions, static reference material, extensive formatting rules. If you instead scatter that content across user messages, or regenerate it slightly differently every turn, you lose the ability to reuse a cached prefix, and you pay to reprocess the same tokens repeatedly. Keeping the system prompt byte-for-byte identical across calls (aside from occasional updates) is both a design-clarity practice and a cost optimization.

Multiple system-style instructions in one conversation

A related mistake is trying to change the assistant's standing behavior mid-conversation by injecting a new "system-like" instruction into a user message ("from now on, only answer in Spanish"). This can work, but it is fragile — the instruction now competes with the original system prompt and with the rest of the conversation history, and its effect can fade over many subsequent turns. If a behavior change is meant to be durable, the more reliable approach is to update the actual system prompt on the next request, not to bury a new standing rule inside a user turn.

Try it

Take a prompt you currently send with a single long user message that mixes "you are an expert X" framing with the actual task. Split it: move the role/format/constraint sentences into system, leave only the task-specific content in the user message. Send both versions for three different task inputs and compare whether the split version keeps the format constraint (e.g., "always answer in three bullet points") more consistently across all three. Then run the split version twice with an identical system prompt and check whether your client library reports a cache hit on the second call.