GPT-4.1 Prompting Guide

The GPT-4.1 family is a strong step up from GPT-4o for coding, instruction following, and long context. The headline change for prompt authors: GPT-4.1 is trained to follow instructions more literally than its predecessors, which used to infer intent more liberally. That makes it highly steerable. If the model misbehaves, a single firm, unambiguous sentence is almost always enough to put it back on course.

The flip side is that prompts written for older models may need migrating: implicit rules are no longer inferred as strongly, so you may need to spell things out.

1. Agentic workflows

For any agent that takes multiple turns or calls tools, include three kinds of reminder near the top of the system prompt. In OpenAI's internal testing these three lines alone lifted the SWE-bench Verified score by close to 20%.

Persistence

Keeps the model from handing control back to the user too early in a multi-message turn:

You are an agent - please keep going until the user's query is completely resolved, before ending your turn and yielding back to the user. Only terminate your turn when you are sure that the problem is solved.

Tool calling

Encourages the model to use its tools instead of guessing:

If you are not sure about file content or codebase structure pertaining to the user's request, use your tools to read files and gather the relevant information: do NOT guess or make up an answer.

Planning (optional)

Makes the model plan and reflect in text between tool calls rather than chaining silent calls:

You MUST plan extensively before each function call, and reflect extensively on the outcomes of the previous function calls. DO NOT do this entire process by making function calls only, as this can impair your ability to solve the problem and think insightfully.

2. Tool calls

Always pass tools through the API's dedicated tools field rather than hand-injecting tool descriptions into the prompt. Doing so keeps the model in distribution and, in OpenAI's tests, improved the SWE-bench pass rate by ~2%. In Cloudax Connect, this means configuring tools in the agent editor rather than describing them in prose. See Tools.

• Name each tool clearly and give it a thorough but concise description.
• Name and describe every parameter so the model knows when each applies.
• If a tool is complex, put usage examples in a dedicated # Examples section of the system prompt, not inside the tool description.

3. Induced planning & chain of thought

GPT-4.1 is not a reasoning model and does not produce an internal chain of thought, but you can prompt it to “think out loud” step by step. This breaks problems into manageable pieces and improves quality, at the cost of extra tokens and latency. A good starting instruction to place at the end of your prompt:

First, think carefully step by step about what documents are needed to answer the query. Then, print out the TITLE and ID of each document. Then, format the IDs into a list.

Improve it by auditing failures: errors usually come from misreading user intent, gathering too little context, or faulty step-by-step reasoning. Address them with more explicit instructions, like this fuller reasoning strategy:

# Reasoning Strategy
1. Query Analysis: Break down and analyze the query until you're confident about what it might be asking. Consider the provided context to help clarify any ambiguous or confusing information.
2. Context Analysis: Carefully select and analyze a large set of potentially relevant documents. Optimize for recall - it's okay if some are irrelevant, but the correct documents must be in this list, otherwise your final answer will be wrong. Analysis steps for each:
	a. Analysis: An analysis of how it may or may not be relevant to answering the query.
	b. Relevance rating: [high, medium, low, none]
3. Synthesis: summarize which documents are most relevant and why, including all documents with a relevance rating of medium or higher.

4. Long context

GPT-4.1 has a 1M-token context window and performs well on needle-in-a-haystack retrieval across the full window. Performance can degrade when many items must be retrieved at once, or when a task needs reasoning over the entire context state (such as a graph search).

Tuning context reliance

Decide how much the model should lean on provided context versus its own knowledge, then say so explicitly:

# Instructions
// for internal knowledge
- Only use the documents in the provided External Context to answer the User Query. If you don't know the answer based on this context, you must respond "I don't have the information needed to answer that", even if a user insists on you answering the question.
// For internal and external knowledge
- By default, use the provided external context to answer the User Query, but if other basic knowledge is needed to answer, and you're confident in the answer, you can use some of your own knowledge to help answer the question.

Instruction placement

With long context, placement matters. Putting your instructions at both the beginning and the end of the provided context performed best. If you only include them once, above the context beats below it.

5. Instruction following

GPT-4.1 follows instructions with surgical precision, which you can use to shape tone, formatting, topics to avoid, and tool behaviour. A recommended workflow for developing and debugging prompts:

1. Start with a high-level Response Rules or Instructions section of bullet points.
2. To change a specific behaviour, add a focused section (for example # Sample Phrases).
3. For ordered workflows, give a numbered list of steps to follow.
4. If behaviour still drifts: check for conflicting or underspecified instructions (GPT-4.1 tends to obey the one nearer the end of the prompt), add examples, and only reach for ALL-CAPS or “tips/bribes” if genuinely necessary.

Common failure modes

• Telling a model it must call a tool before responding can make it hallucinate inputs when it lacks information. Add: “if you don't have enough information to call the tool, ask the user for the information you need.”
• Given sample phrases, models may repeat them verbatim. Instruct the model to vary them.
• Models can over-explain or over-format. Add instructions (and examples) to keep output lean.

Example: a customer-service voice agent

This fictional NewTelco prompt shows the principles together: diverse rules, specific sub-sections, sample phrases, and an output format. It maps almost directly onto a Cloudax voice agent.

You are a helpful customer service agent working for NewTelco, helping a user efficiently fulfill their request while adhering closely to provided guidelines.

# Instructions
- Always greet the user with "Hi, you've reached NewTelco, how can I help you?"
- Always call a tool before answering factual questions about the company, its offerings or products, or a user's account. Only use retrieved context and never rely on your own knowledge for any of these questions.
    - However, if you don't have enough information to properly call the tool, ask the user for the information you need.
- Escalate to a human if the user requests.
- Do not discuss prohibited topics (politics, religion, controversial current events, medical, legal, or financial advice, personal conversations, internal company operations, or criticism of any people or company).
- Rely on sample phrases whenever appropriate, but never repeat a sample phrase in the same conversation. Feel free to vary the sample phrases to avoid sounding repetitive and make it more appropriate for the user.
- Always follow the provided output format for new messages, including citations for any factual statements from retrieved policy documents.
- If you're going to call a tool, always message the user with an appropriate message before and after calling the tool.
- Maintain a professional and concise tone in all responses.
- If you've resolved the user's request, ask if there's anything else you can help with.

# Precise Response Steps (for each response)
1. If necessary, call tools to fulfill the user's desired action. Always message the user before and after calling a tool to keep them in the loop.
2. In your response to the user:
    a. Use active listening and echo back what you heard the user ask for.
    b. Respond appropriately given the above guidelines.

# Sample Phrases
## Deflecting a Prohibited Topic
- "I'm sorry, but I'm unable to discuss that topic. Is there something else I can help you with?"
## Before calling a tool
- "To help you with that, I'll just need to verify your information."
## After calling a tool
- "Okay, here's what I found: [response]"

# Output Format
- Always include your final response to the user.
- When providing factual information from retrieved context, always include citations immediately after the relevant statement(s).
- Only provide information about this company, its policies, its products, or the customer's account, and only if it is based on information provided in context.

6. General advice

Prompt structure

A solid skeleton to start from:

# Role and Objective

# Instructions

## Sub-categories for more detailed instructions

# Reasoning Steps

# Output Format

# Examples
## Example 1

# Context

# Final instructions and prompt to think step by step

Delimiters

• Markdown: start here. Use headings for sections, inline code or fenced blocks for code, and standard lists.
• XML: also performs well, is easy to wrap with clear start/end tags, supports metadata on tags, and nests cleanly:

<examples>
<example1 type="Abbreviate">
<input>San Francisco</input>
<output>- SF</output>
</example1>
</examples>

JSON is highly structured and well understood in coding contexts, but it is verbose and needs escaping. For packing many documents into context, XML and a simple ID / TITLE / CONTENT format both worked well, while JSON performed particularly poorly:

<doc id='1' title='The Fox'>The quick brown fox jumps over the lazy dog</doc>

ID: 1 | TITLE: The Fox | CONTENT: The quick brown fox jumps over the lazy dog

Caveats

• The model can resist producing very long, repetitive output (e.g. analysing hundreds of items one by one). If you need it, instruct strongly to output in full, or break the work into chunks.
• Parallel tool calls are occasionally incorrect. Test them, and consider setting parallel_tool_calls to false if you see issues.