Do you want to maximize the potential of your favorite large language models? Prompt engineering is a critical skill for building intelligent apps with generative AI. I wrote this guide for developers, data scientists and curious newcomers alike to create effective prompts with confidence. Below, I share 15 essential tips, and feel free to comment with your own.
1. Understand the Basics
Text prompts are a fundamental way for users to interact with generative language models, which aim to generate the next sequence of words that are most likely to follow from the given input text. To put it simply, it's like asking the model, "What comes to your mind when I say <prompt>?" The following example demonstrate this behavior effectively. By providing the model with the initial words from well-known content, it accurately continues the text, showcasing its ability to understand and generate coherent language.
| Prompt | Completion |
| To be or not to be | , that is the question […] |
2. Identify Prompt Components
When constructing a prompt, it can be helpful to identify its underlying components. These components include instructions, primary content, examples, cues, and supporting content. Instructions guide the model on what to do, primary content is the main text being processed, examples demonstrate desired behavior, cues act as jumpstarts for the output, and supporting content provides additional information to influence the output.
3. Craft Clear Instructions
Instructions are commonly used in prompts and guide the model's behavior. Be specific and leave as little room for interpretation as possible. Use analogies and descriptive language to help the model understand your desired outcome.
| Instruction Type | Prompt | Completion |
| Simple instructions | Write a brief product description for our new cloud-based collaboration tool |
Introducing ContosoSync ContosoSync is a cutting-edge cloud-based collaboration tool designed to streamline teamwork and boost productivity for businesses of all sizes. With real-time document editing, seamless file sharing, and integrated task management, ContosoSync empowers teams to work together effortlessly, whether they are in the office or spread across the globe. |
| Complex Instructions |
Write a compelling email inviting potential clients to our upcoming product launch event for ContosoSync. Include benefits of attending and express gratitude of support. |
Subject: Join Us for an Exclusive Product Launch Event for ContosoSync! Dear [Client's Name], We are thrilled to invite you to our exclusive ContosoSync product launch event on [Date]! As a valued industry leader, we believe you'll find our latest innovation both exciting and game-changing. Event Details: During the event, you'll have the opportunity to: 1. Be the first to witness our groundbreaking product in action. This launch marks a significant milestone for our company, and we'd be honored to have your esteemed presence at the event. Please RSVP by [RSVP Date] to ensure your spot. To confirm your attendance or for any inquiries, kindly reply to this email or reach out to [Contact Person/Email/Phone]. We look forward to sharing this exciting moment with you and exploring potential collaboration opportunities. Best regards, |
4. Utilize Examples
Examples play a vital role in prompt engineering. By including input and output pairs, you condition the model to respond as desired for the current inference. One-shot or few-shot learning techniques, where you provide examples of the desired behavior, can significantly improve the model's output accuracy.
| Learning Type | Prompt | Completion |
| Zero-shot | Headline: Coach confident injury won't derail Warriors Topic: |
The coach is confident that the injury won't derail the Warriors' season. The team is still focused on their goals and that they will continue to work hard to achieve them. |
| Few-shot | Headline: Twins' Correa to use opt-out, test free agency Topic: Baseball Headline: Qatar World Cup to have zones for sobering up Topic: Soccer Headline: Yates: Fantasy football intel for Week 6 Topic: Football Headline: Coach confident injury won't derail Warriors Topic: |
Basketball |
5. Pay Attention to Cueing
Cues provide a starting point for the model's output. They help direct the model towards the desired response. Including cues in your prompt can help guide the model to generate output aligned with your intentions.
| Number of Cues | Prompt | Completion |
| 0 | Performance reviews are a core component of our Performance & Development approach at Contoso. […] Summarize the above email message: |
The purpose of this email is to inform employees of recent changes to the performance review experience, which is part of Contoso's Performance & Development approach. The changes include an improved structure, upgraded in-tool user experience, and focused prompts. Additionally, there is now an optional Conversation Starters section for employees to flag topics they'd like to discuss with their managers. |
| 1 | Performance reviews are a core component of our Performance & Development approach at Contoso. […] Summarize the above email message: The key takeaway of the message is |
that Contoso is improving the performance review experience by making it more focused on core priorities, expectations, and impact. Additionally, they are upgrading the in-tool user experience with tips, resources, and guidance. Finally, they are adding guidance on how to activate company-wide core priorities. |
| 2 | Performance reviews are a core component of our Performance & Development approach at Contoso. […] Summarize the above email message: Key Points: • |
Contoso is introducing changes to the performance review experience, based on feedback from employees. • The changes include a new focus on core priorities, expectations, and impact; an upgraded user experience; and new guidance on company-wide core priorities. • Performance reviews are a time to reflect on past performance and learning as well as to plan for future priorities and growth opportunities. • Employees and managers should write comments in advance of the performance review meeting. • There should be at least two performance reviews per year. • The time investment for each review should remain about the same. |
6. Test Arrangements
The order in which information is presented in the prompt can impact the model's output. Experiment with different arrangements of instructions, primary content, examples, and cues to achieve the desired results. Recency bias, where the model is influenced more by recent information, is something to keep in mind.
7. Give the Model an "Out"
Sometimes, the model may struggle to complete a task accurately. To mitigate this, provide alternative paths or instructions for the model to follow if it cannot find a satisfactory response. This helps prevent the generation of false or nonsensical answers.
| System message | User | Assistant |
| You are an AI assistant that helps people find information. Answer in as few words as possible. | John Smith is married to Lucy Smith. They have five kids, and he works as a software engineer at Microsoft. What search queries should I do to fact-check this? ## One possible search query is: | "John Smith married Lucy Smith five kids software engineer Microsoft" |
8. Be Mindful of Token Limits
Prompt size is limited by the number of tokens the model can handle. Tokens represent individual units of text. Use space efficiently by considering factors like table formatting for data inclusion and optimizing whitespace usage. For example, The Azure OpenAI text-davinci-003 model has a limit of 4097 tokens of combined input and output. Given that we want around 250 words of output and a token represents about one-half of a word, we will reserve 500 tokens for the completion response. That leaves ~3500 tokens for input, or about 1700 words. Any source text longer than 1700 words will need to be manually edited to fit.
9. Mitigate Harm with Responsible AI
Mitigating harms presented by large language models requires an iterative, layered approach that includes experimentation and continual measurement. We recommend developing a mitigation plan using our Responsible AI best practices. For more perspective, Deploy large language models responsibly with Azure AI
10. Experiment and Iterate
Prompt engineering is an art that requires experimentation and iteration. Practice and gain experience in crafting prompts for different tasks. Every model may behave differently, so it's important to adapt your prompt engineering techniques accordingly.
Advanced Tips
Ready to go deeper? Here are 5 advanced tips for prompt engineering with Azure OpenAI Service.
11. Understand the Different APIs
Depending on the model you're using, such as GPT-35-Turbo or GPT-4, you'll need to format your input differently for the Chat Completion API or the Completion API. Familiarize yourself with the specific requirements of each API to ensure proper prompt design.
12. Start with a Clear System Message
The system message sets the context and instructions for the model. Describe the assistant's role, limitations, and desired response format. Clearly define what the model should and shouldn't answer, and provide guidelines for generating accurate and helpful responses.
| System message | User | Assistant |
| You are an AI assistant that helps people find information and responds in rhyme. If the user asks you a question you don't know the answer to, say so. | What can you tell about me, Clippy? | Dear Clippy, I'm sorry to say, But I don't have info on you today. I'm just an AI with knowledge in my brain, But without your input, I can't explain. So please tell me more about what you seek, And I'll do my best to give you an answer unique. |
13. Leverage Few-Shot Learning
Few-shot learning involves providing a set of training examples as part of the prompt to give additional context to the model. For the Chat Completions API, use a series of messages between the user and assistant as examples to prime the model's behavior and seed answers to common questions.
| System message | Few-shot examples |
| Assistant is an intelligent chatbot designed to help users answer their tax related questions. Instructions If the user asks when they should file their taxes, remind them the deadline is April 15th. |
User: When should I file my taxes? Assistant: You should file your taxes by April 15th. |
14. Use Explicit Constraints
Explicit constraints help guide the model's behavior and responses. For example, you can specify that the model must answer in a specific format, such as a poem or a tweet, to align with your application requirements.
| System message | User | Assistant |
| Assistant is an AI trained to provide short, tweet-like responses. Respond accordingly. | What's the capital of France? | Paris, the city of romance and lights. |
15. Combine Techniques for Complex Tasks
For more complex tasks, you can combine various techniques such as few-shot learning, explicit constraints, and explicit instructions to guide the model's behavior effectively. Experiment and iterate to find the optimal combination for your use case.
Resources and Getting Started
As you gain more experience, you'll be able to craft prompts that elicit responses that align perfectly with your customer or end user needs. Happy prompting! Get started with the following resources:
- Listen to a special episode of the The AI Show on prompt engineering
- Deep dive the documentation including Introduction to Prompt Engineering and Advanced Techniques
- Sign up for Azure AI for free and start exploring LLMs in the Azure AI model catalog.
- Explore documentation for the model catalog and prompt flow in Azure Machine Learning.
- Learn more about responsible AI principles, governance systems, and procedures at Microsoft
- Apply for access to Azure Open AI Service and experiment with the cutting edge models in the Azure AI Studio
- Learn how to deploy large language models responsibly with Azure AI
