Operationalize your Prompt Engineering Skills with Azure Prompt Flow
In today’s AI-driven world, prompt engineering is a game-changing skill for developers and professionals alike. With Azure Prompt Flow, you can harness the power of open-source LLMs to solve real-world operational challenges! This article guides you through using Azure’s robust tools to build, deploy, and refine your own LLM apps—from chatbots to data extraction tools and beyond. Whether you're just starting or looking to sharpen your AI expertise, this guide has everything you need to unlock new possibilities with prompt engineering. Dive in and take your tech journey to the next level!Exploring AI Development and Management: A Journey through Contoso Chat and LLM Ops
In this blog, we'll navigate through the world of AI models, exploring Contoso Chat, Prompt Engineering, limitations of Prompt Engineering, and Large Language Models. We'll introduce tools like the RAG Pattern and Azure AI Studio that can boost AI responses and system performance. Ready to dive into the intricacies of AI development and management? Join us!Navigating Semantic Kernel v1.0.0-beta1: A Path to Agnostic AI Services
Are you a computer science student eager to dive into the world of AI development with the latest technologies? Semantic Kernel's latest release, v1.0.0-beta1, brings you an exciting opportunity to explore agnostic AI services. In this blog post, we'll take you through the significant changes in this version and show you how to adapt your code effectively.AI Agents in Production: From Prototype to Reality - Part 10
This blog post, the tenth and final installment in a series on AI agents, focuses on deploying AI agents to production. It covers evaluating agent performance, addressing common issues, and managing costs. The post emphasizes the importance of a robust evaluation system, providing potential solutions for performance issues, and outlining cost management strategies such as response caching, using smaller models, and implementing router models.Exploring Generative AI: A Hands-on Course on Prompt Engineering for non-tech students - Part 1
Introduction Generative Artificial Intelligence (AI) has transformed the digital landscape through "intent-based outcome specification," a paradigm where users describe desired outcomes via detailed prompts instead of traditional commands. This course - targeting a non-developer audience - delved into the foundational principles of Generative AI and Large Language Models (LLMs), focusing on their core mechanisms and capabilities. Students learned and practiced effective prompting techniques, essential for navigating this powerful yet complex method. The course included the analysis and discussion of recent research on prompt engineering, keeping students abreast of the latest developments. The structure of the course balanced theoretical understanding and practical application, with 30% dedicated to traditional lectures and 70% to hands-on workshops and collaborative group projects. Practical exercises using models like GPT allowed students to apply their theoretical knowledge in real-world scenarios. Group projects focused on specific application domains – including music, literature and cuisine - leading to presentations, peer reviews, and instructor feedback. The course - comprising 20 hours of direct instruction - was conducted at the Fondazione Bruno Kessler (FBK) campuses in Povo. Instructors Antonio Bucchiarone and Nadia Mana guided the learning journey. Additionally, Carlotta Castelluccio from Microsoft conducted a seminar on Responsible AI, emphasizing ethical considerations in AI applications. In this first part of the blog series, we are going to present the methodological framework and the tools used throughout the course. In the second part, we are going to cover the student projects’ main outcomes and key takeaways. The Card Model Template and the Flow of Cards One of the primary goals of the course was to provide a clear and comprehensive understanding of prompt engineering. This was achieved by introducing a structured framework known as the "Card Model" to define and organize generative AI tasks. In the context of this course, a card refers to a structured format or template used to define a specific task or objective for generating content or output using generative AI techniques. The Card Model serves as a conceptual framework that outlines the structure, components, and relationships involved in generating content or output using generative AI techniques. It provides a high-level abstraction of the task, capturing its essential elements and defining their interactions. Here’s a simplified model of a generative AI task: Objective: This is the overarching goal or purpose of the generative AI task, defining what needs to be achieved through the content generation process. Input: Information provided to the generative AI technique to guide the content generation process. This includes: Prompt: A starting point or stimulus to generate content, such as a partial sentence, a question, an instruction, or other forms of input. Context: Additional information or constraints that provide context for the generation task, such as background knowledge, relevant data sources, or specific requirements. Generative Model: The AI model responsible for generating content based on the input provided. Examples include pre-trained language models like OpenAI GPT-3.5Turbo, neural network architectures for text generation, or other generative AI systems. Output: The generated content produced by the generative model in response to the input, including: Generated Text: The actual output, which could be in the form of text, images, or other media. Evaluation Metrics: Criteria used to assess the quality and relevance of the generated content, including measures of coherence, relevance, fluency, and other factors depending on the specific task requirements. Feedback Loop: A mechanism for iteratively improving the generative AI model based on feedback from users or evaluators. This may involve refining the input prompts, adjusting model parameters, or incorporating additional training data to enhance performance. The Card Model helps define the key components involved in the task and their relationships, facilitating the design, execution, and evaluation of generative AI tasks in various applications. Cards Flow Model The concept of flow was also introduced in the course to provide a formal representation of the relationships between different cards composing a generative AI task. This flow model helps in visualizing and understanding the sequential and conditional transitions between different stages of a generative AI task, ensuring a structured and systematic approach to designing, executing, and evaluating generative AI processes. In more details, cards can be combined together to create complex workflows by defining specific transitions and dependencies between them. By linking these cards through directed edges, students can create intricate flows that mirror real-world applications. This pattern also helps students to break down complex tasks into smaller subtasks, described through detailed prompts and potentially addressed by different specialized models, generally leading to a more accurate final outcome. For example, a card flow might begin with a card that generates an initial story prompt. The output of this card could then flow into a card that adds contextual details, which in turn flows into another card responsible for generating the story based on the enhanced prompt. Subsequent cards could be used to evaluate the generated content, refine the prompt based on evaluation metrics, and iterate the process. To ensure a thorough understanding of these flows, students were asked to evaluate different paths within a flow. This involved analyzing how changes in one card could affect the overall output and exploring alternative pathways to achieve the desired outcome. Students were tasked with: Mapping Out Flows: Students mapped out various flows, identifying all possible paths and transitions between cards. Evaluating Paths: They evaluated each path to understand how different sequences and combinations of tasks impacted the final output. Comparing Outcomes: Students compared outcomes from different paths to determine which flow produced the most coherent, relevant, and high-quality results. Feedback and Iteration: They incorporated feedback into their flows, refining cards and transitions to optimize the generative process. By engaging in these activities, students gained hands-on experience in managing complex generative AI tasks, learning to anticipate and handle the dependencies and contingencies that arise in practical applications. This exercise not only reinforced their understanding of prompt engineering but also highlighted the importance of structured planning and iterative improvement in generative AI projects. The Azure AI Proxy Playground Students learned to interact with OpenAI models through the GUI offered by the Azure AI Proxy Playground. The service is an open-source solution which provides a Playground-like experience to explore the Azure OpenAI chat completions using a time-bound event code with different models and parameters. It’s designed for educational scenarios (e.g., a course, a hackathon, or a workshop) where students might not have access to an Azure subscription enabled with Azure OpenAI service and/or are not familiar with the Azure ecosystem and how to provision and consume Azure AI resources. By leveraging this solution, we were able to provide students with a simplified lab environment, where all the complexity related to the Cloud resources provisioning and model deployments was hidden to the final user and managed through a single Azure subscription, connected to the Proxy Playground. This was particularly helpful in the context of a course whose audience was non-technical and whose focus was learning to interact with large language models through prompt engineering techniques. For the sake of the course, we provisioned a gpt-3.5 turbo instance, so all the students’ interactions via the playground happened with that specific model. Tool GUI and Card Model mapping The Playground GUI is composed of several elements. Most of them can be directly mapped with the Card Model components, ensuring consistency between the theoretical concepts and the actual experimentations. User prompt: free-form text field used to enter the user request to the model. It’s the prompt component of the input in the card model. System message: free-form text field used to enter additional information to use in responses, data sources and/or tone and style specifications. It maps with the context component of the input in the card model. Configuration: parameters to tune the degree of randomness of the responses. It also includes a dropdown menu to select the model to use as chat engine, what we call generative model in the card template. Assistant response: in the chat session the user can read the model’s response, aka the generated text component of the output in the card model. Summary In this article, we covered the methodological framework and tools used in the Prompt Engineering course at Fondazione Bruno Kessler, to teach non-tech students to effectively interact with generative AI models. We explored the "Card Model" - a structured approach to define and organize generative AI tasks - and the concept of the "flow", which further structures the relationships between tasks, aiding in the creation of complex workflows. Students utilized the Azure AI Proxy Playground, an open-source GUI, to interact with OpenAI models like GPT-3.5 Turbo, applying their theoretical knowledge in practical scenarios without needing extensive technical skills. In the second part of the blog series, we will delve into the main outcomes of the students' projects and the key takeaways from their practical applications.Introduction to Prompt Engineering
With GPT-3, GPT-3.5, and GPT-4 prompt-based models, the user interacts with the model by entering a text prompt, to which the model responds with a text completion. Basic concepts and elements of GPT prompts Prompt components Instructions Primary Content Examples Cue Supporting content Prompts Basics Text prompts are how users interact with GPT models GPT models attempt to produce the next series of words that are most likely to follow from the previous text. Prompts | Best Practices Be Specific: Leave as little to interpretation as possible. Restrict the operational space Be Descriptive: Use analogies Double Down:Sometimes you may need to repeat yourself to the model. Give instructions before and after your primary content, use an instruction and a cue, etc. Order Matters:The order in which you present information to the model may impact the output. Whether you put instructions before your content (“summarize the following…”) or after (“summarize the above…”) can make a difference in output. Even the order of few-shot examples can matter. This is referred to as recency bias. Give the model an “out” :It can sometimes be helpful to give the model an alternative path if it is unable to complete the assigned task. For example, when asking a question over a piece of text you might include something like "respond with ‘not found’ if the answer is not present". This can help the model avoid generating false responses Prompt components Instructions When we show up to the present moment with all of our senses, we invite the world to fill us with joy. The pains of the past are behind us. The future has yet to unfold. But the now is full of beauty simply waiting for our attention. Instructions are likely the most commonly used prompt component Instructions - instruct the model on what to do Space efficiency TABLES As shown in the examples in the previous section, GPT models can understand tabular formatted data quite easily. This can be a space efficient way to include data, rather than preceding every field with name (such as with JSON). WHITE SPACE Consecutive whitespaces are treated as separate tokens which can be an easy way to waste space. Spaces preceding a word, on the other hand, are typically treated as part of the same token as the word. Carefully watch your usage of whitespace and don’t use punctuation when a space alone will do. Advanced techniques in prompt design and prompt engineering Certain models expect a specialized prompt structure For Azure OpenAI GPT models, there are currently two distinct APIs where prompt engineering comes into play: Chat Completion API Completion API Each API requires input data to be formatted differently Use of affordances | Factual claims, Search queries and Snippets Factual claims: John Smith is married to Lucy Smith John and Lucy have five kids John works as a software engineer at Microsoft Search queries: John Smith married to Lucy Smith John Smith number of children John Smith software engineer Microsoft Snippets: [1] … John Smith’s wedding was on September 25, 2012 … [2] … John Smith was accompanied by his wife Lucy to a party [3]John was accompanied to the soccer game by his two daughters and three sons [4] … After spending 10 years at Microsoft, Smith founded his own startup, Tailspin Toys [5] John M is the town smith, and he married Fiona. They have a daughter named Lucy System message framework and template recommendations for Large Language Models (LLMs) Define the model’s profile, capabilities, and limitations for your scenari Define the specific task(s) Define how the model should complete the tasks Define the scope and limitations Define the posture and tone Define the model's output format Define the language and syntax Define any styling or formatting Provide example(s) to demonstrate the intended behavior of the mode Describe difficult use cases Show the potential “inner monologue” Define additional behavioral guardrail Identify and prioritize the harms you’d like to address.
