Building AI Agent Applications Series - Using AutoGen to build your AI Agents
In the previous content, we learned about AI Agent. If you didn't read it, please read my previous content - Understanding AI Agents. We have many different frameworks to implement AI Agents. AutoGen from Microsoft is a relatively mature AI Agents framework. Now AutoGen is mainly based on two programming languages .NET and Python. The more mature version is the Python version. The content in this article is mainly based on the Python version https://microsoft.github.io/autogen. If you want to learn the .NET version, you can visit here https://microsoft.github.io/autogen-for-net
Localizing GitHub Repositories with LLMs
Large Language Models (LLMs) excel at compared to machine translation, as they process tokens of text in a context window. LLMs use attention mechanisms to weigh the relevance of each token relative to others in the sequence, allowing them to capture nuanced meanings and contextual dependencies effectivelyIntroduction to Prompt Engineering
With GPT-3, GPT-3.5, and GPT-4 prompt-based models, the user interacts with the model by entering a text prompt, to which the model responds with a text completion. Basic concepts and elements of GPT prompts Prompt components Instructions Primary Content Examples Cue Supporting content Prompts Basics Text prompts are how users interact with GPT models GPT models attempt to produce the next series of words that are most likely to follow from the previous text. Prompts | Best Practices Be Specific: Leave as little to interpretation as possible. Restrict the operational space Be Descriptive: Use analogies Double Down:Sometimes you may need to repeat yourself to the model. Give instructions before and after your primary content, use an instruction and a cue, etc. Order Matters:The order in which you present information to the model may impact the output. Whether you put instructions before your content (“summarize the following…”) or after (“summarize the above…”) can make a difference in output. Even the order of few-shot examples can matter. This is referred to as recency bias. Give the model an “out” :It can sometimes be helpful to give the model an alternative path if it is unable to complete the assigned task. For example, when asking a question over a piece of text you might include something like "respond with ‘not found’ if the answer is not present". This can help the model avoid generating false responses Prompt components Instructions When we show up to the present moment with all of our senses, we invite the world to fill us with joy. The pains of the past are behind us. The future has yet to unfold. But the now is full of beauty simply waiting for our attention. Instructions are likely the most commonly used prompt component Instructions - instruct the model on what to do Space efficiency TABLES As shown in the examples in the previous section, GPT models can understand tabular formatted data quite easily. This can be a space efficient way to include data, rather than preceding every field with name (such as with JSON). WHITE SPACE Consecutive whitespaces are treated as separate tokens which can be an easy way to waste space. Spaces preceding a word, on the other hand, are typically treated as part of the same token as the word. Carefully watch your usage of whitespace and don’t use punctuation when a space alone will do. Advanced techniques in prompt design and prompt engineering Certain models expect a specialized prompt structure For Azure OpenAI GPT models, there are currently two distinct APIs where prompt engineering comes into play: Chat Completion API Completion API Each API requires input data to be formatted differently Use of affordances | Factual claims, Search queries and Snippets Factual claims: John Smith is married to Lucy Smith John and Lucy have five kids John works as a software engineer at Microsoft Search queries: John Smith married to Lucy Smith John Smith number of children John Smith software engineer Microsoft Snippets: [1] … John Smith’s wedding was on September 25, 2012 … [2] … John Smith was accompanied by his wife Lucy to a party [3]John was accompanied to the soccer game by his two daughters and three sons [4] … After spending 10 years at Microsoft, Smith founded his own startup, Tailspin Toys [5] John M is the town smith, and he married Fiona. They have a daughter named Lucy System message framework and template recommendations for Large Language Models (LLMs) Define the model’s profile, capabilities, and limitations for your scenari Define the specific task(s) Define how the model should complete the tasks Define the scope and limitations Define the posture and tone Define the model's output format Define the language and syntax Define any styling or formatting Provide example(s) to demonstrate the intended behavior of the mode Describe difficult use cases Show the potential “inner monologue” Define additional behavioral guardrail Identify and prioritize the harms you’d like to address.Introduction to Content filtering and Embeddings in Azure Open AI Service
Content filtering and Embeddings in Azure AI Open Service: Abuse Monitoring Content Classification: Classifier models detect harmful language and/or images in user prompts (inputs) and completions (outputs). The system looks for categories of harms as defined in the Content Requirements, and assigns severity levels as described in more detail on the Content Filtering page. Abuse Pattern Capture: Azure OpenAI Service’s abuse monitoring looks at customer usage patterns and employs algorithms and heuristics to detect indicators of potential abuse. Detected patterns consider, for example, the frequency and severity at which harmful content is detected in a customer’s prompts and completions. Human Review and Decision: When prompts and/or completions are flagged through content classification and abuse pattern capture as described above, authorized Microsoft employees may assess the flagged content, and either confirm or correct the classification or determination based on predefined guidelines and policies. Data can be accessed for human review only by authorized Microsoft employees via Secure Access Workstations (SAWs) with Just-In-Time (JIT) request approval granted by team managers. For Azure OpenAI Service resources deployed in the European Economic Area, the authorized Microsoft employees are located in the European Economic Area. Notification and Action: When a threshold of abusive behavior has been confirmed based on the preceding three steps, the customer is informed of the determination by email. Except in cases of severe or recurring abuse, customers typically are given an opportunity to explain or remediate—and implement mechanisms to prevent recurrence of—the abusive behavior. Failure to address the behavior—or recurring or severe abuse—may result in suspension or termination of the customer’s access to Azure OpenAI resources and/or capabilities. Content filtering Azure OpenAI Service includes a content management system that works alongside core models to filter content. If the system identifies harmful content, you'll receive either An error on the API call Content_filter as the finish_reason on the response Mitigate Mitigating harms presented by large language models such as the Azure OpenAI models requires an iterative, layered approach that includes experimentation and continual measurement. Best practices – Content filtering Consider the following best practices Check the finish_reason to see if the generation is filtered Check that there's no error object in the content_filter_result Applications serving multiple end-users should pass the user parameter with each API call. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cpython#scenario-details Embedding: Cosine similarity Cosine similarity measures the cosine of the angle between two vectors projected in a multi-dimensional space. If two documents are far apart by Euclidean distance because of size, they could still have a smaller angle between them and therefore higher cosine similarity. Azure OpenAI embeddings rely on cosine similarity to compute similarity between documents and a query. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/understand-embeddings#embedding-modelsMastering Azure OpenAI Services: A Comprehensive Learning Path for Aspiring AI Engineers
Are you a computer science student looking to delve into the world of Azure OpenAI Services? Look no further! In this Microsoft Learning Pathway, "Develop Generative AI solutions with Azure OpenAI Service," you'll embark on an exciting journey to harness the power of OpenAI's vast language models like ChatGPT, GPT, Codex, and Embeddings. These models are pivotal for creating innovative Natural Language Processing (NLP) solutions that can comprehend, converse, and generate content.Azure OpenAI Services in teaching and education
With the advent of Large language models(LLMs) like GPT-3, we see a transformation in education. In this article, I present my views on the future of education considering these developments. The views presented here are based on my teaching - but are a personal perspective. Today, there is a lot of excitement and speculation about GPT-3, and it is natural to ask how intelligent GPT-3 is and whether it approaches human-level intelligence. But in many ways, that's the wrong question to ask. Instead, exploring the idea of how we can build ChatGPT-like functionality using our own data is more interesting. When framed this way, we focus on the pragmatic and ignore the esoteric. If the industry adopts the 'co-pilot first approach.', educators must follow this trend to keep up with the new job roles. This will need a complete rethinking of many of the current ideas on education and the adoption of some new ideas that I proposed in this article. The conversation changes from: 'chatGPT is used for exam cheating or not' to: How can I empower my students to take up jobs of the future if the co-pilot first mode of work becomes a default?'
