Generative AI for Beginners - Full Videos Series Released!
With so many new technologies, tools and terms in the world of Generative AI, it can be hard to know where to start or what to learn next. "Generative AI for Beginners" is designed to help you on your learning journey no matter where you are now. We are happy announce that the "Generative AI for Beginners" course has received a major refresh - 18 new videos for each lesson.Entity extraction with Azure OpenAI Structured Outputs
📺 Tune into our live stream on this topic on December 3rd! Have you ever wanted to extract some details from a large block of text, like to figure out the topics of a blog post or the location of a news article? In the past, I've had to use specialized models and domain-specific packages for entity extraction. But now, we can do entity extraction with large language models and get equally impressive results. 🎉 When we use the OpenAI gpt-4o model along with the structured outputs mode, we can define a schema for the details we'd like to extract and get a response that conforms to that schema. Here's the most basic example from the Azure OpenAI tutorial about structured outputs: class CalendarEvent(BaseModel): name: str date: str participants: list[str] completion = client.beta.chat.completions.parse( model="MODEL_DEPLOYMENT_NAME", messages=[ {"role": "system", "content": "Extract the event information."}, {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}, ], response_format=CalendarEvent, ) output = completion.choices[0].message.parsed The code first defines the CalendarEvent class, an instance of a Pydantic model. Then it sends a request to the GPT model specifying a response_format of CalendarEvent . The parsed output will be a dictionary containing a name , date , and participants . We can even go a step farther and turn the parsed output into a CalendarEvent instance, using the Pydantic model_validate method: event = CalendarEvent.model_validate(event) With this structured outputs capability, it's easier than ever to use GPT models for "entity extraction" tasks: give it some data, tell it what sorts of entities to extract from that data, and constrain it as needed. Extracting from GitHub READMEs Let's see an example of a way that I actually used structured outputs, to help me summarize the submissions that we got to a recent hackathon. I can feed the README of a repository to the GPT model and ask for it to extract key details like project title and technologies used. First I define the Pydantic models: class Language(str, Enum): JAVASCRIPT = "JavaScript" PYTHON = "Python" DOTNET = ".NET" class Framework(str, Enum): LANGCHAIN = "Langchain" SEMANTICKERNEL = "Semantic Kernel" LLAMAINDEX = "Llamaindex" AUTOGEN = "Autogen" SPRINGBOOT = "Spring Boot" PROMPTY = "Prompty" class RepoOverview(BaseModel): name: str summary: str = Field(..., description="A 1-2 sentence description of the project") languages: list[Language] frameworks: list[Framework] In the code above, I asked for a list of a Python enum, which will constrain the model to return only options matching that list. I could have also asked for a list[str] to give it more flexibility, but I wanted to constrain it in this case. I also annoted the description using the Pydantic Field class so that I could specify the length of the description. Without that annotation, the descriptions are often much longer. We can use that description whenever we want to give additional guidance to the model about a field. Next, I fetch the GitHub readme, storing it as a string: url = "https://api.github.com/repos/shank250/CareerCanvas-msft-raghack/contents/README.md" response = requests.get(url) readme_content = base64.b64decode(response.json()["content"]).decode("utf-8") Finally, I send off the request and convert the result into a RepoOverview instance: completion = client.beta.chat.completions.parse( model=os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT"), messages=[ { "role": "system", "content": "Extract info from the GitHub issue markdown about this hack submission.", }, {"role": "user", "content": readme_content}, ], response_format=RepoOverview, ) output = completion.choices[0].message.parsed repo_overview = RepoOverview.model_validate(output) You can see the full code in extract_github_repo.py That gives back an object like this one: RepoOverview( name='Job Finder Chatbot with RAG', description='This project is a chatbot application aimed at helping users find job opportunities and get relevant answers to questions about job roles, leveraging Retrieval-Augmented Generation (RAG) for personalized recommendations and answers.', languages=[<Language.JAVASCRIPT: 'JavaScript'>], azure_services=[<AzureService.AISEARCH: 'AI Search'>, <AzureService.POSTGRESQL: 'PostgreSQL'>], frameworks=[<Framework.SPRINGBOOT: 'Spring Boot'>] ) Extracting from PDFs I talk to many customers that want to extract details from PDF, like locations and dates, often to store as metadata in their RAG search index. The first step is to extract the PDF as text, and we have a few options: a hosted service like Azure Document Intelligence, or a local Python package like pymupdf. For this example, I'm using the latter, as I wanted to try out their specialized pymupdf4llm package that converts the PDF to LLM-friendly markdown. First I load in a PDF of an order receipt and convert it to markdown: md_text = pymupdf4llm.to_markdown("example_receipt.pdf") Then I define the Pydantic models for a receipt: class Item(BaseModel): product: str price: float quantity: int class Receipt(BaseModel): total: float shipping: float payment_method: str items: list[Item] order_number: int In this example, I'm using a nested Pydantic model Item for each item in the receipt, so that I can get detailed information about each item. And then, as before, I send the text off to the GPT model and convert the response back to a Receipt instance: completion = client.beta.chat.completions.parse( model=os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT"), messages=[ {"role": "system", "content": "Extract the information from the blog post"}, {"role": "user", "content": md_text}, ], response_format=Receipt, ) output = completion.choices[0].message.parsed receipt = Receipt.model_validate(output) You can see the full code in extract_pdf_receipt.py Extracting from images Since the gpt-4o model is also a multimodal model, it can accept both images and text. That means that we can send it an image and ask it for a structured output that extracts details from that image. Pretty darn cool! First I load in a local image as a base-64 encoded data URI: def open_image_as_base64(filename): with open(filename, "rb") as image_file: image_data = image_file.read() image_base64 = base64.b64encode(image_data).decode("utf-8") return f"data:image/png;base64,{image_base64}" image_url = open_image_as_base64("example_graph_treecover.png") For this example, my image is a graph, so I'm going to have it extract details about the graph. Here are the Pydantic models: class Graph(BaseModel): title: str description: str = Field(..., description="1 sentence description of the graph") x_axis: str y_axis: str legend: list[str] Then I send off the base-64 image URI to the GPT model, inside a "image_url" type message, and convert the response back to a Graph object: completion = client.beta.chat.completions.parse( model=os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT"), messages=[ {"role": "system", "content": "Extract the information from the graph"}, { "role": "user", "content": [ {"image_url": {"url": image_url}, "type": "image_url"}, ], }, ], response_format=Graph, ) output = completion.choices[0].message.parsed graph = Graph.model_validate(output) More examples You can use this same general approach for entity extraction across many file types, as long as they can be represented in either a text or image form. See more examples in my azure-openai-entity-extraction repository. As always, remember that large language models are probabilistic next-word-predictors that won't always get things right, so definitely evaluate the accuracy of the outputs before you use this approach for a business-critical task.Elevate Your AI Expertise with Microsoft Azure: Learn Live Series for Developers
Unlock the power of Azure AI and master the art of creating advanced AI agents. Starting from April 15th, embark on a comprehensive learning journey designed specifically for professional developers like you. This series will guide you through the official Microsoft Learn Plan, focused on the latest agentic AI technologies and innovations. Generative AI has evolved to become an essential tool for crafting intelligent applications, and AI agents are leading the charge. Here's your opportunity to deepen your expertise in building powerful, scalable agent-based solutions using the Azure AI Foundry, Azure AI Agent Service, and the Semantic Kernel Framework. Why Attend? This Learn Live series will provide you with: In-depth Knowledge: Understand when to use AI agents, how they function, and the best practices for building them on Azure. Hands-On Experience: Gain practical skills to develop, deploy, and extend AI agents with Azure AI Agent Service and Semantic Kernel SDK. Expert Insights: Learn directly from Microsoft’s AI professionals, ensuring you're at the cutting edge of agentic AI technologies. Session Highlights Plan and Prepare AI Solutions | April 15th Explore foundational principles for creating secure and responsible AI solutions. Prepare your development environment for seamless integration with Azure AI services. Fundamentals of AI Agents | April 22nd Discover the transformative role of language models and generative AI in enabling intelligent applications. Understand Microsoft Copilot and effective prompting techniques for agent development. Azure AI Agent Service: Build and Integrate | April 29th Dive into the key features of Azure AI Agent Service. Build agents and learn how to integrate them into your applications for enhanced functionality. Extend with Custom Tools | May 6th Enhance your agents’ capabilities with custom tools, tailored to meet unique application requirements. Develop an AI agent with Semantic Kernel | May 8th Use Semantic Kernel to connect to an Azure AI Foundry project Create Azure AI Agent Service agents using the Semantic Kernel SDK Integrate plugin functions with your AI agent Orchestrate Multi-Agent Solutions with Semantic Kernel | May 13th Utilize the Semantic Kernel SDK to create collaborative multi-agent systems. Develop and integrate custom plugin functions for versatile AI solutions. What You’ll Achieve By the end of this series, you'll: Build AI agents using cutting-edge Azure technologies. Integrate custom tools to extend agent capabilities. Develop multi-agent solutions with advanced orchestration. How to Join Don't miss out on this opportunity to level up your development skills and lead the next wave of AI-driven applications. Register now and set yourself apart as a developer equipped to harness the full potential of Azure AI. 🔗 Register for the Learn Live Series 🗓️ Format: Livestream | Language: English | Topic: Core AI Development Take the leap and transform how you develop intelligent applications with Microsoft Azure AI. Does this revision align with your vision for the blog? Let me know if there's anything else you'd like to refine or add!Essential Microsoft Resources for MVPs & the Tech Community from the AI Tour
Unlock the power of Microsoft AI with redeliverable technical presentations, hands-on workshops, and open-source curriculum from the Microsoft AI Tour! Whether you’re a Microsoft MVP, Developer, or IT Professional, these expertly crafted resources empower you to teach, train, and lead AI adoption in your community. Explore top breakout sessions covering GitHub Copilot, Azure AI, Generative AI, and security best practices—designed to simplify AI integration and accelerate digital transformation. Dive into interactive workshops that provide real-world applications of AI technologies. Take it a step further with Microsoft’s Open-Source AI Curriculum, offering beginner-friendly courses on AI, Machine Learning, Data Science, Cybersecurity, and GitHub Copilot—perfect for upskilling teams and fostering innovation. Don’t just learn—lead. Access these resources, host impactful training sessions, and drive AI adoption in your organization. Start sharing today! Explore now: Microsoft AI Tour Resources.Translating AI and ML for Beginners Curriculums in less than a day
I had been tasked with maintaining two of our open-source repositories: AI for Beginners and ML for Beginners. These repositories are crucial for anyone starting their journey in the fields of Artificial Intelligence and serve as the building blocks for further learning. The curriculum offers invaluable hands-on skills to practice one's learning. As of this week, the curriculum receives traffic of up to 5,000 unique views and over 18,000 views over a period of two weeks. That's a lot, right? Each learner is diverse, coming from different locations globally, and our goal was to translate the curriculum into local languages. Introducing Co-op Translator. To make the course accessible to a global audience, I decided to leverage Co-op Translator, an AI package that auto-translates the curriculum into different languages. Using this tool, I translated the courses into German, Spanish, French, Hindi, Italian, Japanese, Korean, Malay, Portuguese, Russian, Swahili, Turkish, Chinese, and soon, Polish. What is Co-op Translator? Co-op Translator is a CLI tool designed to translate your project files, both markdown and images, into multiple languages. Currently, it supports over 40 languages and gives you the liberty to add any new languages as well. The tool uses Azure, Azure AI Services, and Azure OpenAI Service for the translations, making it seamless and easy to get started. Additionally, the tool comes with a disclaimer that the content is AI translated, ensuring everyone who comes across the content understands its translation process. Setting Up Co-op Translator When translating the curriculum, I used pip to install the package in GitHub Codespaces and Azure AI Foundry to get the environment variables as follows: On your terminal, create a python virtual environment: python -m venv .venv Activate the environment: source .venv/bin/activate Install the library: pip install co-op-translator Create a new project in Azure AI Foundry, in the region East US, then get the keys and endpoints. Go to Azure AI Foundry, sign in with your account, and click create project Fill in the details ensuring the region is East US and create your project Head over to Models + endpoints and select deploy models. Deploy gpt-4o-mini model to use for translations. On the overview page, you will find your keys and endpoints for the resources, as highlighted, you will need this to fill your .env file. Authentication and configuration Azure AI Services Keys and Endpoints Azure OpenAI Service keys and endpoints gpt-4o-mini details Once the project is deployed, create and update your .env file as follows: # Azure Credentials AZURE_SUBSCRIPTION_KEY="API Key" AZURE_AI_SERVICE_ENDPOINT="Azure AI Services endpoint" # Azure OpenAI Credentials AZURE_OPENAI_API_KEY="API Key" AZURE_OPENAI_ENDPOINT="Azure OpenAI Service endpoint" AZURE_OPENAI_MODEL_NAME="Model Name" AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="Name" AZURE_OPENAI_API_VERSION="Model version" Adding translations On your terminal, add the first translation to Korean: translate -l "ko" Follow the command reference for any further updates on translation and more. Benefits of Using Co-op Translator As a developer advocate, the tool has been invaluable in ensuring that content created is accessible to anyone, regardless of their location. Using the tool, I was able to save hours of translations and ensure the translations are done as soon as possible. Try Co-op Translator today and see the impact on your projects! What Next? With open source, as with any other project, work is never done. Now that we have the translations in place, we need contributors to join in and evaluate translations. In case you find any translations amiss, create a PR, and we will merge it into the main repository. - Translations for ML for Beginners repository - Translations for AI for Beginners repository All contributions are welcome!
Building Intelligent Apps with Azure Cache for Redis, EntraID, Azure Functions, E1 SKU, and more!
We're excited to announce the latest updates to Azure Cache for Redis that will improve your data management and application performance as we kickoff for Microsoft Build 2024. Coming soon, the Enterprise E1 SKU (Preview) will offer a lower entry price, Redis modules, and enterprise-grade features. The Azure Function Triggers and Bindings for Redis are now in general availability, simplifying your workflow with seamless integration. Microsoft EntraID in Azure Cache for Redis is now in GA, providing enhanced security management. And there's more – we are also sharing added resources for developing intelligent applications using Azure Cache for Redis Enterprise, enabling you to build smarter, more responsive apps. Read the blog below to find out more about these amazing updates and how they can enhance your Azure Cache for Redis experience.
