Is it a bug or a feature? Using Prompty to automatically track and tag issues.
Introduction You’ve probably noticed a theme in my recent posts: tackling challenges with AI-powered solutions. In my latest project, I needed a fast way to classify and categorize GitHub issues using a predefined set of tags. The tag data was there, but the connections between issues and tags weren’t. To bridge that gap, I combined Azure OpenAI Service, Prompty, and a GitHub to automatically extract and assign the right labels. By automating issue tagging, I was able to: Streamline contributor workflows with consistent, on-time labels that simplify triage Improve repository hygiene by keeping issues well-organized, searchable, and easy to navigate Eliminate repetitive maintenance so the team can focus on community growth and developer empowerment Scale effortlessly as the project expands, turning manual chores into intelligent automation Challenge: 46 issues, no tags The Prompty repository currently hosts 46 relevant, but untagged, issues. To automate labeling, I first defined a complete tag taxonomy. Then I built a solution using: Prompty for prompt templating and function calling Azure OpenAI (gpt-4o-mini) to classify each issue Azure AI Search for retrieval-augmented context (RAG) Python to orchestrate the workflow and integrate with GitHub By the end, you’ll have an autonomous agent that fetches open issues, matches them against your custom taxonomy, and applies labels back on GitHub. Prerequisites: An Azure account with Azure AI Search and Azure OpenAI enabled Python and Prompty installed Clone the repo and install dependencies: pip install -r requirements.txt Step 1: Define the prompt template We’ll use Prompty to structure our LLM instructions. If you haven’t yet, install the Prompty VS Code extension and refer to the Prompty docs to get started. Prompty combines: Tooling to configure and deploy models Runtime for executing prompts and function calls Specification (YAML) for defining prompts, inputs, and outputs Our Prompty is set to use gpt-4o-mini and below is our sample input: sample: title: Including Image in System Message tags: ${file:tags.json} description: An error arises in the flow, coming up starting from the "complete" block. It seems like it is caused by placing a static image in the system prompt, since removing it causes the issue to go away. Please let me know if I can provide additional context. The inputs will be the tags file implemented using RAG, then we will fetch the issue title and description from GitHub once a new issue is posted. Next, in our Prompty file, we gave instructions of how the LLLM should work as follows: system: You are an intelligent GitHub issue tagging assistant. Available tags: ${inputs} {% if tags.tags %} ## Available Tags {% for tag in tags.tags %} name: {{tag.name}} description: {{tag.description}} {% endfor %} {% endif %} Guidelines: 1. Only select tags that exactly match the provided list above 2. If no tags apply, return an empty array [] 3. Return ONLY a valid JSON array of strings, nothing else 4. Do not explain your choices or add any other text Use your understanding of the issue and refer to documentation at https://prompty.ai to match appropriate tags. Tags may refer to: - Issue type (e.g., bug, enhancement, documentation) - Tool or component (e.g., tool:cli, tracer:json-tracer) - Technology or integration (e.g., integration:azure, runtime:python) - Conceptual elements (e.g., asset:template-loading) Return only a valid JSON array of the issue title, description and tags. If the issue does not fit in any of the categories, return an empty array with: ["No tags apply to this issue. Please review the issue and try again."] Example: Issue Title: "App crashes when running in Azure CLI" Issue Body: "Running the generated code in Azure CLI throws a Python runtime error." Tag List: ["bug", "tool:cli", "runtime:python", "integration:azure"] Output: ["bug", "tool:cli", "runtime:python", "integration:azure"] user: Issue Title: {{title}} Issue Description: {{description}} Once the Prompty file was ready, I right clicked on the file and converted it to Prompty code, which provided a Python base code to get started from, instead of building from scratch. Step 2: enrich with context using Azure AI Search To be able to generate labels for our issues, I created a sample of tags, around 20, each with a title and a description of what it does. As a starting point, I started with Azure AI Foundry, where I uploaded the data and created an index. This typically takes about 1hr to successfully complete. Next, I implemented a retrieval function: def query_azure_search(query_text): """Query Azure AI Search for relevant documents and tags.""" search_client = SearchClient( endpoint=SEARCH_SERVICE_ENDPOINT, index_name=SEARCH_INDEX_NAME, credential=AzureKeyCredential(SEARCH_API_KEY) ) # Perform the search results = search_client.search( search_text=query_text, query_type=QueryType.SIMPLE, top=5 # Retrieve top 5 results ) # Extract content and tags from results documents = [doc["content"] for doc in results] tags = [doc.get("tags", []) for doc in results] # Assuming "tags" is a field in the index # Flatten and deduplicate tags unique_tags = list(set(tag for tag_list in tags for tag in tag_list)) return documents, unique_tags Step 3: Orchestrate the Workflow In addition, to adding RAG, I added functions in the basic.py file to: fetch_github_issues: calls the GitHub REST API to list open issues and filters out any that already have labels. run_with_rag: on the issues selected, calls the query_azure_search to append any retrieved docs, tags the issues and parses the JSON output from the prompt to a list for the labels label_issue: patches the issue to apply a list of labels. process_issues: this fetches all unlabelled issues, extracts the rag pipeline to generate the tags, and calls the labels_issue tag to apply the tags scheduler loop: this runs every so often to check if there's a new issue and apply a label Step 4: Validate and Run Ensure all .env variables are set (API keys, endpoints, token). Install dependencies and execute using: python basic.py Create a new GitHub issue and watch as your agent assigns tags in real time. Below is a short demo video here to illustrate the workflow. Next Steps Migrate from PATs to a GitHub App for tighter security Create multi-agent application and add an evaluator agent to review tags before publishing Integrate with GitHub Actions or Azure Pipelines for CI/CD Conclusion and Resources By combining Prompty, Azure AI Search, and Azure OpenAI, you can fully automate GitHub issue triage—improving consistency, saving time, and scaling effortlessly. Adapt this pattern to any classification task in your own workflows! You can learn more using the following resources: Prompty documentation to learn more on Prompty Agents for Beginners course to learn how you can build your own agentWeek 2 . Microsoft Agents Hack Online Events and Readiness Resources
https://aka.ms/agentshack 2025 is the year of AI agents! But what exactly is an agent, and how can you build one? Whether you're a seasoned developer or just starting out, this FREE three-week virtual hackathon is your chance to dive deep into AI agent development. Register Now: https://aka.ms/agentshack 🔥 Learn from expert-led sessions streamed live on YouTube, covering top frameworks like Semantic Kernel, Autogen, the new Azure AI Agents SDK and the Microsoft 365 Agents SDK. Week 2 Events: April 14th-18th Day/Time Topic Track 4/14 08:00 AM PT Building custom engine agents with Azure AI Foundry and Visual Studio Code Copilots 4/15 07:00 AM PT Your first AI Agent in JS with Azure AI Agent Service JS 4/15 09:00 AM PT Building Agentic Applications with AutoGen v0.4 Python 4/15 12:00 PM PT AI Agents + .NET Aspire C# 4/15 03:00 PM PT Prototyping AI Agents with GitHub Models Python 4/16 04:00 AM PT Multi-agent AI apps with Semantic Kernel and Azure Cosmos DB C# 4/16 06:00 AM PT Building declarative agents with Microsoft Copilot Studio & Teams Toolkit Copilots 4/16 07:00 AM PT Prompting is the New Scripting: Meet GenAIScript JS 4/16 09:00 AM PT Building agents with an army of models from the Azure AI model catalog Python 4/16 12:00 PM PT Multi-Agent API with LangGraph and Azure Cosmos DB Python 4/16 03:00 PM PT Mastering Agentic RAG Python 4/17 06:00 AM PT Build your own agent with OpenAI, .NET, and Copilot Studio C# 4/17 09:00 AM PT Building smarter Python AI agents with code interpreters Python 4/17 12:00 PM PT Building Java AI Agents using LangChain4j and Dynamic Sessions Java 4/17 03:00 PM PT Agentic Voice Mode Unplugged PythonAI Toolkit for Visual Studio Code Now Supports NVIDIA NIM Microservices for RTX AI PCs
AI Toolkit now supports NVIDIA NIM™ microservice-based foundation models for inference testing in the model playground and advanced features like bulk run, evaluation and building prompts. This collaboration helps AI Engineers streamline development processes with foundational AI models. About AI Toolkit AI Toolkit is a VS Code extension for AI engineers to build, deploy, and manage AI solutions. It includes model and prompt-centric features that allow users to explore and test different AI models, create and evaluate prompts, and perform model finetuning, all from within VS Code. Since its preview launch in 2024, AI Toolkit has helped developers worldwide learn about generative AI models and start building AI solutions. NVIDIA NIM Microservices This January, NVIDIA announced that state-of-the-art AI models spanning language, speech, animation and vision capabilities - offered as NVIDIA NIM microservices - can now run locally on NVIDIA RTX AI PCs. These microservices prepackage optimized AI models with all the necessary runtime components for deployment across NVIDIA GPUs. Developers can now develop and deploy anywhere with the same unified experience and software stack across RTX AI PCs and workstations to the cloud. Developers can jumpstart their AI development journey by downloading and running NIM containers quickly on Windows 11 PCs with GeForce RTX GPUs using Windows Subsystem for Linux (WSL2). The Power of Collaboration Integrating AI Toolkit with NIM provides AI engineers with a more cohesive and efficient workflow: Seamlessly integrate AI Toolkit with NIM to create a unified development environment without the need to switch context. Users can access any NIM supported models from AI Toolkit. Leverage the combined capabilities of both tools to streamline workflows and accelerate AI solution development process around foundation AI models, from within VS Code. How to get started Follow these steps to begin leveraging the power of NIM on AI Toolkit: Download and install the latest version of AI Toolkit for VS Code. Install NIM pre-requisites on RTX PCs using the instructions here. Select a NIM model from the model catalog on AI Toolkit and load it in Playground. Optionally, you can also add your NIM model hosted in the cloud to AI Toolkit by URL Explore NIM models from the playground Start developing prompts with new NIM models in AI Toolkit! Looking Forward We invite you to explore the possibilities of this integration and take your development projects to new heights! Try AI Toolkit today – and please continue sharing your feedback. Stay tuned for more updates and detailed tutorials on how to maximize the benefits of this exciting new collaboration. Together, we are shaping the future of AI development!Essential Microsoft Resources for MVPs & the Tech Community from the AI Tour
Unlock the power of Microsoft AI with redeliverable technical presentations, hands-on workshops, and open-source curriculum from the Microsoft AI Tour! Whether you’re a Microsoft MVP, Developer, or IT Professional, these expertly crafted resources empower you to teach, train, and lead AI adoption in your community. Explore top breakout sessions covering GitHub Copilot, Azure AI, Generative AI, and security best practices—designed to simplify AI integration and accelerate digital transformation. Dive into interactive workshops that provide real-world applications of AI technologies. Take it a step further with Microsoft’s Open-Source AI Curriculum, offering beginner-friendly courses on AI, Machine Learning, Data Science, Cybersecurity, and GitHub Copilot—perfect for upskilling teams and fostering innovation. Don’t just learn—lead. Access these resources, host impactful training sessions, and drive AI adoption in your organization. Start sharing today! Explore now: Microsoft AI Tour Resources.
Getting Started with the AI Dev Gallery
March Update: The Gallery is now available on the Microsoft Store! The AI Dev Gallery is a new open-source project designed to inspire and support developers in integrating on-device AI functionality into their Windows apps. It offers an intuitive UX for exploring and testing interactive AI samples powered by local models. Key features include: Quickly explore and download models from well-known sources on GitHub and HuggingFace. Test different models with interactive samples over 25 different scenarios, including text, image, audio, and video use cases. See all relevant code and library references for every sample. Switch between models that run on CPU and GPU depending on your device capabilities. Quickly get started with your own projects by exporting any sample to a fresh Visual Studio project that references the same model cache, preventing duplicate downloads. Part of the motivation behind the Gallery was exposing developers to the host of benefits that come with on-device AI. Some of these benefits include improved data security and privacy, increased control and parameterization, and no dependence on an internet connection or third-party cloud provider. Requirements Device Requirements Minimum OS Version: Windows 10, version 1809 (10.0; Build 17763) Architecture: x64, ARM64 Memory: At least 16 GB is recommended Disk Space: At least 20GB free space is recommended GPU: 8GB of VRAM is recommended for running samples on the GPU Using the Gallery The AI Dev Gallery has can be navigated in two ways: The Samples View The Models View Navigating Samples In this view, samples are broken up into categories (Text, Code, Image, etc.) and then into more specific samples, like in the Translate Text pictured below: On clicking a sample, you will be prompted to choose a model to download if you haven’t run this sample before: Next to the model you can see the size of the model, whether it will run on CPU or GPU, and the associated license. Pick the model that makes the most sense for your machine. You can also download new models and change the model for a sample later from the sample view. Just click the model drop down at the top of the sample: The last thing you can do from the Sample pane is view the sample code and export the project to Visual Studio. Both buttons are found in the top right corner of the sample, and the code view will look like this: Navigating Models If you would rather navigate by models instead of samples, the Gallery also provides the model view: The model view contains a similar navigation menu on the right to navigate between models based on category. Clicking on a model will allow you to see a description of the model, the versions of it that are available to download, and the samples that use the model. Clicking on a sample will take back over to the samples view where you can see the model in action. Deleting and Managing Models If you need to clear up space or see download details for the models you are using, you can head over the Settings page to manage your downloads: From here, you can easily see every model you have downloaded and how much space on your drive they are taking up. You can clear your entire cache for a fresh start or delete individual models that you are no longer using. Any deleted model can be redownload through either the models or samples view. Next Steps for the Gallery The AI Dev Gallery is still a work in progress, and we plan on adding more samples, models, APIs, and features, and we are evaluating adding support for NPUs to take the experience even further If you have feedback, noticed a bug, or any ideas for features or samples, head over to the issue board and submit an issue. We also have a discussion board for any other topics relevant to the Gallery. The Gallery is an open-source project, and we would love contribution, feedback, and ideation! Happy modeling!
Learn how to develop innovative AI solutions with updated Azure skilling paths
The rapid evolution of generative AI is reshaping how organizations operate, innovate, and deliver value. Professionals who develop expertise in generative AI development, prompt engineering, and AI lifecycle management are increasingly valuable to organizations looking to harness these powerful capabilities while ensuring responsible and effective implementation. In this blog, we’re excited to share our newly refreshed series of Plans on Microsoft Learn that aim to supply your team with the tools and knowledge to leverage the latest AI technologies, including: Find the best model for your generative AI solution with Azure AI Foundry Create agentic AI solutions by using Azure AI Foundry Build secure and responsible AI solutions and manage generative AI lifecycles From sophisticated AI agents that can autonomously perform complex tasks to advanced chat models that enable natural human-AI collaboration, these technologies are becoming essential business tools rather than optional enhancements. Let’s take a look at the latest developments and unlock their full potential with our curated training resources from Microsoft Learn. Simplify the process of choosing an AI model with Azure AI Foundry Choosing the optimal generative AI model is essential for any solution, requiring careful evaluation of task complexity, data requirements, and computational constraints. Azure AI Foundry streamlines this decision-making process by offering diverse pre-trained models, fine-tuning capabilities, and comprehensive MLOps tools that enable businesses to test, optimize, and scale their AI applications while maintaining enterprise-grade security and compliance. Our Plan on Microsoft Learn titled Find the best model for your generative AI solution with Azure AI Foundry will guide you through the process of discovering and deploying the best models for creating generative AI solutions with Azure AI Foundry, including: Learn about the differences and strengths of various language models Find out how to integrate and use AI models in your applications to enhance functionality and user experience. Rapidly create intelligent, market-ready multimodal applications with Azure models, and explore industry-specific models. In addition, you’ll have the chance to take part in a Microsoft Azure Virtual Training Day, with interactive sessions and expert guidance to help you skill up on Azure AI features and capabilities. By engaging with this Plan on Microsoft Learn, you’ll also have the chance to prove your skills and earn a Microsoft Certification. Leap into the future of agentic AI solutions with Azure After choosing the right model for your generative AI purposes, our next Plan on Microsoft Learn goes a step further by introducing agentic AI solutions. A significant evolution in generative AI, agentic AI solutions enable autonomous decision-making, problem-solving, and task execution without constant human intervention. These AI agents can perceive their environment, adapt to new inputs, and take proactive actions, making them valuable across various industries. In the Create agentic AI solutions by using Azure AI Foundry Plan on Microsoft Learn, you’ll find out how developing agentic AI solutions requires a platform that provides scalability, adaptability, and security. With pre-built AI models, MLOps tools, and deep integrations with Azure services, Azure AI Foundry simplifies the development of custom AI agents that can interact with data, make real-time decisions, and continuously learn from new information. You’ll also: Learn how to describe the core features and capabilities of Azure AI Foundry, provision and manage Azure AI resources, create and manage AI projects, and determine when to use Azure AI Foundry. Discover how to customize with RAG in Azure AI Foundry, Azure AI Foundry SDK, or Azure OpenAI Service to look for answers in documents. Learn how to use Azure AI Agent Service, a comprehensive suite of feature-rich, managed capabilities, to bring together the models, data, tools, and services your enterprise needs to automate business processes There’s also a Microsoft Virtual Training Day featuring interactive sessions and expert guidance, and you can validate your skills by earning a Microsoft Certification. Safeguard your AI systems for security and fairness Widespread AI adoption demands rigorous security, fairness, and transparency safeguards to prevent bias, privacy breaches, and vulnerabilities that lead to unethical outcomes or non-compliance. Organizations must implement responsible AI through robust data governance, explainability, bias mitigation, and user safety protocols, while protecting sensitive data and ensuring outputs align with ethical standards. Our third Plan on Microsoft Learn, Build secure and responsible AI solutions and manage generative AI lifecycles, is designed to introduce the basics of AI security and responsible AI to help increase the security posture of AI environments. You’ll not only learn how to evaluate and improve generative AI outputs for quality and safety, but you’ll also: Gain an understanding of the basic concepts of AI security and responsible AI to help increase the security posture of AI environments. Learn how to assess and improve generative AI outputs for quality and safety. Discover how to help reduce risks by using Azure AI Content Safety to detect, moderate, and manage harmful content. Learn more by taking part in an interactive, expert-guided Microsoft Virtual Training Day to deepen your understanding of core AI concepts. Got a skilling question? Our new Ask Learn AI assistant is here to help Beyond our comprehensive Plans on Microsoft Learn, we’re also excited to introduce Ask Learn, our newest skilling innovation! Ask Learn is an AI assistant that can answer questions, clarify concepts, and define terms throughout your training experience. Ask Learn is your Copilot for getting skilled in AI, helping to answer your questions within the Microsoft Learn interface, so you don’t have to search elsewhere for the information. Simply click the Ask Learn icon at the top corner of the page to activate! Begin your generative AI skilling journey with curated Azure skilling Plans Azure AI Foundry provides the necessary platform to train, test, and deploy AI solutions at scale, and with the expert-curated skilling resources available in our newly refreshed Plans on Microsoft learn, your teams can accelerate the creation of intelligent, self-improving AI agents tailored to your business needs. Get started today! Find the best model for your generative AI solution with Azure AI Foundry Create agentic AI solutions by using Azure AI Foundry Build secure and responsible AI solutions and manage generative AI lifecyclesRAG Time: Ultimate Guide to Mastering RAG!
RAG Time is a brand-new AI learning series designed to help developers unlock the full potential of Retrieval-Augmented Generation (RAG). If you’ve been looking for a way to build smarter, more efficient AI systems—join us in RAG Time, every Wednesday 9AM PT from March 5 through April 2 on Microsoft Developer YouTube. What's in RAG Time? RAG Time is a five-part learning journey, with new videos and blog posts releasing every week in March. The series features: 🔥 Expert-led discussions breaking down RAG fundamentals and best practices 🎤 Exclusive leadership interviews with AI leaders ⚡ Hands-on demos & real-world case studies showing RAG in action 🎨 Creative doodle summaries making complex concepts easier to grasp and remember 🛠 Samples & resources in the RAG Time repository so you can start building today What You’ll Learn The series is structured into five learning journeys, each tackling a crucial aspect of RAG-powered AI: 📌 March 5th, 9AM PT - Journey 1: RAG and Knowledge Retrieval Fundamentals – Start with the basics! Learn how RAG, search indexing, and vector search work together to create smarter AI retrieval systems. 📌 March 12th, 9AM PT - Journey 2: Build the Ultimate Retrieval System for RAG – Go beyond the fundamentals with hybrid search, semantic ranking, and relevance tuning to refine how AI retrieves the most relevant information. 📌 March 19th, 9AM PT - Journey 3: Optimize Your Vector Index for Scale – Learn how to scale vector search efficiently, optimize storage, and implement advanced techniques like quantization and Matryoshka learning for large-scale AI applications. 📌 March 26th, 9AM PT - Journey 4: RAG for All Your Data: Multimodal and Beyond – Move beyond text-based retrieval! Discover how to integrate images, audio, and structured data into your RAG workflows and leverage multimodal pipelines for next-level AI capabilities. 📌 April 2nd, 9AM PT - Journey 5: Hero Use Cases for RAG – Explore real-world implementations, industry-leading examples, and best practices, while diving into Responsible AI considerations to ensure ethical and impactful solutions. Why You Should Watch If you're a developer, data scientist, or AI enthusiast, this series is built for you! Whether you’re just getting started or looking to master enterprise-grade retrieval systems, RAG Time delivers practical knowledge, hands-on resources, and expert insights to help you stay ahead. Journey starts here 🚀 Start your journey from the RAG Time repo: https://aka.ms/rag-time. You'll find all the information about the video series, samples, documentation and doodles in the repo! Share your experience and feedback on GitHub discussions.
