Model Mondays S2E11: Exploring Speech AI in Azure AI Foundry
1. Weekly Highlights This week’s top news in the Azure AI ecosystem included: Lakuna — Copilot Studio Agent for Product Teams: A hackathon project built with Copilot Studio and Azure AI Foundry, Lakuna analyzes your requirements and docs to surface hidden assumptions, helping teams reflect, test, and reduce bias in product planning. Azure ND H200 v5 VMs for AI: Azure Machine Learning introduced ND H200 v5 VMs, featuring NVIDIA H200 GPUs (over 1TB GPU memory per VM!) for massive models, bigger context windows, and ultra-fast throughput. Agent Factory Blog Series: The next wave of agentic AI is about extensibility: plug your agents into hundreds of APIs and services using Model Connector Protocol (MCP) for portable, reusable tool integrations. GPT-5 Tool Calling on Azure AI Foundry: GPT-5 models now support free-form tool calling—no more rigid JSON! Output SQL, Python, configs, and more in your preferred format for natural, flexible workflows. Microsoft a Leader in 2025 Gartner Magic Quadrant: Azure was again named a leader for Cloud Native Application Platforms—validating its end-to-end runway for AI, microservices, DevOps, and more. 2. Spotlight On: Azure AI Foundry Speech Playground The main segment featured a live demo of the new Azure AI Speech Playground (now part of Foundry), showing how developers can experiment with and deploy cutting-edge voice, transcription, and avatar capabilities. Key Features & Demos: Speech Recognition (Speech-to-Text): Try real-time transcription directly in the playground—recognizing natural speech, pauses, accents, and domain terms. Batch and Fast transcription options for large files and blob storage. Custom Speech: Fine-tune models for your industry, vocabulary, and noise conditions. Text to Speech (TTS): Instantly convert text into natural, expressive audio in 150+ languages with 600+ neural voices. Demo: Listen to pre-built voices, explore whispering, cheerful, angry, and more styles. Custom Neural Voice: Clone and train your own professional or personal voice (with strict Responsible AI controls). Avatars & Video Translation: Bring your apps to life with prebuilt avatars and video translation, which syncs voice-overs to speakers in multilingual videos. Voice Live API: Voice Live API (Preview) integrates all premium speech capabilities with large language models, enabling real-time, proactive voice agents and chatbots. Demo: Language learning agent with voice, avatars, and proactive engagement. One-click code export for deployment in your IDE. 3. Customer Story: Hilo Health This week’s customer spotlight featured Helo Health—a healthcare technology company using Azure AI to boost efficiency for doctors, staff, and patients. How Hilo Uses Azure AI: Document Management: Automates fax/document filing, splits multi-page faxes by patient, reduces staff effort and errors using Azure Computer Vision and Document Intelligence. Ambient Listening: Ambient clinical note transcription captures doctor-patient conversations and summarizes them for easy EHR documentation. Genie AI Contact Center: Agentic voice assistants handle patient calls, book appointments, answer billing/refill questions, escalate to humans, and assist human agents—using Azure Communication Services, Azure Functions, FastAPI (community), and Azure OpenAI. Conversational Campaigns: Outbound reminders, procedure preps, and follow-ups all handled by voice AI—freeing up human staff. Impact: Hilo reaches 16,000+ physician practices and 180,000 providers, automates millions of communications, and processes $2B+ in payments annually—demonstrating how multimodal AI transforms patient journeys from first call to post-visit care. 4. Key Takeaways Here’s what you need to know from S2E11: Speech AI is Accessible: The Azure AI Foundry Speech Playground makes experimenting with voice recognition, TTS, and avatars easy for everyone. From Playground to Production: Fine-tune, export code, and deploy speech models in your own apps with Azure Speech Service. Responsible AI Built-In: Custom Neural Voice and avatars require application and approval, ensuring ethical, secure use. Agentic AI Everywhere: Voice Live API brings real-time, multimodal voice agents to any workflow. Healthcare Example: Hilo’s use of Azure AI shows the real-world impact of speech and agentic AI, from patient intake to after-visit care. Join the Community: Keep learning and building—join the Discord and Forum. Sharda's Tips: How I Wrote This Blog I organize key moments from each episode, highlight product demos and customer stories, and use GitHub Copilot for structure. For this recap, I tested the Speech Playground myself, explored the docs, and summarized answers to common developer questions on security, dialects, and deployment. Here’s my favorite Copilot prompt this week: "Generate a technical blog post for Model Mondays S2E11 based on the transcript and episode details. Focus on Azure Speech Playground, TTS, avatars, Voice Live API, and healthcare use cases. Add practical links for developers and students!" Coming Up Next Week Next week: Observability! Learn how to monitor, evaluate, and debug your AI models and workflows using Azure and OpenAI tools. Register For The Livestream – Sep 1, 2025 Register For The AMA – Sep 5, 2025 Ask Questions & View Recaps – Discussion Forum About Model Mondays Model Mondays is your weekly Azure AI learning series: 5-Minute Highlights: Latest AI news and product updates 15-Minute Spotlight: Demos and deep dives with product teams 30-Minute AMA Fridays: Ask anything in Discord or the forum Start building: Register For Livestreams Watch Past Replays Register For AMA Recap Past AMAs Join The Community Don’t build alone! The Azure AI Developer Community is here for real-time chats, events, and support: Join the Discord Explore the Forum About Me I'm Sharda, a Gold Microsoft Learn Student Ambassador focused on cloud and AI. Find me on GitHub, Dev.to, Tech Community, and LinkedIn. In this blog series, I share takeaways from each week’s Model Mondays livestream.Model Mondays S2E10: Automating Document Processing with AI
1. Weekly Highlights We kicked off with the top news and updates in the Azure AI ecosystem: Agent Factory Blog Series: A new 6-part blog series on designing reliable, agentic AI—exploring multi-step, collaborative agents that reflect, plan, and adapt using tool integrations and design patterns. Text PII Preview in Azure AI Language: Now redacts PII (like date of birth, license plates) in major European languages, with better accuracy for UK bank entities. Claude Opus 4.1 in Copilot Pro & Enterprise: Public preview brings smarter summaries, tool assistant thinking, and "Ask Mode" in VS Code.Now leverages stronger computer vision algorithms for table parsing—achieving 94-97% accuracy across Latin, Chinese, Japanese, and Korean—with sub-10ms latency. Mistral Document AI in Azure Foundry: Instantly turn PDFs, contracts, and scanned docs into structured JSON with tables, headings, and LaTeX support. Serverless, multilingual, secure, and perfect for regulated industries. 2. Spotlight On: Document Intelligence with Azure & Mistral This week’s spotlight was a hands-on exploration of document processing, featuring both Microsoft and Mistral AI experts. Why Document Processing? Unstructured data—receipts, forms, handwritten notes—are everywhere. Modern document AI can extract, structure, and even annotate this data, fueling everything from search to RAG pipelines. Azure Document Intelligence: State-of-the-art OCR and table extraction with super-high accuracy and speed. Handles multi-language, complex layouts, and returns structured outputs ready for programmatic use. Mistral Document AI: Transforms PDFs and scanned docs into JSON, retaining complex formatting, tables, images, and even LaTeX. Supports custom schema extraction, image/document annotations, and returns everything in one API call. Integrates seamlessly with Azure AI Foundry and developer workflows. Demo Highlights: Extracting Receipts: OCR accurately pulls out store, date, and transaction details from photos. Handwriting Recognition: Even historical documents (like Thomas Jefferson’s letters) are parsed with surprising accuracy. Tables & Structured Data: Financial statements and reports converted into structured markdown and JSON—ready for downstream apps. Advanced Annotations: Define your own schema (via JSON Schema or Pydantic), extract custom fields, classify images, summarize documents, and even translate summaries—all in a single call. 3. Customer Story: Oracle Health Oracle Health shared how agentic AI and fine-tuned models are revolutionizing clinical workflows: Problem: Clinicians spend hours on documentation, searching records, and manual data entry—reducing time for patient care. Solution: Oracle’s clinical AI agents automate chart reviews, data extraction, and even conversational Q&A—while keeping humans in the loop for safety. Technical Highlights: Multi-agent architecture understands provider specialty and context. Orchestrator model "routes" requests to the right agent or plugin, extracting needed arguments from context. Fine-tuning was key: For low latency, Oracle used lightweight models (like GPT-4 Mini) and fine-tuned on their data—achieving sub-800ms responses, with accuracy matching larger models. Fine-tuning also allowed for nuanced tool selection, argument extraction, and rule-based orchestration—better than prompt engineering alone. Used LoRA for efficient, targeted fine-tuning without erasing base model knowledge. Live Demo: Agent summarizes patient history, retrieves lab results, filters for abnormals, and answers follow-up questions—all conversationally. Fine-tuned orchestrator chooses the right tool and context for each doctor’s workflow. Result: 1-2 hours saved per day, more time for patients, and happier doctors! 4. Key Takeaways Here are the key learnings from this episode: Document AI is Production-Ready: Azure Document Intelligence and Mistral Document AI offer fast, accurate, and customizable document parsing for real enterprise needs. Schema-Driven Extraction & Annotation: Define your own schemas and extract exactly what you want—no more one-size-fits-all. Fine-Tuning Unlocks Performance: For low latency and high accuracy, fine-tuning lightweight models beats prompt engineering in complex, rule-based agent workflows. Agentic Workflows in Action: Multi-agent systems can automate complex tasks, route requests, and keep humans in control, especially in regulated domains like healthcare. Community & Support: Join the Discord and Forum to ask questions, share use cases, and connect with the team. Sharda's Tips: How I Wrote This Blog Writing this recap is all about sharing what I learned and making it practical for the community! I start by organizing the key highlights, then walk through customer stories and demos, using simple language and real-world examples. Copilot helps me structure and clarify my notes, especially when summarizing technical sections. Here’s the prompt I used for Copilot this week: "Generate a technical blog post for Model Mondays S2E10 based on the transcript and episode details. Focus on document processing with Azure AI and Mistral, include customer demos, and highlight practical workflows and fine-tuning. Make it clear and approachable for developers and students." Every episode inspires me to try these tools myself, and I hope this blog makes it easy for you to start, too. If you have questions or want to share your own experience, I’d love to hear from you! Coming Up Next Week Next week: Text & Speech AI Playgrounds! Learn how to build and test language and speech models, with live demos and expert guests. | Register For The Livestream – Aug 25, 2025 | Register For The AMA – Aug 29, 2025 | Ask Questions & View Recaps – Discussion Forum About Model Mondays Model Mondays is a weekly series to build your Azure AI IQ with: 5-Minute Highlights: News & updates on Mondays 15-Minute Spotlight: Deep dives into new features, models, and protocols 30-Minute AMA Fridays: Live Q&A with product teams and experts Get started: Register For Livestreams Watch Past Replays Register For AMA Recap Past AMAs Join The Community Don’t build alone! Join the Azure AI Developer Community for real-time chats, events, support, and more: Join the Discord Explore the Forum About Me I'm Sharda, a Gold Microsoft Learn Student Ambassador focused on cloud and AI. Find me on GitHub, Dev.to, Tech Community, and LinkedIn. In this blog series, I share takeaways from each week’s Model Mondays livestream.Model Mondays S2E8: On-Device & Local AI
Model Mondays S2E8: On-Device & Local AI Welcome to Episode 8! This week, we explored how AI is moving from the cloud to your own device, making it faster, more private, and more accessible. We also saw a real-world customer story from Xander Glasses, showing how AI can help people with hearing loss. RFD Observability tools in Azure AI Foundry: Real-time model telemetry, auto evals, quick evals, Python grader. GitHub Copilot Pro with Spark: AI pair programmer for code explanation and workflow suggestions. Synthetic Data for Vision Models: Training accurate models with procedurally generated data. Agent-Friendly Websites: Making sites accessible to AI agents via APIs, semantic markup, and OpenAPI specs. MCP (Model Context Protocol): Standardizing agent memory and context for scalable AI.How Microsoft Semantic Kernel Transforms Proven Workflows into Intelligent Agents
Most developers today face a common challenge when integrating AI into their applications: the gap between natural language prompts and actual code execution. While services like OpenAI's ChatGPT excel at generating responses, they can't directly interact with your existing systems, databases, or business logic. You're left building complex orchestration layers, managing function calls manually, and creating brittle workflows that break when requirements change. Microsoft Semantic Kernel changes this paradigm entirely. Unlike traditional LLM integrations where you send a prompt and receive text, Semantic Kernel acts as an AI orchestration layer that bridges natural language with your existing codebase. Semantic Kernel intelligently decides which of your trusted functions to execute, chains your reliable workflows together automatically, and handles the complete workflow from user intent to business outcome using your proven business logic rather than asking the LLM to handle complex tasks with the risk of hallucinating solutions. What Makes Semantic Kernel Different The Traditional (Novice) LLM Integration Problem Meet Kemi, a data analyst who has spent months perfecting a Python script that generates exactly the sales visualizations her team needs. Her workflow is reliable: run the script, review the charts, write insights based on patterns she knows matter to the business, and deliver a concise report. Excited about AI's potential, Kemi decides to "upgrade" her process with ChatGPT. She uploads her sales data and asks the model to create visualizations and analysis. The LLM responds by generating an entirely new script with a dozen different chart types - many irrelevant to her business needs. She then has to upload the generated images back to the model for analysis, hoping it will provide accurate insights. The result? Instead of streamlining her proven workflow, Kemi now has: Unreliable outputs: The LLM generates different charts each time, some irrelevant to business decisions Loss of domain expertise: Her carefully crafted analysis logic is replaced by generic AI interpretations Broken workflow: What was once a single script is now a multi-step process of uploading, generating, downloading, and re-uploading Reduced confidence: She can't trust the AI's business recommendations the way she trusted her own tested methodology More complexity, not less: Her "AI upgrade" created more steps and uncertainty than her original manual process Kemi's experience reflects a common pitfall: replacing proven business logic with unpredictable LLM generation rather than enhancing existing workflows with intelligent orchestration. A Better Approach: Semantic Kernel Integration In this article, I present a better approach that solves Kemi's problem entirely. Instead of replacing her proven workflows with unpredictable AI generation, we'll show how Microsoft Semantic Kernel transforms her existing script into an intelligent agent that preserves her business logic while adding natural language control. By the end of this article, you'll have a solid grasp of how to integrate Semantic Kernel into your own workflows - whether that's connecting weather APIs for automated marketing campaigns, database queries for sales reporting, or Teams notifications for development task management. The principles you'll learn here apply to automating any specific marketing, sales, or development task where you want AI orchestration without sacrificing the reliability of your existing business logic. The Semantic Kernel Transformation Let's see how Semantic Kernel solves Kemi's workflow problem by transforming her existing script into an intelligent agent that preserves her business logic while adding natural language orchestration. The Complete Example Before: Kemi's Original Script After: Smart Business Agent Full Repository: semantic-kernel-business-agent Kemi's Original Functions Kemi's script contains two core functions that she's refined over months: get_sales_summary(): Calculates total sales, daily averages, and key metrics create_basic_chart(): Generates a reliable sales trend visualization These functions work perfectly for her needs, but require manual orchestration and individual execution. Setting Up the Foundation First, Kemi needs to install the required libraries and set up her OpenAI credentials: pip install semantic-kernel pandas matplotlib python-dotenv She creates a .env file to securely store her OpenAI API key: OPENAI_API_KEY=your-openai-api-key-here Get your OpenAI API key from platform.openai.com → API Keys Step 1: From Manual Function Calls to Kernel Functions In her original script, Kemi had to manually orchestrate everything: # From basic_data_analysis.py - Kemi's manual workflow analyzer = DataAnalyzer() print(analyzer.get_sales_summary()) # She manually calls this analyzer.create_basic_chart() # Then manually calls this With Semantic Kernel, she transforms these exact same functions into AI-discoverable capabilities: from semantic_kernel.functions import kernel_function from typing import Annotated @kernel_function( description="Get sales performance summary with total sales, averages, and trends", name="get_sales_summary" ) def get_sales_summary(self) -> Annotated[str, "Sales summary with key metrics"]: # Kemi's exact same trusted business logic - unchanged! total_sales = self.sales_data['sales'].sum() avg_daily_sales = self.sales_data['sales'].mean() return f"Total: ${total_sales:,}, Daily Avg: ${avg_daily_sales:.2f}" She's not replacing her proven logic with AI generation - she's making her existing, reliable functions available to intelligent orchestration. Step 2: Enhancing Her Chart Function with Smart Parameters Kemi's original create_basic_chart() only made one type of chart. With SK, she can enhance it to be more versatile while keeping the core logic: ( description="Create and save a sales performance chart visualization", name="create_sales_chart" ) def create_sales_chart( self, chart_type: Annotated[str, "Type of chart: 'trend', 'regional', or 'product'"] = "trend" ) -> Annotated[str, "Confirmation that chart was created"]: # Kemi's same matplotlib logic, now with intelligent chart selection plt.figure(figsize=(12, 8)) if chart_type == "trend": plt.plot(self.sales_data['date'], self.sales_data['sales'], marker='o') plt.title('Sales Trend Over Time', fontsize=16) # ... rest of her charting logic Step 3: Adding New Capabilities She Always Wanted Now she can add functions she never had time to build manually, like automated insights and report sending: ( description="Send performance report via email to team", name="send_report" ) def send_report(self, recipient: Annotated[str, "Email address"]) -> Annotated[str, "Confirmation"]: # For now, simulated - but she could easily integrate real email here return f"📧 Performance report sent to {recipient}" Step 4: Creating the Intelligent Agent Here's where the magic happens - connecting her functions to Semantic Kernel: from semantic_kernel import Kernel from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion from semantic_kernel.connectors.ai import FunctionChoiceBehavior from dotenv import load_dotenv load_dotenv() # Load her OpenAI key class SmartBusinessAgent: def __init__(self): # Initialize the kernel self.kernel = Kernel() # Connect to OpenAI self.kernel.add_service( OpenAIChatCompletion( service_id="business_agent", api_key=os.getenv("OPENAI_API_KEY"), ai_model_id="gpt-4o-mini" ) ) # Register Kemi's functions as AI-accessible tools self.kernel.add_plugin(SmartBusinessPlugin(), plugin_name="business") # Enable automatic function orchestration self.execution_settings = OpenAIChatPromptExecutionSettings( function_choice_behavior=FunctionChoiceBehavior.Auto() ) Step 5: The Natural Language Interface Now Kemi can interact with her proven workflows using natural language: async def process_request(self, user_request: str) -> str: result = await self.kernel.invoke_prompt( prompt=f"You are a business intelligence agent. You can analyze sales data, create charts, generate insights, and send reports.\n\nRequest: {user_request}", arguments=KernelArguments(settings=self.execution_settings) ) return str(result) The Transformation in Action Before - Kemi's manual, step-by-step process: analyzer = DataAnalyzer() summary = analyzer.get_sales_summary() # She decides to call this chart = analyzer.create_basic_chart() # Then she decides to call this # Then she manually writes insights and sends emails After - Intelligent orchestration of her same functions: agent = SmartBusinessAgent() response = await agent.process_request( "Analyze our sales performance, create relevant charts, and email the full report to sarah@company.com" ) # SK automatically calls: get_sales_summary() → create_sales_chart("trend") → # create_sales_chart("regional") → get_business_insights() → send_report("sarah@company.com") The breakthrough: Kemi keeps her trusted business logic intact while gaining an intelligent interface that can understand complex requests, automatically determine which of her functions to call, and handle multi-step workflows - all while using her proven, reliable analysis methods instead of unpredictable AI generation. This is the core power of Semantic Kernel: enhancing existing workflows with AI orchestration rather than replacing proven business logic with risky hallucination-prone generation. Whether you're working with weather APIs for marketing automation, database queries for sales reporting, or Teams notifications for development workflows, these same patterns apply. You can keep your proven logic and enhance with AI orchestration. Try It Yourself Ready to transform your own workflows? Here's how to get started: 1. Clone and Run the Complete Example git clone https://github.com/your-username/semantic-kernel-business-agent cd semantic-kernel-business-agent pip install -r requirements.txt 2. Set Up Your Environment # Add your OpenAI API key cp .env.example .env # Edit .env and add: OPENAI_API_KEY=your-key-here 3. Experience the Transformation # Run Kemi's original manual script python basic_data_analysis.py # Then run the intelligent agent python smart_business_agent.py 4. Experiment with Natural Language Requests Try these prompts with the smart agent: "Give me a comprehensive sales analysis with multiple chart types" "Create regional performance charts and send insights to my email" "What trends should we focus on for next quarter's strategy?" Watch how Semantic Kernel automatically orchestrates Kemi's trusted functions to fulfill complex, multi-step requests. Next Steps: Adapt to Your Workflow Take your own scripts and apply the same transformation: Identify your core functions (like Kemi's get_sales_summary() and create_basic_chart()) Add decorators with clear descriptions Create your agent class connecting to your preferred LLM Test with natural language requests that combine multiple functions The full repository includes additional examples and documentation to help you extend these concepts to your specific use cases. The goal isn't to replace your expertise with AI - it's to make your expertise accessible through intelligent, natural language orchestration. Start with the working example, then gradually transform your own workflows. You'll discover that Semantic Kernel doesn't just automate tasks - it amplifies your existing capabilities while keeping you in control of the business logic that matters. Further Reading Introduction to Semantic KernelFix Broken Migrations with AI Powered Debugging in VS Code Using GitHub Copilot
Data is at the heart of every application. But evolving your schema is risky business. One broken migration, and your dev or prod environment can go down. We've all experienced it: mismatched columns, orphaned constraints, missing fields, or that dreaded "table already exists" error. But what if debugging migrations didn’t have to be painful? What if you could simply describe the error or broken state, and AI could fix your migration in seconds? In this blog, you’ll learn how to: Use GitHub Copilot to describe and fix broken migrations with natural language Catch schema issues like incorrect foreign keys before they block your workflow Validate and deploy your database changes using GibsonAI CLI Broken migrations are nothing new. Whether you're working on a side project or part of a large team, it’s all too easy to introduce schema issues that can block deployments or corrupt local environments. Traditionally, fixing them means scanning SQL files, reading error logs, and manually tracking down what went wrong. But what if you could skip all that? What if you could simply describe the issue in plain English and AI would fix it for you? That’s exactly what GitHub Copilot let you do, right from within VS Code. What You Need: Visual Studio Code Installed Account in GitHub Sign up with GitHub Copilot GibsonAI CLI installed and logged in Let’s Break (and Fix) a Migration: Here’s a common mistake. Say you create two tables: users and posts. CREATE TABLE users ( id UUID PRIMARY KEY, name TEXT, email TEXT UNIQUE ); CREATE TABLE posts ( id UUID PRIMARY KEY, title TEXT, user_id UUID REFERENCES user(id) ); The problem? The posts table refers to a table called user, but you named it users. This one-word mistake breaks the migration. If you've worked with relational databases, you’ve probably run into this exact thing. Just Ask a GitHub Copilot: Instead of troubleshooting manually, open Copilot Chat and ask: “My migration fails because posts.user_id references a missing user table. Can you fix the foreign key?” Copilot understands what you're asking. It reads the context and suggests the fix: CREATE TABLE posts ( id UUID PRIMARY KEY, title TEXT, user_id UUID REFERENCES users(id) ); It even explains what changed, so you learn along the way. Wait — how does Copilot know what I mean? GitHub Copilot is smart enough to understand your code, your errors, and even what you’re asking in plain English. It doesn’t directly connect to GibsonAI. You’ll use the GibsonAI CLI for that, but Copilot helps you figure things out and fix your code faster. Validating with GibsonAI Once Copilot gives you the fixed migration, it’s time to test it. Run: gibson validate This checks your migration and schema consistency. When you're ready to apply it, just run: gibson deploy GibsonAI handles the rest so no broken chains, no surprises. Why This Works Manual debugging of migrations is frustrating and error prone. GibsonAI with GitHub Copilot: Eliminates guesswork in debugging You don’t need to Google every error Reduces time to fix production schema issues You stay in one tool: VS Code You learn while debugging Whether you're a student learning SQL or a developer on a fast moving team, this setup helps you recover faster and ship safer. Fixing migrations used to be all trial and error, digging through files and hoping nothing broke. It was time-consuming and stressful. Now with GitHub Copilot and GibsonAI, fixing issues is fast and simple. Copilot helps you write and correct migrations. GibsonAI lets you validate and deploy with confidence. So next time your migration fails, don’t panic. Just describe the issue to GitHub Copilot, run a quick check with GibsonAI, and get back to building. Ready to try it yourself? Sign up atgibsonai.com Want to Go Further? If you’re ready to explore more powerful workflows with GibsonAI, here are two great next steps: GibsonAI MCP Server – Enable Copilot Agent Mode to integrate schema intelligence directly into your dev environment. Automatic PR Creation for Schema Changes – The in-depth guide on how to automate pull requests for database updates using GibsonAI. Want to Know More About GitHub Copilot? Explore these resources to get the most out of Copilot: Get Started with GitHub Copilot Introduction to prompt engineering with GitHub Copilot GitHub Copilot Agent Mode GitHub Copilot Customization Use GitHub Copilot Agent Mode to create a Copilot Chat application in 5 minutes Deploy Your First App Using GitHub Copilot for Azure: A Beginner’s Guide That's it, folks! But the best part? You can become part of a thriving community of learners and builders by joining the Microsoft Student Ambassadors Community. Connect with like minded individuals, explore hands-on projects, and stay updated with the latest in cloud and AI. 💬 Join the community on Discord here and explore more benefits on the Microsoft Learn Student Hub.S2:E7 · AI-Assisted Azure Development
Welcome to Episode 7! This week, we explore how AI is transforming Azure development. We’ll break down two key tools—Azure MCP Server and GitHub Copilot for Azure—and see how they make working with Azure resources easier for everyone. We’ll also look at a real customer story from SightMachine, showing how AI streamlines manufacturing operations.
CampusSphere: Building the Future of Campus AI with Microsoft's Agentic Framework
Project Overview We are a team of Imperial College Students committed to improving campus life through innovative multi-agent solutions. CampusSphere leverages Microsoft Azure AI capabilities to automate core university campus services. We created an end-to-end solution that allows both students and staff to access a multi-agent framework for room/gym booking, attendance tracking, calendar management, IoT monitoring and more. 🔭 Our Initial Vision: Reimagining Campus Technology When our team at Imperial College London embarked on the CampusSphere project as part of Microsoft's Agentic Campus initiative, we had one clear ambition: to create an intelligent campus ecosystem that would fundamentally change how students, faculty, and staff interact with university services. The inspiration came from a simple observation—despite living in an age of advanced AI, campus technology remained frustratingly fragmented. Students juggled multiple portals for course registration, room booking, dining services, and academic support. Faculty members navigated separate systems for teaching, research, and administrative tasks. The result? Countless hours wasted on mundane navigation tasks that could be better spent on learning, teaching, and innovation. Our vision was ambitious: create a single, intelligent interface that could understand natural language, anticipate user needs, and seamlessly integrate with existing campus infrastructure. We didn't just want to build another campus app—we wanted to demonstrate how Microsoft's agentic AI technologies could create a truly intelligent campus companion. 🧠 Enter CampusSphere CampusSphere is an intelligent campus assistant made up of multiple AI agents, each with a specific domain of expertise — all communicating seamlessly through a centralized architecture. Think of it as a digital concierge for campus life, where your calendar, attendance, IoT data, and facility bookings are coordinated by specialized GPT-powered agents. Here’s what we built: TriageAgent – the brain of the system, using Retrieval-Augmented Generation (RAG) to understand user intent CalendarAgent – handles scheduling, bookings, and reminders AttendanceAgent – tracks check-ins automatically IoTAgent – monitors real-time sensor data from classrooms and labs GymAgent – manages access and reservations for sports facilities 30+ MCP Tools – perform SQL queries, scrape web data, and connect with external APIs All of this is built on Microsoft Azure AI, Semantic Kernel, and Model Context Protocol (MCP) — making it scalable, secure, and lightning fast. 🖥️ The Tech Stack Our Azure-powered architecture showcases a modular and scalable approach to real-time data processing and intelligent agent coordination. The frontend is built using React with a Vite development server, providing a fast and responsive user interface. When users submit a prompt, it travels to a Flask backend server acting as the Triage agent, which intelligently delegates tasks to a FastAPI agent service. This FastAPI service asynchronously communicates with individual agents and handles responses efficiently. Complex queries are routed to MCP Tools, which interact with the CosmosDB-powered Campus Database. Simultaneously, real-time synthetic IoT data is pushed into the database via Azure Function Apps and Azure IoT Hub. Authentication is securely managed: users log in through the frontend, receive a token from the database API server, and use it for authorized access to MCP services, with permissions enforced based on user roles using our custom MCP server implementation. This robust architecture enables seamless integration, real-time data flow, and secure multi-agent collaboration across Azure services. Our system leverages a multi-agent architecture designed to intelligently coordinate task execution across specialized services. At the core is the TriageAgent, which uses Retrieval-Augmented Generation (RAG) to interpret user prompts, enrich them with relevant context, and determine the optimal response path. Based on the nature of the request, it may handle the response directly, seek clarification, or delegate tasks to specific agents via FastAPI. Each specialized agent has a clearly defined role: AttendanceAgent: Interfaces with CosmosDB-backed FastAPI endpoints to check student attendance, using filters like event name, student ID, or date. IoTAgent: Monitors room conditions (e.g., temperature, CO₂ levels) and flags anomalies using real-time data from Azure IoT Hub, processed via FastAPI. CalendarAgent: Handles scheduling, availability checks, and event creation by querying or updating CosmosDB through FastAPI. Future integration with Microsoft Graph API is planned for direct calendar syncing. Gym Slot Agent: Checks available times for gym sessions using dedicated MCP tools. The triage agent serves as the orchestrator, breaking down complex requests (like "Book a gym session") into subtasks. It consults relevant agents (e.g., calendar and gym slot agents), merges results, and then confirms the final action with the user. This distributed and asynchronous workflow reduces backend load and enhances both responsiveness and reliability of the system. 🔮 What’s Next? Integrating CampusSphere with live systems via Microsoft OAuth is crucial for enhancing its capabilities. This integration will grant the agent authenticated access to a wider range of student data, moving beyond synthetic datasets. This expanded access to real-world information will enable deeply personalized advice, such as tailored course selection, scholarship recommendations, event suggestions, and deadline reminders, transforming CampusSphere into a sophisticated, proactive personal assistant. 🤝Meet the Team Behind CampusSphere Our success stemmed from a diverse team of innovators who brought together expertise from multiple domains: Benny Liu - https://www.linkedin.com/in/zong-benny-liu-393a4621b/ Lucas Ng - https://www.linkedin.com/in/lucas-ng-11b317203/ Lu Ju - https://www.linkedin.com/in/lu-ju/ Bruno Duaso - https://www.linkedin.com/in/bruno-duaso-jimeno-744464262/ Martim Coutinho - https://www.linkedin.com/in/martim-pereira-coutinho-116308233/ Krischad Pourpongpan - https://www.linkedin.com/in/krischadpua/ Yixu Pan - https://www.linkedin.com/in/yixu-pan/ Our collaborative approach enabled us to create a sophisticated agentic AI system that demonstrates the powerful potential of Microsoft's AI technologies in educational environments. 🧑💻 Project Repository: GitHub - Imperial-Microsoft-Agentic-Campus/CampusSphere Contribute to Imperial-Microsoft-Agentic-Campus/CampusSphere development by creating an account on GitHub. github.com Have questions about implementing similar solutions at your institution? Connect with our team members on LinkedIn—we're always excited to share knowledge and collaborate on innovative campus technology projects. 📚Get Started with Microsoft's AI Tools Ready to explore the technologies that made CampusSphere possible? Here are essential resources: Microsoft Semantic Kernel: The core framework for building AI agent orchestration systems. Learn how to create, coordinate, and manage multiple AI agents working together seamlessly. AI Agents for Beginners: A comprehensive guide to understanding and building AI agents from the ground up. Perfect for getting started with agentic AI development. Model Context Protocol (MCP): Learn about the protocol that enables secure connections between AI models and external tools and services—essential for building integrated AI systems. Windows AI Toolkit: Microsoft's toolkit for developing AI applications on Windows, providing local AI model development capabilities and deployment tools. Azure Container Apps: Understand how to deploy and scale containerized AI applications in the cloud, perfect for hosting multi-agent systems. Azure Cosmos DB Security: Essential security practices for managing data in AI applications, covering encryption, access control, and compliance.Preparing for Your Organization’s AI Workloads – Student Learning Pathways
This structured plan helps students: Plans | Microsoft Learn Build foundational knowledge of AI in the cloud. Learn how enterprise-level infrastructure supports responsible, scalable AI deployments. Explore governance and monitoring strategies to ensure security and compliance. And the best part? It’s built using Microsoft’s existing training resources plus some brand-new modules to give you an edge. Your AI Readiness Journey on Azure 🎯 Milestone 1: Getting Started with AI on Azure https://learn.microsoft.com/training/paths/introduction-to-ai-on-azure/ Begin with the basics—from machine learning concepts to practical uses of Azure AI services. 🛡️ Milestone 2: Infrastructure Essentials https://learn.microsoft.com/training/paths/manage-iam-for-ai-workloads-on-azure/ https://learn.microsoft.com/training/paths/manage-network-access-ai-workloads/ Learn how enterprises secure access and manage identities—critical for real-world applications. 📊 Milestone 3: Monitoring AI Services https://learn.microsoft.com/training/paths/monitor-ai-workloads-on-azure/ Discover how businesses ensure their models perform safely and consistently at scale. 🏛️ Milestone 4: Advanced Management & Governance https://learn.microsoft.com/training/paths/ai-workloads-governance/ Master how organizations prevent data leaks and enforce responsible AI usage. 🆕 New Training Content Just for You To make this roadmap even more student-friendly, Microsoft has introduced updated and brand-new modules, including: Azure ML Authentication & Authorization Secure Azure AI Services Restrict Workspace Network Traffic Monitor Azure ML Prevent Data Exfiltration Govern AI Services with Azure Policy 🔗 Ready to Dive In? Whether you're exploring a career in AI or just getting started with Azure, these learning paths will level up your skills while helping you understand how real-world teams manage complex AI workloads. Start your journey on Microsoft Learn and become the architect of tomorrow’s intelligent systems. 💡 Would you like a version formatted for your internal newsletter or maybe something more conversational for social media? I can easily tailor it to fit the tone or medium you're aiming for.S2E01 Recap: Advanced Reasoning Session
About Model Mondays Want to know what Reasoning models are and how you can build advanced reasoning scenarios like a Deep Research agent using Azure AI Foundry? Check out this recap from Model Mondays Season 2 Ep 1. Model Mondays is a weekly series to help you build your model IQ in three steps: 1. Catch the 5-min Highlights on Monday, to get up to speed on model news 2. Catch the 15-min Spotlight on Monday, for a deep-dive into a model or tool 3. Catch the 30-min AMA on Friday, for a Q&A session with subject matter experts Want to follow along? Register Here- to watch upcoming livestreams for Season 2 Visit The Forum- to see the full AMA schedule for Season 2 Register Here - to join the AMA on Friday Jun 20 Spotlight On: Advanced Reasoning This week, the Model Mondays spotlight was on Advanced Reasoning with subject matter expert Marlene Mhangami. In this blog post, I'll talk about my five takeaways from this episode: Why Are Reasoning Models Important? What Is an Advanced Reasoning Scenario? How Can I Get Started with Reasoning Models ? Spotlight: My Aha Moment Highlights: What’s New in Azure AI 1. Why Are Reasoning Models Important? In today's fast-evolving AI landscape, it's no longer enough for models to just complete text or summarize content. We need AI that can: Understand multi-step tasks Make decisions based on logic Plan sequences of actions or queries Connect context across turns Reasoning models are large language models (LLMs) trained with reinforcement learning techniques to "think" before they answer. Rather than simply generating a response based on probability, these models follow an internal thought process producing a chain of reasoning before responding. This makes them ideal for complex problem-solving tasks. And they’re the foundation of building intelligent, context-aware agents. They enable next-gen AI workflows in everything from customer support to legal research and healthcare diagnostics. Reason: They allow AI to go beyond surface-level response and deliver solutions that reflect understanding, not just language patterning. 2. What does Advanced Reasoning involve? An advanced reasoning scenario is one where a model: Breaks a complex prompt into smaller steps Retrieves relevant external data Uses logic to connect dots Outputs a structured, reasoned answer Example: A user asks: What are the financial and operational risks of expanding a startup to Southeast Asia in 2025? This is the kind of question that requires extensive research and analysis. A reasoning model might tackle this by: Retrieving reports on Southeast Asia market conditions Breaking down risks into financial, political, and operational buckets Cross-referencing data with recent trends Returning a reasoned, multi-part answer 3. How Can I Get Started with Reasoning Models? To get started, you need to visit a catalog that has examples of these models. Try the GitHub Models Marketplace and look for the reasoning category in the filter. Try the Azure AI Foundry model catalog and look for reasoning models by name. Example: The o-series of models from Azure Open AI The DeepSeek-R1 models The Grok 3 models The Phi-4 reasoning models Next, you can use SDKs or Playground for exploring the model capabiliies. 1. Try Lab 331 - for a beginner-friendly guide. 2. Try Lab 333 - for an advanced project. 3. Try the GitHub Model Playground - to compare reasoning and GPT models. 4. Try the Deep Research Agent using LangChain - sample as a great starting project. Have questions or comments? Join the Friday AMA on Azure AI Foundry Discord: 4. Spotlight: My Aha Moment Before this session, I thought reasoning meant longer or more detailed responses. But this session helped me realize that reasoning means structured thinking — models now plan, retrieve, and respond with logic. This inspired me to think about building AI agents that go beyond chat and actually assist users like a teammate. It also made me want to dive deeper into LangChain + Azure AI workflows to build mini-agents for real-world use. 5. Highlights: What’s New in Azure AI Here’s what’s new in the Azure AI Foundry: Direct From Azure Models - Try hosted models like OpenAI GPT on PTU plans SORA Video Playground - Generate video from prompts via SORA models Grok 3 Models - Now available for secure, scalable LLM experiences DeepSeek R1-0528 - A reasoning-optimized, Microsoft-tuned open-source model These are all available in the Azure Model Catalog and can be tried with your Azure account. Did You Know? Your first step is to find the right model for your task. But what if you could have the model automatically selected for you_ based on the prompt you provide? That's the magic of Model Router a deployable AI chat model that dynamically selects the best LLM based on your prompt. Instead of choosing one model manually, the Router makes that choice in real time. Currently, this works with a fixed set of Azure OpenAI models, including a reasoning model option. Keep an eye on the documentation for more updates. Why it’s powerful: Saves cost by switching between models based on complexity Optimizes performance by selecting the right model for the task Lets you test and compare model outputs quickly Try it out in Azure AI Foundry or read more in the Model Catalog Coming Up Next Next week, we dive into Model Context Protocol, an open protocol that empowers agentic AI applications by making it easier to discover and integrate knowledge and action tools with your model choices. Register Here to get reminded - and join us live on Monday! Join The Community Great devs don't build alone! In a fast-pased developer ecosystem, there's no time to hunt for help. That's why we have the Azure AI Developer Community. Join us today and let's journey together! Join the Discord - for real-time chats, events & learning Explore the Forum - for AMA recaps, Q&A, and help! About Me. I'm Sharda, a Gold Microsoft Learn Student Ambassador interested in cloud and AI. Find me on Github, Dev.to,, Tech Community and Linkedin. In this blog series I have summarizef my takeaways from this week's Model Mondays livestream .Create Stunning AI Videos with Sora on Azure AI Foundry!
Special credit to Rory Preddy for creating the GitHub resource that enable us to learn more about Azure Sora. Reach him out on LinkedIn to say thanks. Introduction Artificial Intelligence (AI) is revolutionizing content creation, and video generation is at the forefront of this transformation. OpenAI's Sora, a groundbreaking text-to-video model, allows creators to generate high-quality videos from simple text prompts. When paired with the powerful infrastructure of Azure AI Foundry, you can harness Sora's capabilities with scalability and efficiency, whether on a local machine or a remote setup. In this blog post, I’ll walk you through the process of generating AI videos using Sora on Azure AI Foundry. We’ll cover the setup for both local and remote environments. Requirements: Azure AI Foundry with sora model access A Linux Machine/VM. Make sure that the machine already has the package below: Java JRE 17 (Recommended) OR later Maven Step Zero – Deploying the Azure Sora model on AI Foundry Navigate to the Azure AI Foundry portal and head to the “Models + Endpoints” section (found on the left side of the Azure AI Foundry portal) > Click on the “Deploy Model” button > “Deploy base model” > Search for Sora > Click on “Confirm”. Give a deployment name and specify the Deployment type > Click “Deploy” to finalize the configuration. You should receive an API endpoint and Key after successful deploying Sora on Azure AI Foundry. Store these in a safe place because we will be using them in the next steps. Step one – Setting up the Sora Video Generator in the local/remote machine. Clone the roryp/sora repository on your machine by running the command below: git clone https://github.com/roryp/sora.git cd sora Then, edit the application.properties file in the src/main/resources/ folder to include your Azure OpenAI Credentials. Change the configuration below: azure.openai.endpoint=https://your-openai-resource.cognitiveservices.azure.com azure.openai.api-key=your_api_key_here If port 8080 is used for another application, and you want to change the port for which the web app will run, change the “server.port” configuration to include the desired port. Allow appropriate permissions to run the “mvnw” script file. chmod +x mvnw Run the application ./mvnw spring-boot:run Open your browser and type in your localhost/remote host IP (format: [host-ip:port]) in the browser search bar. If you are running a remote host, please do not forget to update your firewall/NSG to allow inbound connection to the configured port. You should see the web app to generate video with Sora AI using the API provided on Azure AI Foundry. Now, let’s generate a video with Sora Video Generator. Enter a prompt in the first text field, choose the video pixel resolution, and set the video duration. (Due to technical limitation, Sora can only generate video of a maximum of 20 seconds). Click on the “Generate video” button to proceed. The cost to generate the video should be displayed below the “Generate Video” button, for transparency purposes. You can click on the “View Breakdown” button to learn more about the cost breakdown. The video should be ready to download after a maximum of 5 minutes. You can check the status of the video by clicking on the “Check Status” button on the web app. The web app will inform you once the download is ready and the page should refresh every 10 seconds to fetch real-time update from Sora. Once it is ready, click on the “Download Video” button to download the video. Conclusion Generating AI videos with Sora on Azure AI Foundry is a game-changer for content creators, marketers, and developers. By following the steps outlined in this guide, you can set up your environment, integrate Sora, and start creating stunning AI-generated videos. Experiment with different prompts, optimize your workflow, and let your imagination run wild! Have you tried generating AI videos with Sora or Azure AI Foundry? Share your experiences or questions in the comments below. Don’t forget to subscribe for more AI and cloud computing tutorials!
