Model Mondays S2E12: Models & Observability
1. Weekly Highlights This week’s top news in the Azure AI ecosystem included: GPT Real Time (GA): Azure AI Foundry now offers GPT Real Time (GA)—lifelike voices, improved instruction following, audio fidelity, and function calling, with support for image context and lower pricing. Read the announcement and check out the model card for more details. Azure AI Translator API (Public Preview): Choose between fast Neural Machine Translation (NMT) or nuanced LLM-powered translations, with real-time flexibility for multilingual workflows. Read the announcement then check out the Azure AI Translator documentation for more details. Azure AI Foundry Agents Learning Plan: Build agents with autonomous goal pursuit, memory, collaboration, and deep fine-tuning (SFT, RFT, DPO) - on Azure AI Foundry. Read the announcement what Agentic AI involves - then follow this comprehensive learning plan with step-by-step guidance. CalcLM Agent Grid (Azure AI Foundry Labs): Project CalcLM: Agent Grid is a prototype and open-source experiment that illustrates how agents might live in a grid-like surface (like Excel). It's formula-first and lightweight - defining agentic workflows like calculations. Try the prototype and visit Foundry Labs to learn more. Agent Factory Blog: Observability in Agentic AI: Agentic AI tools and workflows are gaining rapid adoption in the enterprise. But delivering safe, reliable and performant agents requires foundation support for Observability. Read the 6-part Agent Factory series and check out the Top 5 agent observability best practices for reliable AI blog post for more details. 2. Spotlight On: Observability in Azure AI Foundry This week’s spotlight featured a deep dive and demo by Han Che (Senior PM, Core AI/ Microsoft ), showing observability end-to-end for agent workflows. Why Observability? Ensures AI quality, performance, and safety throughout the development lifecycle. Enables monitoring, root cause analysis, optimization, and governance for agents and models. Key Features & Demos: Development Lifecycle: Leaderboard: Pick the best model for your agent with real-time evaluation. Playground: Chat and prototype agents, view instant quality and safety metrics. Evaluators: Assess quality, risk, safety, intent resolution, tool accuracy, code vulnerability, and custom metrics. Governance: Integrate with partners like Cradle AI and SideDot for policy mapping and evidence archiving. Red Teaming Agent: Automatically test for vulnerabilities and unsafe behavior. CI/CD Integration: Automate evaluation in GitHub Actions and Azure DevOps pipelines. Azure DevOps GitHub Actions Monitoring Dashboard: Resource usage, application analytics, input/output tokens, request latency, cost breakdown, and evaluation scores. Azure Cost Management SDKs & Local Evaluation: Run evaluations locally or in the cloud with the Azure AI Evaluation SDK. Demo Highlights: Chat with a travel planning agent, view run metrics and tool usage. Drill into run details, debugging, and real-time safety/quality scores. Configure and run large-scale agent evaluations in CI/CD pipelines. Compare agents, review statistical analysis, and monitor in production dashboards 3. Customer Story: Saifr Saifr is a RegTech company that uses artificial intelligence to streamline compliance for marketing, communications, and creative teams in regulated industries. Incubated at Fidelity Labs (Fidelity Investments’ innovation arm), Saifr helps enterprises create, review, and approve content that meets regulatory standards—faster and with less manual effort. What Saifr Offers AI-Powered Compliance: Saifr’s platform leverages proprietary AI models trained on decades of regulatory expertise to automatically detect potential compliance risks in text, images, audio, and video. Automated Guardrails: The solution flags risky or non-compliant language, suggests compliant alternatives, and provides explanations—all in real time. Workflow Integration: Saifr seamlessly integrates with enterprise content creation and approval workflows, including cloud platforms and agentic AI systems like Azure AI Foundry. Multimodal Support: Goes beyond text to check images, videos, and audio for compliance risks, supporting modern marketing and communications teams. 4. Key Takeaways Observability is Essential: Azure AI Foundry offers complete monitoring, evaluation, tracing, and governance for agentic AI—making production safe, reliable, and compliant. Built-In Evaluation and Red Teaming: Use leaderboards, evaluators, and red teaming agents to assess and continuously improve model safety and quality. CI/CD and Dashboard Integration: Automate evaluations in GitHub Actions or Azure DevOps, then monitor and optimize agents in production with detailed dashboards. Compliance Made Easy: Safer’s agents and models help financial services and regulated industries proactively meet compliance standards for content and communications. Sharda's Tips: How I Wrote This Blog I focus on organizing highlights, summarizing customer stories, and linking to official Microsoft docs and real working resources. For this recap, I explored the Azure AI Foundry Observability docs, tested CI/CD pipeline integration, and watched the customer demo to share best practices for regulated industries. Here’s my Copilot prompt for this episode: "Generate a technical blog post for Model Mondays S2E12 based on the transcript and episode details. Focus on observability, agent dashboards, CI/CD, compliance, and customer stories. Add correct, working Microsoft links!" Coming Up Next Week Next week: Open Source Models! Join us for the final episode with Hugging Face VP of Product, live demos, and open model workflows. Register For The Livestream – Sep 15, 2025 About Model Mondays Model Mondays is your weekly Azure AI learning series: 5-Minute Highlights: Latest AI news and product updates 15-Minute Spotlight: Demos and deep dives with product teams 30-Minute AMA Fridays: Ask anything in Discord or the forum Start building: Watch Past Replays Register For AMA Recap Past AMAs Join The Community Don’t build alone! The Azure AI Developer Community is here for real-time chats, events, and support: Join the Discord Explore the Forum About Me I'm Sharda, a Gold Microsoft Learn Student Ambassador focused on cloud and AI. Find me on GitHub, Dev.to, Tech Community, and LinkedIn. In this blog series, I share takeaways from each week’s Model Mondays livestream.AZ-500: Microsoft Azure Security Technologies Study Guide
The AZ-500 certification provides professionals with the skills and knowledge needed to secure Azure infrastructure, services, and data. The exam covers identity and access management, data protection, platform security, and governance in Azure. Learners can prepare for the exam with Microsoft's self-paced curriculum, instructor-led course, and documentation. The certification measures the learner’s knowledge of managing, monitoring, and implementing security for resources in Azure, multi-cloud, and hybrid environments. Azure Firewall, Key Vault, and Azure Active Directory are some of the topics covered in the exam.Microsoft 365 Champion community call | June 2025 | AM
Join our next community call on June 24, 2025, to learn more about SharePoint agents, agent governance, and the Get Stuff Done campaign with Karuana Gatimu. Host: Tiffany Lee Guest: Karuana Gatimu Moderator: Jessie Hwang 📢 NOTE: our community call format has changed to using Teams webinars to enable more dynamic discussions! Join link is still the same but you must register to be able to join the call when it starts: https://aka.ms/M365ChampionCallAM ⏰ 🗨️ Each call includes an open Q&A discussion section, where you'll have a chance to ask your questions about Microsoft 365. 👋 Join the Microsoft 365 Champion program today! Champions combine technical acumen with people skills to drive meaningful change. Our community calls are open to everyone but only Champions have access to the presentation resources (access link in the initial welcome email and in the monthly newsletters). Join now: https://aka.ms/M365Champions. Note: If you are unable to watch the recording on YouTube, try watching it here.Microsoft 365 Champion community call | June 2025 | PM
Join our next community call on June 24, 2025, to learn more about SharePoint agents, agent governance, and the Get Stuff Done campaign with Karuana Gatimu. Host: Tiffany Lee Guest: Karuana Gatimu Moderator: Jessie Hwang 📢 NOTE: our community call format has changed to using Teams webinars to enable more dynamic discussions! Join link is still the same but you must register to be able to join the call when it starts: https://aka.ms/M365ChampionCallPM ⏰ 🗨️ Each call includes an open Q&A discussion section, where you'll have a chance to ask your questions about Microsoft 365. 👋 Join the Microsoft 365 Champion program today! Champions combine technical acumen with people skills to drive meaningful change. Our community calls are open to everyone but only Champions have access to the presentation resources (access link in the initial welcome email and in the monthly newsletters). Join now: https://aka.ms/M365Champions. Note: If you are unable to watch the recording on YouTube, try watching it here.MVP Collective Launches In-Depth Guide on SharePoint Content AI
MVP and Regional Director Gokan Ozcifci, together with eight fellow Microsoft MVPs, has co-authored SharePoint Content AI, Solutions and Advanced Administration - a new book that delves into the intersection of artificial intelligence and SharePoint. We spoke with the authors to learn more about their collaboration, what inspired the project, and the key themes they explore. What inspired you to collaborate on this eBook, and how did the idea for focusing on AI and SharePoint come about? Gokan Ozcifci, Belgium: The inspiration behind this book stems from observing how rapidly AI is transforming the way organizations manage content, particularly within Microsoft 365 and SharePoint. As this transformation accelerated, I noticed a growing gap: many industry experts, digital transformation directors, and business leaders were eager to embrace these AI-driven tools but found it difficult to understand how capabilities like Autofill, AI-powered metadata, OCR, or governance solutions translate into real-world value. That’s when a group of SharePoint enthusiasts, MVPs, consultants, and practice owners joined forces with a shared goal—to bridge that gap. We set out to create more than just a technical guide. We aimed to build a resource grounded in practical experience, offering clear explanations and actionable insights for those navigating the evolving world of Content AI. SharePoint naturally became our focal point due to its central role in enterprise content management and its rapid evolution in tandem with Microsoft’s AI strategy. Our goal was to demystify the technology, understand the requirements, translate the needs, and show how AI can empower organizations to manage content more intelligently, securely, and efficiently. Can you share a real-world example where AI significantly enhanced SharePoint content management or administration? Frane Borozan – MVP, Croatia: A global enterprise had a use case to classify thousands of legacy contracts in SharePoint. SharePoint content AI extracts metadata, such as expiration dates, the names of signatories, and the validity period of the contract, as well as similar information that is available within the contract. This cut manual effort by almost 90% to hire the workforce to review all these legacy contracts. Noorez Khamis – MVP, Canada: For one client they previously had 5–10 interns manually logging into 20 banking websites each month to download client statements, upload them to SharePoint, tag metadata, and update Salesforce—an inefficient and error-prone process. Now, an automated solution using Power Automate Desktop Flows handles document retrieval, Syntex extracts key metadata, and a validation Power App ensures data accuracy before integrating with Salesforce for approvals and updates. This end-to-end system eliminates manual effort, increases accuracy, and streamlines document processing across platforms. Drew Madelung – MVP, USA: As an M365 consultant, I work with multiple customers and write unique and complex statements of work. I actively utilize SharePoint Content AI features, such as autofill columns, to help summarize and provide key metadata from statements of work, enabling me to discover details from prior and existing projects where overlap is likely to occur. Mike Maadarani – MVP, Canada: AI was deployed at a university to manage their application admissions process. The Content AI significantly improved the classification and extraction of information from various application formats. This process reduced manual work by over 98%, resulting in substantial savings and a high return on investment for the client. Antonio Maio – MVP, Canada: I had a client who greatly benefited from Content AI models in SharePoint Online. They’re a corporate real estate firm that utilizes Content AI models to process lease agreements and rental contracts, automatically extracting key metadata values from these content types. This metadata automatically populates columns in SharePoint libraries, which then drives business process automation and retention policies for those documents. This all happens by users simply uploading new documents into SharePoint libraries. How do you see the role of AI evolving in the SharePoint ecosystem over the next few years? Frane Borozan – MVP, Croatia: SharePoint, as the content management platform, doesn't have a future without the help of AI. Use cases are varied, ranging from extracting metadata from content to helping create new content. I believe that with the help of AI, the possibilities of SharePoint are unlimited. Noorez Khamis – MVP, Canada: Copilot is making SharePoint the go-to content management system by transforming how you discover, create, and interact with content. You can now ask for what you need, generate pages from your existing documents, and get personalized answers through AI-powered SharePoint Agents. With more intelligent automation, beautiful intranet design, and fewer clicks, SharePoint feels more like an intelligent assistant than a static site. Vlad Catrinescu – MVP, Canada: AI will continue transforming how we work, and SharePoint is no exception. Today, we’re already seeing AI help fill in metadata for document libraries. But imagine if AI could go further: automatically suggest and create the right columns, build content types based on the documents you upload, or even configure web parts through natural language prompts. SharePoint has always been a powerful platform, but it hasn’t always been the easiest to use. AI has the potential to make that power more accessible to every user, not just the experts. Drew Madelung – MVP, USA: I would like to see SharePoint Content AI evolve in a way that identifies opportunities for AI using existing content that automatically configures it without user intervention to improve discoverability. The ability to configure and work with these AI features should have a minimal learning curve and be integrated seamlessly, without requiring specialized technical skills. Mike Maadarani – MVP, Canada: As artificial intelligence continues to advance, it is anticipated that content management will become fully automated, reducing the need for administrators to establish and enforce rules. AI algorithms will be significantly more sophisticated, enhancing their ability to comprehend an organization's policies, the nature of the content being added, and the necessary actions required by the rules. Antonio Maio – MVP, Canada: I think auto-fill columns will have a significant impact on SharePoint Online. Metadata is a core element of good information management, but we know that users don’t want to fill in metadata. They’re busy and often move too quickly from task to task, leaving little time to provide a wealth of metadata elements. SharePoint’s auto-fill columns offer an easy way for us to automatically extract metadata, based on a prompt that’s supplied to the column. Joanne Klein – MVP, Canada: As a security and compliance professional, I observe the evolution of AI data governance, which aims to control the proliferation, access, and lifecycle of AI solutions across SharePoint within the enterprise. We need to elevate AI to be a core pillar within an organization's holistic data governance strategy. What advice would you give to SharePoint professionals who are just beginning to explore AI-powered capabilities? Frane Borozan – MVP, Croatia: If you're starting with AI in SharePoint, taxonomy tagging is a perfect first step. It shows how AI can reduce manual effort and bring structure to your content. Set up a managed metadata column linked to your term store and let AI handle tagging based on document content. It’s a simple way to improve search, consistency, and governance—without heavy customization. Start here, learn the basics, and expand as you go. Noorez Khamis – MVP, Canada: Start with the basics and learn how to prompt effectively while using the full Microsoft 365 Copilot capabilities across all the workplace tools you use every day, such as Teams, SharePoint, Word, Excel, PowerPoint, and Outlook. Take the time to revise your prompts and use templates that have been proven to work, saving you time, streamlining tasks, and boosting productivity. In SharePoint specifically, explore how to create pages, rewrite content, and use AI-powered SharePoint agents. Vlad Catrinescu – MVP, Canada: Get hands-on as early as possible. Theory is great, but real understanding comes from testing in your environment. Set up a lab, start small, and explore practical use cases where AI can help automate or enhance existing processes. With the Pay-As-You-Go model, there’s no upfront cost—you only pay for what you use. That said, I highly recommend setting a budget cap in Azure to avoid surprises. Drew Madelung – MVP, USA: SharePoint remains essentially unchanged in many ways, and it is still essential to understand the concepts of content types, columns, and permission hierarchies to implement advanced AI solutions against your organization's content effectively. Mike Maadarani – MVP, Canada: IT professionals must stay current with evolving technologies, particularly in the rapidly changing field of AI. With the emergence of AI agents, I recommend acquiring skills in creating, managing, and deploying agents within Microsoft 365 to enhance the integration and utilization of AI in their organizations. Antonio Maio – MVP, Canada: Be curious about AI - play with the AI technology that’s built into SharePoint; experiment with it to see what best benefits your organization to improve how you specifically manage information. Your experience will be different than everyone else’s, so try different things to figure out what works for you and your users. Joanne Klein – MVP, Canada: If your SharePoint setup is a mess, your AI will be too. Solid, well-defined structure and smart governance (site owner stewardship, explicit permissions, retention/deletion to clean up ROT, and data protection controls) are like laying concrete before building—skip it, and your AI’s standing on quicksand. Access the full 194-page e-book, SharePoint Content AI, Solutions and Advanced Administration, at the following link SharePoint Content AI, Solutions and Advanced AdministrationUpcoming June 2025 Microsoft 365 Champion Community Call
Join our next community call on June 24, 2025, where learn more about SharePoint agents, agent governance, and the Get Stuff Done campaign with Karuana Gatimu. Our community calls are now in the Teams webinar format, which means you must register to ensure you will be able to join the call when it starts. An on-demand recording will still be available on our Driving Adoption > Events pages, as well as on our Microsoft Community Learning YouTube channel. The calls will still start at 5 minutes past the hour for both sessions (at 8:05 AM and 5:05 PM PT), and it will still end at the top of the hour (9:00 AM and 6:00 PM PT, respectively). The registration links for both sessions are below. Once you register, you will receive an email confirmation and calendar invite with the event join link. https://aka.ms/M365ChampionCallAM https://aka.ms/M365ChampionCallPM Since our calls are open to everyone, you must be a member of the Microsoft 365 Champion Program in order to access the presentation materials - the access link is in the initial welcome email and the monthly newsletter emails sent the week before the community calls. If you have not yet joined our Champion community, sign up here to get access to the monthly newsletters, calendar invites, and program assets (e.g., the presentations).
Microsoft Purview: Comprehensive solutions for data governance, protection, compliance & management.
Microsoft Purview provides a unified data governance solution to help manage and govern your on-premises, multicloud, and software as a service (SaaS) data, Office Apps, Microsoft Office 365 services, Devices and Cloud Apps. Easily create a holistic, up-to-date map of your data landscape with automated data discovery, sensitive data classification, and end-to-end data lineage. Enable data consumers to access valuable, trustworthy data management.
