Meta’s next generation model, Llama 3.1 405B is now available on Azure AI
Microsoft, in collaboration with Meta, is launching Llama 3.1 405B, now available via Azure AI’s Models as a Service. Also introducing fine-tuned versions of Llama 3.1 8B and 70B. Leverage powerful AI for synthetic data generation and distillation. Access these models and more through Azure AI Studio and popular developer tools like prompt flow, OpenAI, LangChain, LiteLLM, and more. Streamline development and enhance efficiency with Azure AI.Accelerate enterprise GenAI application development with tracing in Azure AI Foundry
We are excited to announce the public preview of tracing in Azure AI Foundry, a powerful capability designed to enhance monitoring and debugging capabilities for your machine learning models and applications. Tracing allows you to gain deeper insights into the performance and behavior of your models, to help ensure they operate efficiently and effectively. Enable comprehensive monitoring and analysis of your application's execution Tracing allows you to trace application processes from input to output, review intermediate results, and measure execution times. Additionally, detailed logs for each function call in your workflow are accessible. You can inspect parameters, metrics, and outputs of each AI model used, for easier debugging and optimization of your application. The Azure AI Foundry SDK supports tracing to various endpoints including local viewers (Prompty trace viewer and Aspire dashboard), Azure AI Foundry, and Azure Monitor Application Insights. This flexibility helps you integrate tracing with any application, facilitating testing, evaluation, and deployment across different orchestrations and existing GenAI frameworks. Key Capabilities Basic debugging In situations where your application encounters an error, the trace functionality becomes extremely useful. It allows you to delve into the function causing the error, assess the frequency of exceptions, and troubleshoot using the provided exception message and stack trace. Detailed execution logs Tracing captures detailed traces of your model's execution, including data preprocessing, feature extraction, model inference, and post-processing steps. These details provide valuable insights into the inner workings of your models, helping you identify bottlenecks and optimize performance. For example, understanding the call flow of an application is crucial for complex AI systems where multiple components and services interact. By enabling tracing, developers can identify bottlenecks, understand dependencies, and optimize the flow for better performance. Performance metrics In addition to execution logs, tracing collects key performance metrics, such as latency and token utilization. These metrics allow you to monitor the efficiency of your models and make data-driven decisions to improve their performance. Building monitoring dashboards with the data collected from tracing can provide real-time visibility into the system's health. These dashboards can track key performance indicators (KPIs), provide alerts on anomalies, and help ensure that the AI services are running as expected. Error tracking Tracing helps you identify and troubleshoot errors in your models by capturing detailed error logs. Whether it's a data preprocessing issue or a model inference error, tracing provides the information you need to diagnose and fix problems quickly. This is particularly useful for capturing runtime exceptions, such as rate-limiting, which are critical for maintaining the reliability of your applications. Evaluations and user feedback You can attach evaluations metrics and user feedback to traces via online evaluation capabilities in Azure AI Foundry. Online evaluation allows you to incorporate real-world performance data and user insights into your monitoring process, to assess whether your models meet the desired quality standards. The Azure AI Foundry SDK simplifies the process of downstream evaluation, facilitating continuous improvement and validation of AI models against real-world data. Additionally, capturing user evaluations and interactions can provide insights into how users are engaging with the AI features, to inform user-centric improvements. Visualize Traces Azure AI Foundry provides robust tools for visualizing traces, both for local debugging and production-level monitoring. You can use these tools to gain a better understanding of your model's behavior and performance. The visualization capabilities include: Local debugging: Visualize traces during development to identify and resolve issues early, helping ensure that models are optimized before deployment. Visualize the data via Azure AI Foundry portal and Azure Monitor: In the post-deployment phase, developers often want to delve deeper into their applications' performance to optimize it further. For instance, you might want to monitor your GenAI application's performance, usage, and costs. In this scenario, the trace data for each request, the aggregated metrics, and user feedback become vital. Tracing seamlessly integrates with Azure Monitor, allowing you to visualize and analyze your model's performance metrics and logs using a customizable dashboard in Azure Monitor Application Insights. This integration provides a holistic view of your model's health and performance, enabling you to make informed decisions. Getting Started To start using tracing in Azure AI Foundry and Azure Monitor, follow these simple steps: Log Traces: Enable Tracing via Azure AI SDK for enabling tracing on Model inference API. Configure Logging: Set up the logging configuration to capture the desired level of detail for your model's execution. Enable Tracing in AI Studio: In your Azure AI Project, navigate to the Tracing and enable the feature for your models. Monitor and Analyze: Use Azure Monitor to visualize and analyze the collected logs and metrics, gaining insights into your model's performance. Find detailed guidance in our documentation: Overview of tracing capabilities in Azure AI Foundry Learn how to implement and use tracing with the Azure AI Foundry SDK Visualize your traces Build production-ready GenAI apps with Azure AI Foundry Want to learn about more ways to build and monitor enterprise-ready GenAI applications? Here are other exciting announcements from Microsoft Ignite to support your GenAIOps workflows: New ways to evaluate generative AI outputs for quality and safety New ways to monitor performance with Azure AI Foundry and Azure Monitor Whether you’re joining in person or online, we can’t wait to see you at Microsoft Ignite 2024. We’ll share the latest from Azure AI and go deeper into best practices for GenAIOps with these sessions: Microsoft Ignite Keynote Multi-agentic GenAIOps from prototype to production with dev tools Azure AI and the dev toolchain you need to infuse AI in all your apps3 Innovative Ways Developers Are Building with AI
From enhancing education with personalized learning experiences to redefining accessibility and maximizing existing data, developers are leveraging AI to address real-world challenges responsibly. Get inspired by these projects and learn how you can start your own AI journey today.The Future of AI: Maximize your fine-tuned model performance with the new Azure AI Evaluation SDK
In this article, we will explore how to effectively evaluate fine-tuned AI models using the new Azure AI Evaluation SDK. This comprehensive guide is the fourth part of our series on making large language model distillation easier. We delve into the importance of model evaluation, outline a systematic process for assessing the performance of a distilled student model against a baseline model, and demonstrate the use of advanced metrics provided by Azure's SDK. Join us as we navigate the intricacies of AI evaluation and provide insights for continuous model improvement and operational efficiency.Ensuring Platform Resiliency: The Next Step in AI Deployment
In this post, we dive into a crucial factor that determines the long-term success of AI solutions: platform resiliency. Learn how to build robust AI platforms that can withstand system failures, mitigate risks, and ensure continuous availability. From disaster recovery plans to leveraging Azure Front Door and API Management, we explore strategies that ensure your AI deployments remain stable, secure, and reliable, even in the face of unexpected challenges.
WebNN: Bringing AI Inference to the Browser
Unlock the Future of AI with WebNN: Bringing Machine Learning to Your Browser Discover how the groundbreaking Web Neural Network API (WebNN) is revolutionizing web development by enabling powerful machine learning computations directly in your browser. From real-time AI interactions to privacy-preserving data processing, WebNN opens up a world of possibilities for creating intelligent, responsive web applications. Dive into our comprehensive guide to understand the architecture, see code examples, and explore exciting use cases that showcase the true potential of WebNN. Whether you're a seasoned developer or just curious about the future of web-based AI, this article is your gateway to the cutting-edge of technology. Read on to find out more!
