We launched public preview of managed endpoints in May 2021 and continued to release new features based on customer feedback. Today we are thrilled to announce the General Availability along with new capabilities in the areas of MLOps and security! We are ready for your production workloads!
To recap, Managed online endpoints handle serving, scaling, securing & monitoring of your machine learning (ML) models, freeing you from the overhead of setting up and managing the underlying infrastructure. In this blog, we will review the feature benefits, see what our users have to say, and learn how you can get started today.
1. Safely rollout new version of a model with mirror traffic support (public preview)
With our initial release we supported native blue-green deployments by providing a way to shift traffic gradually to a new deployment. Now we are releasing mirror-traffic support with which you can copy (or 'mirror') a percentage of the live traffic to a new deployment. Mirroring traffic doesn't change results returned to clients. Requests still flow 100% to the production deployment(s). The mirrored percentage of the traffic is copied and submitted to the new deployment so you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients. For example, to check if latency is within acceptable bounds and that there are no HTTP errors. Learn about the concept here and try a hands-on example here
2. Secure the communications of the endpoint using network isolation support (public preview)
When deploying a machine learning model to a managed online endpoint, you can secure communication of the online endpoint by using private endpoints. By using our declarative API's you can secure the network communications for both ingress and egress of your endpoint and deployment. Learn more about it and try hands-on experience here.
3. We are ready for your high scale production workloads
Have high scale production workloads? Checkout the below demo on how to scale easily using our platform. The demo showcases scaling to 100k request per second in 7 mins.
“We make it our mission to try new ideas and go beyond to differentiate AXA UK from other insurers. We see managed endpoints in Azure Machine Learning as a key enabler for our digital ambition.” - Nic Bourven: Chief Information Officer, AXA Insurance UK
Read the full case study here
“At Trapeze, we have been able to predict travel time on bus routes in large transit systems, improving the customer experience for bus riders with the help of Azure Machine Learning. We love the turn key solution Managed Online Endpoints offers for highly scalable ML model deployments along with MLOps capabilities like controlled out, monitoring and MLOps friendly interfaces. This has simplified our AI deployment, reduced technical complexity and optimized cost.” - Farrokh Mansouri| Lead, Data Science | Trapeze Group Americas
“We’re already using Azure Machine Learning to make predictions on the packages,” says Eric Brosch. “We look forward to using managed endpoints to deploy our model and inference at scale as it will decrease the time taken to manage infrastructure, allowing us to focus on the business problem.” - Eric Brosch: Data Scientist Principal, FedEx Services
Managed online endpoints is Generally Available now. We are ready for your production workloads!
Take managed online endpoints for a spin with this end to end tutorial. You can test drive the safe rollout experience using mirror traffic and the network isolation support. You can also brush up on the concepts.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.