Running A(P)I platforms at-scale is not a walk in the park – As traffic flows through the system, it needs handle the load and provide insights on where the inefficiencies are.
Finding the needle in a haystack
Azure API Management provides a broad set of observability capabilities across its managed and self-hosted gateway offerings, although availability varies by gateway type:
- Azure Application Insights integration leveraging requests, traces from policies, custom metrics from policies & dependency tracking to integrate with your apps APM
- Request tracing with API Inspector
- Built-in analytics for (business) reporting (docs)
- Azure Monitor logs & metrics for our managed gateway or OpenTelemetry metrics for our self-hosted gateway
- Logging to Azure Event Hubs in your desired format through policies
These capabilities are valuable, but the teams operating API platforms do not always define the APIs or author their policies. As a result, operators may lack visibility into the downstream work performed during each request:
- A single inbound request does not always map to a single backend request; policies can cause it to fan out into multiple downstream calls.
- Rate limiting happens, so calls downstream can retry and infuse latency
All of these can infuse latency to the end-to-end experience for their customers and can only be diagnosed with detailed insights – They are looking for the needle in a haystack.
In recent months, support cases have shown that customers can struggle to identify the source of latency when relying on Application Insights telemetry alone.
Here are some examples showing high incoming latency but it’s difficult to understand the cause.
Example #1:
Example #2:
Example #3:
Introducing external dependency calls in Application Insights for policies
We want to empower our customers by shifting our internal insights left to help customers be more efficient/self-diagnose API platforms at scale.
I’m excited to share the first release of external dependency telemetry in Application Insights for Azure API Management policies. It covers the following policies:
- authentication-managed-identity
- authentication-token
- azure-openai-semantic-cache-lookup
- cosmosdb-request-handler
- forward-request
- get-authorization-context
- http-data-source
- invoke-dapr-binding
- llm-content-safety
- llm-semantic-cache-lookup
- send-request
- send-one-way-request
- send-service-bus-message
- sql-data-source
- validate-jwt
This telemetry helps customers see where request time is spent and can reduce the need to open a support ticket. The examples below show how it explains the scenarios introduced earlier:
Example #1 was retrying calls to the backend with a wait in between:
Example #2 performed JWT validation, which required retrieving OpenID Connect metadata. It then made an initial slow backend call before the backend call visible to the customer.
Example #3 combined three downstream operations in one request: validating a JWT, sending a message to Azure Service Bus, and then forwarding the request to the backend.
What’s next?
Improving your application landscape telemetry in Application Insights is just the beginning!
We’re continuing to expand the diagnostic information available to customers in two areas:
- Enhance Azure Monitor diagnostic logs with additional per-request details and outbound dependency information.
- Add dependency telemetry for more policies and scenarios.
Together, these improvements will give platform builders deeper insight into their A(P)I platforms and make that information easier to integrate with existing monitoring solutions.
We’re excited to deliver this richer Application Insights telemetry, get started by reading our Azure Application Insights integration guidance. Let us know in the comments how you use it and which scenarios you would like us to cover next.
Thanks for reading,
Tom
Microsoft