Leverage Azure API Management to distribute API traffic to multiple backend services
Published Jan 28 2024 07:53 PM 3,320 Views
Microsoft
 

Load-balanced pool

Limitation

Logico_jp_1-1706500276422.png

 

Circuit breaker

 

Limitation

Please refer to “NOTE” in the document for more details.

Azure API Management backends | Microsoft Learn

 

Configuration

Preparation

Circuit breakers

 

{
  "type": "Microsoft.ApiManagement/service/backends",
  "apiVersion": "2023-05-01-preview",
  "name": "test1",
  "properties": {
    "description": "backend (test1)",
    "type": "Single",
    "protocol": "http",
    "url": "<URL1>/api/test",
    "circuitBreaker": {
      "rules": [
        {
          "failureCondition": {
            "count": "3",
            "errorReasons": [
              "Server errors"
            ],
            "interval": "PT5M",
            "statusCodeRanges": [
              {
                "min": "500",
                "max": "599"
              }
            ]
          },
          "name": "myBreakerRule",
          "tripDuration": "PT10M"
        }
      ]
    }
  }
}

 

Load-balanced pool

 

{
    "type": "Microsoft.ApiManagement/service/backends",
    "apiVersion": "2023-05-01-preview",
    "name": "backendPool",
    "properties": {
        "description": "Load balanced backends",
        "type": "Pool",
        "protocol": "http",
        "url": "http://localhost",
        "pool": {
            "services": [
                {
                    "id": "/backends/test1"
                },
                {
                    "id": "/backends/test2"
                }
            ]
        }
    }
}

 

Let’s try it.

 

Logico_jp_0-1706500394476.png

How about using these features with retry policy?

 

 

<!-- retryCount is specified in Named Values. -->
<!-- Load-balanced pool "backendPool4AOAI" is already configured -->
<policies>
    <inbound>
        <base />
        <set-backend-service backend-id="backendPool4AOAI" />
        <set-variable name="retryCount" value="@(int.Parse(" {{retryCount}}"))" />
        <set-variable name="maxRetryCount" value="@((int)context.Variables[" retryCount"] -1)" />
        <authentication-managed-identity resource="https://cognitiveservices.azure.com"
            output-token-variable-name="msi-access-token"ignore-error="false" />
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value>
        </set-header>
    </inbound>
    <backend>
        <retry condition="@(context.Response.StatusCode >= 300)" count="@((int)context.Variables[" maxRetryCount"])"
            interval="1" max-interval="10" delta="1" first-fast-retry="false">
            <!-- forward request and request body is stored for retry -->
            <forward-request buffer-request-body="true" />
        </retry>
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

 

Logico_jp_0-1706498833372.png

Conclusion

Co-Authors
Version history
Last update:
‎Jan 28 2024 07:55 PM
Updated by: