Auto-delete on idle: Service Bus (SB) Queues or Topic-Subscriptions are automatically deleted & the SB client application fails with error code 404 (Not Found) or MessagingEntityNotFoundException.
Scenario:
The applications using Service Bus (SB) Queues or Topic- Subscriptions are failing with error code 404 (Not Found) or MessagingEntityNotFoundException . On investigation, you find that SB Queues or Topic- Subscriptions are suddenly removed or automatically deleted after it has been working for a while.
You have SB queues or topics-subscriptions that were provisioned. It was working without problems until now. You encounter errors\exceptions that indicate that the entity “does not exist” or is “not found”. You investigate and find that the entity is missing. You check with your colleagues, your Azure administrator if they have deleted the SB queues or subscriptions and they reply negatively.
You start wondering…Why did the queues or subscriptions suddenly go missing?... Why are they suddenly removed or automatically deleted?
Cause:
One of the properties that can be set on SB Queues or Topic-Subscriptions is Auto-delete on idle. Autodelete on idle enables you to specify an idle interval after which a queue or topic subscription is automatically deleted.
SB Queue, Topic-Subscription can use Auto-delete on idle property to make itself Temporary i.e., which will be removed if they are not used for a specified period. You can read more about Temporary Entities here. You will find what is considered Idleness here for Queues, Topics-Subscriptions.
Verification:
Questions:
The Proactive Approach:
The Proactive Approach is to enable Diagnostic Settings at the SB Namespace Level, esp. the OperationalLogs. These logs can be viewed from Log Analytics using simple queries. You can look for Auto Delete events to check if the Auto-delete on idle has set in.
The Reactive Approach:
The SB users can always open a Support Ticket with Microsoft from Azure Portal. Kindly provide the below details while opening a case for such an issue.
Required Information:
No, there could be various other reasons why Queues or Topic-Subscriptions may be suddenly removed… Let me state them below from simple to complex scenarios (This is not an exhaustive list):
Reasons why Entity may suddenly go missing |
How to prevent it? |
The user may delete the entity in error and not inform the team, via Portal, CLI, PowerShell, or RESTAPIs. |
Be careful to whom you provide write\delete privilege. Follow the principle of least privilege. Only users with write privilege will be able to see/access the SAS keys and those with delete privilege can delete resources.
You can check in the ARM Activity Logs if a user has deleted the entity via Portal, PowerShell, or CLI.
Use ARM Resource Lock to prevent undesirable changes to SB Entities. |
Applications can make use of SB Resource Provider (RP) APIs to delete the Entities. ( SB Resource Provider APIs calls are not logged in Azure Logs like Activity, Operation, Diagnostic, etc. It is the responsibility of the applications making SB RP API calls to log the operations it is performing.)
Service Bus libraries & tools like ServiceBusExplorer use these SB RP APIs.
Refer: Manage using Azure Resource Manager-based libraries & Manage using Service Bus client libraries.
|
Instead of using SAS Keys in your application, consider using Role-based Authentication and Access. You can Disable local or shared access key authentication with Azure Service Bus. Refer: Managed identities for Azure resources with Service Bus
Have proper logging in your application, esp. when CRUD operations are performed. Also, Turn on OperationalLog in Diagnostics settings. |
Auto-delete on idle. |
Discussed in detail in this article. |
SAS Key\Token is Compromised |
Follow the principle of Least Privilege. Keep audit of SAS Key distribution. If SAS Key is compromised immediately recycle the Keys. |
Others |
Open a Support Ticket with Azure and provide all the relevant details. |
We do not collect application or server information that performs the delete operation. So, the support professionals will not be able to provide information regarding Application\Server that made the delete request.
The customers can configure VNets and firewalls to control the inflow of requests and monitor them. This has helped some customers identify the clients (machines) making these requests. VNetAndIPFilteringLogs and OperationalLogs in the Diagnostic Setting will come in handy.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.