403 Access Denied authorization error caused by Sticky Bit in ADLS Gen2
Published Aug 02 2022 08:23 PM 1,441 Views
Microsoft

It's common that the user of ADLS Gen2 (Azure Data Lake Storage) need to control the permission of different users. One possible way is to use the ACL (Access Control List). For more details about this permission control mode, it can be found in this official document. This blog will mainly focus on one specific setting of the ACL called Sticky Bit and the authorization failure caused by this setting.

 

There are four parts in this blog:

 

What is Sticky Bit in ADLS Gen2?

According to Access control lists in Azure Data Lake Storage Gen2 | Microsoft Docs , ACL is a POSIX-like access control system. The description of sticky bit in this document is:

The sticky bit is a more advanced feature of a POSIX container. In the context of Data Lake Storage Gen2, it is unlikely that the sticky bit will be needed. In summary, if the sticky bit is enabled on a directory, a child item can only be deleted or renamed by the child item's owning user, the directory's owner, or the Superuser ($superuser).

The sticky bit isn't shown in the Azure portal.

 

For example, if there is an ADLS Gen2 storage account and a container called container, and there is folder path like folder/child-folder. User A was configured in ACL setting to have X permission on root and folder, and have WX permission on child-folder, then according to Access control lists in Azure Data Lake Storage Gen2 | Microsoft Docs , this user A should have enough permission to create/upload a new file test.txt into path container/folder/child-folder/test.txt. But if Sticky Bit is enabled on child-folder, then user A will get 403 authorization error.

 

How to identify whether the 403 Access Denied is caused by Sticky Bit?

When a user tried to upload new files into ADLS Gen2 Storage Account folder and he is using ACL as authorization method, either this user doesn't have enough permission including the parent folders, or this user has enough permission but Sticky Bit is enabled and this user isn't the owner of this folder, the 403 Access Denied error will be returned and the error message will be the same.

 

The only thing we can do from user side is to verify the ACL setting of the folder, and the parent folders and then compare it with the official document. If the permission configured on the user is already enough, then possibly the 403 error was caused by the Sticky Bit. We can follow next part to verify the Sticky Bit setting.

 

How to verify the Sticky Bit setting of a folder?

From user side, we can use lots of ways to check this setting, such as REST API call, PowerShell command, Azure CLI etc. Among these ways, the most recommended way is the Azure CLI because it doesn't need user to additionally install any software and the command is easy to understand.

  • Please login into your Azure Portal with your account. Please make sure that this account has Storage Blob Data Owner role assignment on the ADLS Gen2 storage account.
  • Click on the Cloud Shell button from Azure Portal.

Azure CLI in Azure PortalAzure CLI in Azure Portal

  • Use the following command to get the ACL and Sticky Bit setting: 
    az storage fs access show -p {folder name} -f {container name} --account-name {storage account name} --auth-mode login​
    For example, az storage fs access show -p folder -f container --account-name account --auth-mode login means to check the ACL and Sticky Bit setting of container/folder. If we want to check the root directory, which is the container level ACL and Sticky Bit setting, we can use az storage fs access show -p / -f container --account-name account --auth-mode login.
  • The result will be such as:

Azure CLI get ACL/Sticky Bit setting resultAzure CLI get ACL/Sticky Bit setting result

 

In the response Json body, we need to focus on permissions. It will normally contain 9 or sometimes 10 bits with an additional + signal. For more information about these letters, please refer to Access control lists in Azure Data Lake Storage Gen2 | Microsoft Docs .

 

In above example, it indicates that all permissions of all users are enabled, and also the Sticky Bit is enabled. For details about how to read this permission notation, please refer to File-system permissions - Wikipedia , Notation of traditional Unix permissions part. Here I’ll only focus on the Sticky Bit related configuration.

 

The ninth letter has 4 possible values: “-“, “x”, “t”, “T”. If the value of this letter is “t” or “T”, it means that the Sticky Bit is enabled. The “t” is “x“ with Sticky Bit enabled and “T” is “-“ with Sticky Bit enabled.

 

rwxrwxrwt can be explained into:

  • r,w,x permissions are enabled for Owner
  • r,w,x permissions are enabled for Owning group
  • r,w,x permissions are enabled for Other users and Sticky Bit is enabled

 

For you to understand more easily, provide another example here.

rwxr-xr-T can be explained to:

  • r,w,x permissions are enabled for Owner
  • r and x permissions are enabled for Owning group
  • Only r permission is enabled for Other users and Sticky Bit is enabled

 

And here I’ll also give a piece of information about how to translate above notation to short forms. According to Access control lists in Azure Data Lake Storage Gen2 | Microsoft Docs , short form permission will be calculated every group of 3 letters. (r as 4, w as 2 and x as 1) For example: rw-rwx--x will be equals to 4+2+0 4+2+1 0+0+1, 671. Based on this calculation rule, we’ll only need to add the fourth letter at the beginning. If sticky bit is enabled, we set it as 1. If sticky bit is disabled, we set it as 0.

So the above examples will be:

  • rwxrwxrwt => 1777

  • rwxr-xr-T => 1754

  • rw-rwx--x => 0671

 

How can user disable/enable this setting?

The operation from user side to disable/enable this setting is the same one. User only needs to set the permissions to the expected value.

 

The account used to modify this setting must have Storage Blob Data Owner role on the target ADLS Gen2 Storage Account.

 

Since there are so many possible ways to modify this setting, here is a piece of information about whether every SDK supports this setting at this moment:

Here I’ll give an example of Azure CLI. For other ways, please kindly check the SDK or command documents.

  • As previous part, please login into your Azure Portal with your account having Storage Blob Data Owner role assignment on the target ADLS Gen2 storage account.
  • Click on the Cloud Shell button from Azure Portal.

Azure CLI in Azure PortalAzure CLI in Azure Portal

  • Use the following command to set the ACL and Sticky Bit setting: 
    az storage fs access set --permissions {permission notation} -p {folder name} -f {container name} --account-name {storage account name} --auth-mode login​
    For example, az storage fs access set --permissions rwxrwxrwt -p folder -f container --account-name account --auth-mode login means to set the ACL of container/folder to rwxrwxrwx and to enable the Sticky Bit at same time. If we want to modify the setting of the root directory, which is the container level ACL and Sticky Bit setting, we can use az storage fs access set --permissions rwxrwxrwt -p / -f container --account-name account --auth-mode login.
  • The {permission notation} in above command can accept both long and short form notations. This means that the following command is also qualified. az storage fs access set --permissions 1776 -p folder -f container --account-name account --auth-mode login
  • The successful result of the command will be like following one:

Azure CLI set ACL/Sticky Bit setting resultAzure CLI set ACL/Sticky Bit setting result

 

 

Summary

As described in this official document, the Sticky Bit is an advanced feature not often needed in the ACL setting of ADLS Gen2 storage account. Normally we can easily use the mask feature to control the maximum permission of named users, the owning group and named groups. This almost does the same thing as Sticky Bit and it can be just simply configured in Azure Portal. The purpose of this document is to let users know what this feature is and how to check this setting when you accidentally configured this setting and got issue.

Co-Authors
Version history
Last update:
‎Aug 03 2022 01:15 AM
Updated by: