Mount Blob storage on Linux VM using Managed Identities or Service Principal with Blobfuse

[Sindhu_Hegde]

Scenario:

You want to mount the Azure Blob storage container on Linux VM and access the data using either Managed Identities or Service Principal.

Prerequisites:

Azure storage account

Linux VM

Action:

To mount the Azure Blob storage container as a filesystem on Linux VM, you can make use of Blobfuse which allows you to access the existing data in your storage account through the Linux filesystem.

Mounting of storage account using the Storage account key has been explained in our article:

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux

Below are the steps to mount the storage account either using Managed Service Identity or using Service Principal.

Step 1:

Configure the Linux software repository for Microsoft products using the below command:

For Ubuntu:

wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb

sudo dpkg -i packages-microsoft-prod.deb

sudo apt-get update

For RHEL:

sudo rpm -Uvh https://packages.microsoft.com/config/rhel/7/packages-microsoft-prod.rpm

Note: Change the URL accordingly based on the Ubuntu version and RHEL Distribution that you’re using.

Step 2:

Install the blobfuse in your Linux VM.

For Ubuntu:

sudo apt-get install blobfuse

For RHEL:

sudo yum install blobfuse

Step 3:

Blobfuse requires a temporary path in the file system to buffer and cache any open files.

You can make use of SSD disks available on your VMs for blobfuse. You can also make use of ramdisk and create a directory for blobfuse.

To use SSD as a temporary path, below is the command:

sudo mkdir /mnt/resource/blobfusetmp -p

sudo chown <youruser> /mnt/resource/blobfusetmp

Or to use ramdisk for the temporary path, below is the command:

sudo mkdir /mnt/ramdisk

sudo mount -t tmpfs -o size=16g tmpfs /mnt/ramdisk

sudo mkdir /mnt/ramdisk/blobfusetmp

sudo chown <youruser> /mnt/ramdisk/blobfusetmp

Step 4:

Blobfuse requires the authentication methods and credentials to be configured either in a configuration file or as an environment variable.

To create a configuration file and to restrict the access to the file so that no other users can read it, use the below commands:

touch ~/fuse_connection.cfg

chmod 600 fuse_connection.cfg

To mount the storage to the VM, you can make use of either System Assigned Managed Identity or User-assigned managed Identity or Service Principal.

1. Using System Assigned Managed Identity
   1. To use this configuration, please enable ‘System-assigned’ managed identity on the Linux VM that you’re using as shown below:
   2. Ensure that the Object ID or the system managed identity is given sufficient RBAC role at the storage account level.

Note: Please make sure that you give minimum of ‘Reader’ and ‘Storage Blob Data Reader’ role to the managed identity at the storage account level.

You can assign these roles here: Storage account -> Access Control (IAM) -> Add role assignment and selecting Virtual Machine in ‘Assign access to’ option as shown below:

2. Using User-Assigned Managed Identity

i. If you’re using User assigned managed identity, please add the identity in ‘User assigned’ configuration of your Linux VM as shown below:

ii. Ensure that the managed identity is given necessary RBAC roles at the storage account level as shown below:

For both scenarios, update the configuration file that was created earlier with the storage account credentials and mention authType as ‘MSI’ as shown below:

accountName <storage account name>

authType MSI

containerName <container name>

3. Using Service Principal

i. Ensure that SPN is given sufficient RBAC roles at the storage account level.

ii. Update the configuration file with Storage account details and Service Principal details. Also, the authType for Service Principal authentication would be SPN as shown below:

accountName <storage account name>

authType SPN

servicePrincipalClientId <Client ID or Application ID of the Service Principal>

servicePrincipalTenantId <Tenant ID of the Service Principal>

containerName <container name>

iii. The client secret for your application or the Service Principal must be saved as an Environment Variable and should not be mentioned in the configuration file. It will be saved as AZURE_STORAGE_SPN_CLIENT_SECRET. Please save it in /etc/environment in the below format:

AZURE_STORAGE_SPN_CLIENT_SECRET="your client secret"

Step 5:

Create an empty directory for mounting using the below command:

mkdir ~/mycontainer

Step 6:

To mount the blob storage using blobfuse, run the below command which will mount the specified container in the configuration file onto the empty directory that we created:

sudo blobfuse ~/mycontainer --tmp-path=/mnt/resource/blobfusetmp  --config-file=/path/to/fuse_connection.cfg -o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120

Note: To allow access to all users, please use the switch -o allow_other while mounting.

Once the container is mounted, you can access the blobs using the regular file system APIs in your Linux VM.

Hope that helps!

[Madhuri910]

Hello @Sindhu_Hegde ,

I am trying to use MSI with System assigned Identity to mount blobfuse.
I granted "Storage Blob Data Contributor" role at the storage account level to the VM.

Blobfuse seems to mount the container right, however when I try to create a file in the container using "touch a.txt", it gives me an error:

cat "hello" > a.txt
-bash: a.txt: No such file or directory

/var/log/messages:
Jun 30 07:38:57 vm-biv3test blobfuse[10091]: Failure to open cache file /.a.txt.swp in azs_open. errno = 2#012.

Blobfuse version: blobfuse 1.3.6

is there any additional role to be given to the VM to be able to write?.

[Sindhu_Hegde]

Hello @Madhuri910 ,

Could you please check the permissions that you have on the mounted container? Have you mounted the container using root permissions or your permission?

Also, can you please let me know if you have used the switch -o allow_other?

What happens when you use the above switch?

[litmusblox]

@Sindhu_Hegde 

Hi!

 Have followed the instructions above to connect a blob container to a vm. Have used Managed Identify on the vm. Want to connect to a blob container. I get the following error. 

 Have assigned contributor role to the vm. Here is the role assignment in the storage resource.

What am I missing? Any help will be appreciated!

Thanks!

Sonal Dedhia

[Sindhu_Hegde]

@litmusblox ,

I suggest you to have a look at the config file and confirm if you have provided the values correctly. Also, suggest you to also provide a data plane role for the user assigned managed identity to access the data.

[skyboy]

Does this also work for Microsoft App services?

I have developed a dash app which depends on data. This data is stored as .csv file in the blob storage in azure.

Isnt there a more easy way to directly access that data from the web app?