Blog Post

Apps on Azure Blog
5 MIN READ

Azure PowerShell Functions - connect SFTP and Storage Account

Mengyang's avatar
Mengyang
Icon for Microsoft rankMicrosoft
Aug 17, 2023

Background

Recently, I have touched on a case when the customer would like to interact with Azure PowerShell Function to work with Storage Account and SFTP server. There are some advantages why customers would like to apply for this solution:

  1. PowerShell allows IT admins to apply system-level integration with other components more easily.
  2. Azure Functions provide the convenience of URI/timer-triggered maintenance tasks.

Demand Analysis

We will first start with the demand analysis. The task will be divided into the following sections:

  1. Pull the required file contents from Azure Storage Account, and store them inside Azure Function App.
  2. Upload files to another VM via SFTP.
  3. To work with Azure resources, we also need to apply proper authentication to enhance security.

Sample Code

If you are looking for a quick sample code to start with, please navigate to the link below:

https://github.com/SuperChenSSS/GrabSA_SendSFTP

The credentials and sensitive data in the project have been erased, so that you can directly use them by changing the property name. For more details about the implementation, please check the following part.

Prepare for authentication

In order to work with different Azure resources, we need to grant access to the Azure Function level. The solution will use:

  1. Key Vault to retrieve the SFTP login credentials.
  2. Managed Identity to allow Azure Functions to work with Key Vault and Storage Account.
  3. Service Principal for local debugging during test phase.

You can expand to learn more details, or skip this part if you have already configured these.

Key Vault

Key Vault is the product for retrieving credentials in a safer way. For info on how to store the credentials and retrieve values, see below links:

Quickstart - Set & retrieve a secret from Key Vault using PowerShell

Azure Quickstart - Set and retrieve a secret from Key Vault using Azure portal

 

 

# Azure KeyVault
$kvName = "kvSFTP0"
# SFTP credentials
$sftpServer = Get-AzKeyVaultSecret -VaultName $kvName -Name "sftpServer" -AsPlainText
$sftpUsername = Get-AzKeyVaultSecret -VaultName $kvName -Name "sftpUsername" -AsPlainText
$sftpPassword = Get-AzKeyVaultSecret -VaultName $kvName -Name "sftpPassword" -AsPlainText

 

 

Managed Identity

Managed Identity allows Azure Function products to authenticate and work with other resources easily. Normally you would need to maintain many credentials in order to get the permission, but managed identity enables much less configuration.

Please refer to the below links for detailed steps:

Managed identities - Azure App Service

Tutorial: Use a managed identity to access Azure Key Vault - Windows - Microsoft Entra

No code-level implementation is needed for managed identity.

Service Principal

In most cases, before publishing to Azure Functions, you need to conduct code tests to make sure it works as expected. Managed Identity won’t work if it’s in local debugging. This time we will need to add Service Principal for such authentication.

Service Principal is another way to authenticate, and it will use some parameters to grant the permissions. Please follow the below steps:

  1. Create a Service Principal to access storage via Azure Active Directory (AAD). Access storage with Azure Active Directory
  2. Go to Key Vault → IAM Control → Assign Key Vault Contributor role under the newly created Service Principal, so that we have permissions from both sides.
  3. Invoke the Service Principal during code implementation.

 

 

# ServicePrincipal for Local Testing
$AppId = $ENV:APPID
$AppSecret = $ENV:APPSECRET
$TenantID= $ENV:TENANTID
$SecureSecret = $AppSecret | ConvertTo-SecureString -AsPlainText -Force
$Credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $AppId,$SecureSecret
Connect-AzAccount -ServicePrincipal -Credential $Credential -Tenant $TenantID 
Write-Host "Service Principal Connected"

 

 

Note that I have referenced the local env variables, and this can be configured in local.settings.json file with project folder. All the parameters can be found by searching for your Service Principal Name → Overview and get all credentials.

Before publishing to the Azure side, please comment on this section to allow function app to work with managed identity instead.

Once this is ready, we can move forward to connect to storage and publish via SFTP.

Connect to Storage Account

This part will mainly deal with downloading blob files from Storage Account into Azure Functions. We will use Azure Storage SDK to achieve this.

 

 

# Azure Storage info
$kvName = ""
$storageAccountName = ""
$containerName = ""
$blob_name = ""

# Interact with query parameters or the body of the request.
$cName = $Request.Query.cName
$bName = $Request.Query.bName
$saName = $Request.Query.saName
if(-not $cName -or -not $bName -or -not $saName) {
    $body = "One of the need params (saName, bName, cName) is missing, Will use default storage account, container and blob: ", $containerName, $blob_name, $storageAccountName , "\\nspecify cName and bName in query parameters to override"
    Write-Host $body
} else {
    $containerName = $cName
    $blob_name = $bName
    $storageAccountName = $saName
    $body = "Will download from following storage account, container and blob: ", $storageAccountName, $containerName, $blob_name
    Write-Host $body
}

# Connect and download from storage
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName
$blob = Get-AzStorageBlob -Context $ctx -Container $containerName -Blob $blob_name
$sourcePath = "../" + $blob.Name # Store Location in Azure Function
Get-AzStorageBlobContent -Context $ctx -Container $containerName -Blob $blob.Name -Destination $sourcePath -Force # Download

 

 

Note that for this task, I simply retrieve one blob file from Storage Account, and the SDK would allow you to achieve much more than this. Reference to Azure Storage SDK:

New-AzStorageContext (Az.Storage)

Get-AzStorageBlob (Az.Storage)

Get-AzStorageBlobContent (Az.Storage)

Publish file contents with SFTP

Secure File Transfer Protocol (SFTP) is a safer way to publish the site contents compared with FTP, and is commonly used in enterprise-level publishing events. To connect with SFTP, we can use default port 22, or custom port to enhance the security.

 

 

# Use Posh-SSH module to create and use SFTP session
Import-Module Posh-SSH

# SFTP credentials
$sftpServer = Get-AzKeyVaultSecret -VaultName $kvName -Name "sftpServer" -AsPlainText
$sftpUsername = Get-AzKeyVaultSecret -VaultName $kvName -Name "sftpUsername" -AsPlainText
$sftpPassword = Get-AzKeyVaultSecret -VaultName $kvName -Name "sftpPassword" -AsPlainText
$sftpPort = 22

# Establish SFTP session
$session = New-SFTPSession -ComputerName $sftpServer -Port $sftpPort -Credential (New-Object System.Management.Automation.PSCredential($sftpUsername, (ConvertTo-SecureString $sftpPassword -AsPlainText -Force))) -AcceptKey
# Judge if connection is successful
If (($session.Host -ne $sftpServer) -or !($session.Connected)){
    Write-Host $session.Connected + " " + $session.Host
    Write-Host "SFTP server Connectivity failed..!"
    exit 1
 }

# Upload each file to the SFTP server
$destinationPath = "/" #SFTP Server Location
$sourcePath = "../" + $blob.Name # Store Location in Azure Function
Set-SFTPItem -SessionId $session.SessionId -Path $sourcePath -Destination $destinationPath -Force # Upload to SFTP Server

# Close the SFTP session
Remove-SFTPSession -SessionId $session.SessionId

 

 

Instead of directly invoking sftp from SSH session, we will need to create one in the PowerShell function. Note that we will use Posh-SSH as the supported module to create and use the SFTP session. Below are references to the SDK:

GitHub - New-SFTPSession

GitHub - Set-SFTPItem

GitHub - Remove-SFTPSession

Result

As you can see, we have managed to retrieve the blob files and send them to VM via SFTP.

Conclusion

Before publishing the blog, I have done some research and found there are rarely any posts talking about this implementation, while it is a common scenario if the IT admin wants to use Azure Functions to perform some periodic maintenance tasks.

Hope the explanation could be helpful, and feel free to let me know if any further questions or queries, thank you.

Updated Aug 17, 2023
Version 1.0
azure functions
PowerShell
  • ItsBhatti's avatar
    ItsBhatti
    Brass Contributor
    Aug 17, 2023

    To connect an SFTP (Secure File Transfer Protocol) server and a Storage Account using Azure PowerShell Functions, you can follow these steps:

    1. Install Azure PowerShell Module (if not already installed): If you haven't already, you need to install the Azure PowerShell module. Open a PowerShell window and run the following command:

      powershellCopy code
      Install-Module -Name Az -AllowClobber -Scope CurrentUser

      If prompted to install the NuGet provider, choose "Yes".

    2. Connect to your Azure Account: Run the following command and follow the prompts to authenticate with your Azure account:

      powershellCopy code
      Connect-AzAccount

    3. Install WinSCP .NET Assembly: To interact with SFTP, you can use the WinSCP .NET assembly. Install it using the following command:

      powershellCopy code
      Install-Package -Name WinSCP -Source https://www.nuget.org/api/v2 -Scope CurrentUser

    4. Create the PowerShell Function: Create a new PowerShell script with your desired function. Below is an example of a function that connects to an SFTP server, downloads a file, and then uploads it to an Azure Storage Account blob:

      powershellCopy code
      # Import the WinSCP .NET assembly Add-Type -Path (Join-Path $env:USERPROFILE '\.nuget\packages\winscp\5.19.2\lib\netstandard2.0\WinSCPnet.dll') # Azure Storage Account information $storageAccountName = "yourstorageaccount" $storageContainerName = "yourcontainer" $storageBlobName = "uploadedfile.txt" $storageAccessKey = "yourstorageaccesskey" # SFTP server information $sftpHost = "sftp.example.com" $sftpUsername = "yourusername" $sftpPassword = "yourpassword" # Connect to SFTP server $sessionOptions = New-Object WinSCP.SessionOptions -Property @{ Protocol = [WinSCP.Protocol]::Sftp HostName = $sftpHost UserName = $sftpUsername Password = $sftpPassword } $session = New-Object WinSCP.Session $session.Open($sessionOptions) # Download a file from SFTP server $localFilePath = "C:\LocalPath\downloadedfile.txt" $remoteFilePath = "/path/to/remote/file.txt" $session.GetFiles($remoteFilePath, $localFilePath).Check() # Close SFTP session $session.Dispose() # Upload the downloaded file to Azure Storage Account $storageContext = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccessKey Set-AzStorageBlobContent -Context $storageContext -Container $storageContainerName -Blob $storageBlobName -File $localFilePath

      Replace placeholders (like yourstorageaccount, yourcontainer, etc.) with your actual details.

    5. Run the PowerShell Function: Save the PowerShell script and run it in your PowerShell window. The script will connect to the SFTP server, download the file, and then upload it to your Azure Storage Account.

    Remember to handle any error scenarios and enhance security by using managed identities or better authentication mechanisms if required.

  • Mengyang's avatar
    Mengyang
    Icon for Microsoft rankMicrosoft
    Aug 17, 2023

    Hi ItsBhatti 

     

    Many thanks for your reply, there are something worth mentioning:

    1. I have reviewed the solution you have mentioned in the discussion, and this solution is for onprem PowerShell usage, not for Azure Functions. Please note that Azure Functions don't allow popup window for authentication, but executing: Connect-AzAccount will bring up the popups, and you will end up getting the error message.

    2. We will have a PowerShell option when you are trying to create a new Azure Function App. Simply use this and you don't need to install other things. This may require if you are running PowerShell in a VM, but not on Azure Functions.

    3. The post doesn't cover how to setup the SFTP itself, we would assume you have already setup the SFTP server. Please have the research if you haven't prepared the SFTP server, and there are lots of articles talking about how to setup the SFTP server. The #2 step mentioned WinSCP is also an option to establish the SFTP server.

     

    Hope the info above could be helpful, thank you.