If you're hosting Linux servers in Azure, are you using secure methods to authenticate to them? Are you still using username and passwords? Or perhaps you are using public/private keys to SSH to them. If you're using private keys, how are you storing those keys?
These are all really good questions that security teams should be asking their infrastructure teams because cloud infrastructure most times are a completely different team from the traditional infrastructure teams managing servers on prem. If you're using Azure to hose these Linux servers, there are two really good and secure methods to authenticate to them.
In order to use Azure AD to authenticate, you need to use an SSH client that supports OpenSSH based certificates. Az CLI or Azure Cloud Shell are both native clients that will work. Your VM's in Azure must be a supported Linux distribution. When the VM is created, you have to use SSH public key for authentication type and make sure both System assigned managed identity and Login with Azure AD is checked under the management tab. Depending on your security policies and connectivity, this feature will work with both VM's with or without a public IP address as long as you have connectivity to it (i.e. VPN).
Make sure the username is recorded for this machine as you'll need it to authenticate using the key. You also need to make sure that the users who are accessing the VM have either Virtual Machine Administrator Login or Virtual Machine Administrator Login at the Subscription or Resource Group level. The cloud app you would use for conditional access is called "Azure Linux VM Sign-in." Another benefit is this feature will work with Azure Arc enabled servers as well.
The second method is to use Azure Bastion. I like to think of Azure Bastion as a jumpbox-as-a-service. This does not require any tools to be installed on your local device. Everything is done through a browser. Bastion needs to be deployed in the VNET that your VM's are in. Your VM's do NOT need a public IP address assigned. This is one of the key benefits of Azure Bastion and very important to some organizations that need to comply with regulations.
Once you have Azure Bastion deployed, the VM's are created the same way. However, you can escrow the private key in Azure Key Vault. This allows you to never have to store that private key locally or anywhere else. Azure Key Vault can be RBAC'd to limit access to secrets and keys.
So make sure you have a key vault created before you try to upload the key. One of the issues is that you cannot just drag and drop the key you download when you create your VM directly into Azure Key Vault. The formatting gets messed up. So you must upload your key via Powershell. To do this, install the Azure Az modules required by running this command:
Install-Module -Name Az -AllowClobber -Scope CurrentUser
Once that's installed, you need to authenticate to your Azure tenant by using Connect-AzAccount. Assuming you have the key stored locally on your machine in one of the text formats (.pem, .ppk, etc...), you will need to run this command in order to convert the key and upload it to Azure Key Vault:
If you do not get any errors, you should see the key pop into the Secrets tab with the -SecretName you used. Now you can go back to your VM and find either the "Connect" or "Bastion" tab. Enter the username that was used when the VM was created and select "SSH Private Key from Azure Key Vault" as the Authentication Type. Select the subscription, key vault, and secret name and hit connect. This will open up a browser ssh session to the machine. You can add addtional security by scoping a passphrase for the SSH key if you want to as well.
Currently, there is no cloud app for conditional access for only the Bastion service. In order to use conditional access, you would apply your rules to the "Microsoft Azure Management" cloud app.
So, both methods are very secure and have different use cases. If you are not using a secure method to log in and/or store your private keys, I highly recommend you take a look at both of these solutions.