How to Connect Remotely to A Virtual Machine in Microsoft Azure

Microsoft

Jun 03, 2025

At the heart of this post is Kairos IMS, an innovative Impact Management System designed to empower human-serving nonprofits and social impact organizations. Co-developed by the Urban League of Broward County and our trusted technology partner, Impactful, Kairos IMS reduces administrative burdens, enhances holistic care, and enables organizations to leverage data for increased agility and seamless service delivery. In this blog series, we’ll take a closer look at the powerful technologies that fuel Kairos IMS, from Azure services to security frameworks, offering insight into how modern infrastructure supports mission-driven impact. Click here to learn more.

Imagine it's midafternoon on a Friday. Your team lead needs you to spin up a windows virtual machine for testing purposes. You manage to spin up the virtual machine, however, Houston, there's a problem. You need to connect remotely since you are currently out of town. Whether you're debugging a deployment or testing a new app environment, remote access to virtual machines is a must-have skill. Microsoft Azure, a leading cloud services provider, offers a robust platform to deploy, manage, and connect to VMs. This guide presents a step-by-step walkthrough on how to connect to a VM in Azure using standard tools like Remote Desktop Protocol (RDP) and Secure Shell (SSH).

Prerequisites

Before proceeding, ensure the following requirements are met:

✅ An active Microsoft Azure subscription

✅ A pre-provisioned VM in Azure

✅ Admin credentials (username/password or SSH key)

✅ A stable internet connection

✅ Access to a terminal or RDP client

What Are Virtual Machines?

Virtual machines (VMs) are software-based computers that run inside physical computers. They function like physical computers, with their own CPU, memory, storage, and network connectivity, but exist only as code within a physical host machine. Azure is one of the largest cloud providers worldwide with host of services, boasting security at scale for your computing needs. In fact, Azure has many different Virtual Machine options from you to choose from.

VMs are important in cloud computing for several reasons:

Security: VMs are separated from the main computer, so any issues inside a VM won't affect the main system.
Efficiency: Multiple VMs can run on one physical computer, saving space and reducing costs.
Flexibility: VMs can run different programs and operating systems at the same time on the same physical computer.
Scalability: VMs can be quickly added or removed based on need, making it easy to adjust resources.
Cost Savings: Using VMs means fewer physical computers are needed, which lowers expenses.

These benefits make VMs a key part of cloud computing, helping to create efficient, secure, and adaptable computing environments.

Step 1: Access the Azure Portal

Launch your preferred web browser.
Navigate to https://portal.azure.com.
Authenticate using your Microsoft Azure credentials.

Note: The Azure Portal provides a centralized interface to manage your Azure resources, including networking, storage, and compute services.

Step 2: Locate the Virtual Machine Resource

Identify the Subscription under which the VM is deployed.
Select the appropriate Resource Group that contains the VM.
From the list of resources in the group, locate and click on the Virtual Machine you intend to access.
On the VM's overview page, review critical information such as its name, region, and assigned IP address.

Step 3: Verify VM Operational Status

Within the VM overview page, confirm that the Status is set to Running.
If the VM is stopped, select the Start option to initiate the instance.

Step 4: Establish a Remote Connection

Remote Desktop Protocol (RDP) is a proprietary protocol developed by Microsoft that allows users to connect to another computer over a network connection. RDP provides a graphical interface to the user for remote interaction with the desktop of a remote system. It operates over TCP port 3389 and ensures encryption between the client and server during communication. RDP also supports features such as:

Session redirection (printers, drives, and audio)
Secure transmission using encryption and authentication protocols
Efficient bandwidth usage by compressing data

Understanding these capabilities is crucial for effectively managing and troubleshooting RDP connections, especially in enterprise environments.

For Windows-based Virtual Machines (via RDP):

Login to Azure Portal: https://portal.azure.com.
Go to your Virtual Machine's overview page.
Select Connect, then choose RDP.
Ensure that the public IP address and port 3389 are accessible. If necessary, configure your Network Security Group (NSG) to allow inbound RDP traffic.
Download the provided RDP file by clicking Download RDP File.
Open the downloaded .rdp file using the Remote Desktop Client on your local machine.
When prompted, enter the username and password you created when you deployed the Windows VM.

For Linux-based Virtual Machines (via SSH):

From the VM overview, click Connect and navigate to the SSH tab.
Copy the autogenerated SSH command, which should look like:
1. ssh -i ~/Directory saved/ssh/PRIVATE -KEY/VM-ADMIN-USERNAME/IP-ADDRESS
2. Ex. ssh -i ~/ssh/id_rsa.pem/azureuser@PublicIPAddress
Open a terminal (macOS/Linux) or PowerShell/Command Prompt (Windows).
Paste and execute the SSH command.
If applicable, provide the passphrase for your private key or the VM password.

5. Troubleshooting Tips

Can’t connect via RDP/SSH?
- Double-check VM status (Running)
- Ensure correct IP and credentials
- Review NSG rules for open ports (3389 for RDP, 22 for SSH)
- Restart the VM if needed
Dynamic IP changes?
- Consider reserving a static public IP address to avoid connection loss after reboots.
SSH Key issues?

Verify your key pair matches what was added to the VM
Check file permissions (chmod 600 for private key on Linux/macOS)

Cleaning Up Resources

Once you're done working with your virtual machine, it’s important to clean up your resources to avoid unnecessary charges. Azure services are billed based on usage—even if you're not actively connected to the VM, you could still be charged for the compute, storage, and networking resources it consumes. Here’s how to properly shut things down:

Option 1: Stop the VM (Preserves Configuration)

Navigate to your VM’s overview page in the Azure Portal.
Click the Stop button at the top.
This shuts down the VM but keeps all associated resources (disks, IP addresses, etc.).
Use this option if you plan to return to the VM later.

Option 2: Delete the VM (Frees All Resources)

From the VM overview page, click Delete at the top.
Confirm the deletion when prompted.
This removes the VM and associated compute charges but may leave behind other resources (e.g., disks, NICs, public IPs).
To fully clean up:
- Go to the Resource Group where your VM was deployed.
- Review and delete any remaining resources you no longer need.

Pro Tip: Always double-check which resources you're deleting—some may be shared across projects.

6. Best Practices for Secure VM Access

✅ Shut down unused VMs to reduce costs
✅ Use strong, unique credentials and SSH keys
✅ Avoid hardcoding secrets into scripts
✅ Enable Azure Bastion for browser-based secure access
✅ Implement Role-Based Access Control (RBAC) to limit user permissions
✅ Regularly rotate credentials and audit access logs

Conclusion

Connecting to a virtual machine in Azure is a foundational task for anyone working in cloud environments. In this guide, we walked through the essential steps—from accessing the Azure Portal and locating your VM, to establishing a secure connection using RDP for Windows or SSH for Linux. Along the way, we highlighted important best practices to ensure your access is both efficient and secure. Now that you’re comfortable connecting to a VM, you're ready to take the next step: creating and configuring your own virtual machines from scratch.

What’s Next?

In the next guide, we’ll explore provisioning new VMs in Azure, including choosing appropriate VM sizes, configuring network interfaces, and deploying software packages. Stay tuned as we dive deeper into cloud infrastructure management with Azure.