Ultra Disk Storage is now generally available for the HPC and GPU VM sizes.
Ultra Disks
Ultra Disks are the highest performance tier of Azure managed disks for data-intensive workloads. They deliver high throughput and IOPS, and consistent low latency disk storage. Customers can dynamically change the performance of the disks without the need to restart your virtual machines (VM).
For HPC/AI workloads, the Ultra Disks can be used as a higher performance tier to Premium Disks for remote storage scenarios (NFS pools, parallel file systems, etc.)
Considerations for Ultra Disks vis-a-vis Premium Disks
Note that the availability of Ultra Disks by region/AZ/SKU is different (and restrictive today) from that of Premium Disks. Even though a VM size is supported, the support is different per region/zone.
- Today Ultra Disk and VMs have to be co-located, so Ultra Disks are only available to deploy for those H/N clusters that are co-located with Ultra Disk. Hence NOT ALL H/N clusters will be deployable with Ultra Disk.
HPC and GPU VM SKUs with Ultra Disks
In addition to the support for Premium Disks, the Ultra Disks can now also be attached to the following H* and N* VM sizes (depending on region/zone as outlined above):
- HPC: HBv2, HB, HC
- Currently HBv3 is not co-located with Ultra Disk. So as an example, Ultra Disk cannot be attached to an HBv3 VM in any region where HBv3 is live today.
- GPU: NDv4, NDv2, ND, NC_T4_v3, NCv3, NCv2, NVv4, NVv3
Performance
The table below demonstrated averaged bandwidth, IOPS and latency numbers on some of the HPC and GPU SKUs. These are obtained obtained using fio.
- The biggest improvement from Ultra Disks on the H*/N* VMs is to latencies that are now sub-millisecond (10x lower than Premium Disks).
- For most scenarios, the VM limits are the bottleneck with Ultra Disks and the disk limits becomes the bottleneck with Premium Disks. The suspected VM limits are anticipated to be increased in the coming weeks.
- VM limits are pre-assigned limits for managed disk performance to each VM size.
- HBv2 and HC obtain max write bandwidth with Ultra Disk. For many other SKUs, However the bandwidth obtained with an Ultra Disk is similar to what is obtained with Premium Disk.
- A single Ultra Disk can hit the VM Storage IO limit, hence removing the need for striping multiple disks and making disk management easier.
- The IOPS obtained with a single Ultra Disk is 2.5-4x of a single Premium Disk.
- Note the results obtained below are using both the Disks at the highest performance tiers
| VM SKU |
Premium (20000 GB) |
Ultra (1024 GB) |
| Spec: BW 900 MBps, 20K IOPS |
Spec: BW 2000 MBps, 80K IOPS (single VM) |
| BW (MBps) |
IOPS (K) |
Latency (ms) |
BW (MBps) |
IOPS (K) |
Latency (ms) |
| HC44 |
876 (R, W) |
20 |
2.495 |
700 (R), 1944 (W) |
64 |
0.215 |
| HB120_v2 |
876 (R, W) |
18 |
2.422 |
840 (R), 1946 (W) |
30 |
0.235 |
| NV48_v3 |
880 (R, W) |
20 |
1.51 |
1172 (R, W) |
80 |
0.339 |
| NV32_v4 |
703 (R, W) |
20 |
2.258 |
703 (R, W) |
49 |
0.242 |
| NDv2 |
876 (R, W) |
20 |
2.207 |
710 (R), 1173 (W) |
78 |
0.216 |
| NC64_T4_v3 |
704 (R, W) |
20 |
2.149 |
704 (R, W) |
49 |
0.254 |
Impact of Accelerated Networking
None. AccelNet does not improve Managed Disks performance. Exceptions maybe corner cases of the VMs doing heavy VM-VM network traffic over the Ethernet interface while doing VM-Managed Disk traffic at the same time.
Resources
