Blog Post

Azure High Performance Computing (HPC) Blog
5 MIN READ

Running DeepSeek-R1 on a single NDv5 MI300X VM

jesselopez's avatar
jesselopez
Icon for Microsoft rankMicrosoft
Feb 01, 2025
Contributors: Davide Vanzo, Yuval Mazor, Jesse Lopez   DeepSeek-R1 is an open-weights reasoning model built on DeepSeek-V3, designed for conversational AI, coding, and complex problem-solving. It...
Updated Apr 15, 2025
Version 11.0
ai infrastructure
hpc
virtual machines
caseygu's avatar
caseygu
Copper Contributor
Mar 07, 2025

Super helpful, good stuff!

Commenting with a couple of issues I encountered that may help others:

  1. The driver installation script tries to install azcopy from a domain that no longer exists.
    In in `azhpc-images/common/install_azcopy.sh`, change  `AZCOPY_DOWNLOAD_URL="https://azcopyvnext.azureedge.net/releases/release-${azcopy_release}/${TARBALL}"` to `AZCOPY_DOWNLOAD_URL="https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/releases/release-${azcopy_release}/${TARBALL}"` instead. 
  2. I followed the steps to create and save a custom VMI to my gallery, but any VM I created using the image had a lot of issues (different ones every time). Was never able to get SGLang running using a VM bootstrapped with the custom VMI.
    I went back to provisioning the VM manually which resolved all related issues. 
  3. Encountered oom issues when using docker image `rocm/sglang-staging:20250212` which couldn't be resolved by limiting mem block sizes; resolved by using newer `rocm/sglang-staging:20250303` instead. 
  • jesselopez's avatar
    jesselopez
    Icon for Microsoft rankMicrosoft
    Mar 10, 2025

    Hi!  Thanks for the feedback.  A public image has been published and the blog has been updated to reflect that along with an example of using cloud-init to automate some of the additional steps recommended to ease onboarding.  The article now also recommends an updated container image.