Blog Post

Azure High Performance Computing (HPC) Blog
4 MIN READ

A quick start guide to benchmarking LLM models in Azure: NVIDIA NeMo Megatron - Results

HugoAffaticati's avatar
HugoAffaticati
Icon for Microsoft rankMicrosoft
Oct 24, 2022
By Hugo Affaticati (Technical Program Manager) and Jon Shelley (Principal TPM Manager)   Useful resources:  NeMo Megatron from NVIDIA: NVIDIA NeMo Megatron Container from NVIDIA: NVIDIA NGC ...
Updated Oct 24, 2022
Version 2.0
benchmarking
hpc
jpietsch's avatar
jpietsch
Copper Contributor
Dec 17, 2022

Very interesting post.

 

Any chance you'd be willing to say for each model size, the number of layers, hidden dimensions, attention heads and global batch size?