More performance and choice with new Azure HBv3 virtual machines for HPC

HPC-optimized virtual machines featuring AMD EPYC 7003-series processors

Today Azure advances the velocity at which we bring the latest technologies to our HPC customers and the compute performance we put at their fingertips. We are excited to announce Azure HBv3-series virtual machines (VMs) for high-performance computing (HPC) are generally available in the East US, South Central US, and West Europe Azure regions. HBv3 VMs will also be available in the West US3 and Southeast Asia regions soon.

HBv3 VMs are designed to deliver leadership-class performance, message passing interface (MPI) scalability, and cost efficiency for a variety of real-world HPC workloads such as CFD, finite element analysis, computational geoscience, rendering, weather simulation, financial risk analysis, and RTL simulation.

HBv3 VMs bring new capabilities to Azure’s fleet of HPC and AI-optimized VM offerings:

• Azure’s first VMs with AMD EPYC 7003 series processors.
• Up to 19 percent higher per core performance.
• Up to 2.6 times higher per VM performance for lightly-threaded workloads.
• Up to 1.9 times higher performance for large-scale MPI workloads.
• Multiple VM sizes so customers can deploy fit-for-purpose HPC compute.
• H-series’ highest performance local SSD configuration to date, with up to 7 GB/s of read performance, 3 GB/s of write performance, and 200,000 IOPs.
• Simpler NUMA topology to increase application compatibility.

Virtual machine sizes for a diverse set of HPC customer needs

HBv3 VMs feature up to 120 AMD EPYC™ 7003 series CPU cores, 448 GB of RAM, and no simultaneous multi-threading. HBv3-series VMs also provide up to 340 GB/sec of memory bandwidth, up to 32 MB of L3 cache per core, up to 7 GB/s of block device SSD performance, and clock frequencies up to 3.675 GHz. All HBv3-series VMs feature NVIDIA Mellanox HDR 200 Gb/s InfiniBand to enable 80,000 core MPI workloads.

‘r’ denotes support for remote direct memory access (RDMA). ‘s’ denotes support for Premium SSD disks

For the first time on an Azure H-series, HBv3 VMs offer Constrained Cores sizes. This enables customers to pick a lower number of cores to expose to the VM while keeping all other assets constant. Doing so increases how those assets are allocated on a per-core basis. In HPC, common scenarios for which this is useful include:

• Providing more memory bandwidth per CPU core for CFD workloads.

• Allocating more L3 cache per core for RTL simulation workloads.

• Driving higher CPU frequencies to fewer cores in license-bound scenarios.

• Giving more memory or local SSD to each core.

Generational leap in scalable HPC performance

Azure HBv2 VMs with EPYC 7002-series processors have already shown the ability to run MPI workloads like weather modeling and molecular dynamics at more than 80,000 cores, scale 12 times higher than demonstrated elsewhere on the public cloud, and even outperform one of the Top 10 supercomputers in the world.

HBv3 VMs extend Azure’s leadership in scalable HPC performance by offering significant performance improvements for MPI workloads. In testing of workloads scaled from 4,000 cores to more than 33,000 cores, HBv3 shows a 1.23 times to 1.89 times performance improvement over HBv2 series.

HBv3 VMs join HBv2 VMs in featuring HDR 200 Gb/s InfiniBand from our technology partners at NVIDIA Networking (formerly Mellanox Technologies). Just like HBv2, the InfiniBand fabric backing HBv3 VMs is a non-blocking fat-tree with a low-diameter design for consistent, ultra-low latencies with In-Network Computing acceleration engines. Customers can use standard Mellanox/OFED drivers just as they would on a bare-metal environment. HBv3 VMs support RDMA verbs and all MPI types and versions, such as HPC-X, OpenMPI, MVAPICH2, and Intel MPI. Customers can also leverage adaptive routing and hardware acceleration of MPI collectives to realize additional performance and scaling efficiency.

Across a single virtual machine scale set, customers can run a single MPI job on HBv3 Virtual Machines at up to 36,000 cores. For our largest customers, HBv2 Virtual Machines support up to 80,000 cores for single jobs.

Significant gains for single and lightly threaded HPC workloads

HBv3 VMs also offer large performance improvements for single-threaded and lightly-threaded HPC workloads. This is critical for customers needing to run large ensembles of smaller HPC jobs (such as a parameter sweep over a design space) and those using commercial software the expense of which comprises a majority of the total cost of the solution.

Thanks to 19 percent higher IPC from the “Zen3” core architecture, 2 times larger L3 caches, and clock frequencies as high as 3.675 GHz, HBv3 VMs with Constrained Cores offer up to a 2.6 times increase in performance over our existing HPC-optimized VMs at the same lightly threaded core count.

Empowering customers and partners with cutting edge HPC abilities

“Microsoft Azure has once again raised the bar for HPC hardware in the cloud,” said Mulyanto Poort, VP of HPC Engineering at Rescale. “By combining best-in-class low latency networking and the latest and extremely versatile CPU offering from AMD into a single virtualized package we are able to add a flagship HPC offering to our already expansive hardware catalog.”

“HBv3’s versatility promises to unlock improved performance for the many different HPC workloads our customers run on Rescale today, from massively scalable CFD simulations to the IO-intensive workloads seen in Weather modeling and Acoustics simulations,” said Gerhard Esterhuizen, SVP of Engineering at Rescale. “Additionally, the aggressive price point of HBv3 allows us to package different turn-key solutions that optimize total simulation economics in scenarios where software license costs are high.”

“We are excited to be able to offer HBv3 to our customers and look forward to seeing how our customers will leverage this great new technology.” said Ethan Rasa, Senior Director of Corp Dev and Partnerships at Rescale. 

“Drizti is very excited to be a launch partner for the HBv3 series on Microsoft Azure. The incredible performance of AMD EPYC Zen 3 Milan combined with InfiniBand HDR and superfast local NVMEs on HBv3 gives a significant performance boost to users of HPC. With the combined capabilities of Azure and HPCBOX, we can upgrade HPC from one generation to the next within minutes, right on launch day!” said Devarajan Subramanian, CEO at Drizti.

“The new HBv3 VMs from Azure, powered by 3rd Gen AMD EPYC processors will continue to provide customers access to high-performance computing capabilities, in the cloud,” said Kumaran Siva, corporate vice president, Cloud Business, AMD. “We’re excited to continue our work with Azure to push the performance boundaries of cloud computing and can’t wait to see what customers will do with these VMs.”

“Azure HBv3 VMs are unique for offering HPC performance in the public cloud and enabling customers to benefit from the massive throughput and acceleration NVIDIA Mellanox InfiniBand brings to advancing a wide range of HPC research,” said Gilad Shainer, senior vice president of networking at NVIDIA.

• Find out more about  high-performance computing in Azure.

• AMD Launch Hub.

• HBv3 overview with Mark Russinovich.

• TechCommunity blog.

• Azure HPC optimized OS images.

• Azure HPC virtual machines.