# Source: https://developer.nvidia.com/dgx-cloud/benchmarking.md

1. [Home](https://developer.nvidia.com/)
2. [DGX Cloud](https://developer.nvidia.com/dgx-cloud)

DGX Cloud Benchmarking

# NVIDIA DGX Cloud Benchmarking

NVIDIA DGX™ Cloud Benchmarking gauges training and inference performance across AI workloads and platforms, accounting for chips, cloud platforms, and application configurations.

[Try Now](/dgx-cloud/benchmarking/join)[Documentation](https://github.com/NVIDIA/dgxc-benchmarking)

* * *

## How NVIDIA DGX Cloud Benchmarking Works 

NVIDIA DGX Cloud Benchmarking analyzes real-time metrics across GPU configurations and environments, with ready-to-use benchmarking templates and on-demand benchmarking for custom workloads. Baseline performance results for comparison are provided via interactive dashboards.

Specifications:   
  
 - Scaling analysis from 8 to 2,048 GPUs  
 - Precision comparisons: FP8 versus BF16  
 - Support for popular AI frameworks and models  
 - Performance data across NVIDIA NeMo™ framework versions

 ![A chart showing DGX Cloud Benchmarking results](https://developer.download.nvidia.com/images/dgx-cloud-services/diagram-cloud-benchmarking-service.jpg)

Example of improved metrics gained over time derived from NVIDIA DGX Cloud Benchmarking results.

### Introductory Blog

NVIDIA DGX Cloud introduces ready-to-use templates and recipes to benchmark AI platform performance.

[Read Blog](/blog/nvidia-dgx-cloud-introduces-ready-to-use-templates-to-benchmark-ai-platform-performance/)

### Benchmark Recipes

The Benchmarking Collection provides an easy path to reproduce the latest performance results for deep learning workloads.

[View Recipes](https://github.com/NVIDIA/dgxc-benchmarking)

### Explainer Blog

Discover how NVIDIA DGX Cloud Benchmarking accurately measures performance in real-world environments and identifies optimization opportunities in AI training and inference workloads.

[Read Blog](/blog/measure-and-improve-ai-workload-performance-with-nvidia-dgx-cloud-benchmarking)

* * *

## Get Started With NVIDIA DGX Cloud Benchmarking

 ![](https://developer.download.nvidia.com/icons/m48-monitor-performance.svg)
### Benchmark AI Workloads

Understand end-to-end platform performance and learn best practices for cluster sizing and workload optimization with NVIDIA DGX Cloud Benchmarking.

[Try Now](/dgx-cloud/benchmarking/join)

 ![](https://developer.download.nvidia.com/icons/m48-guideline-document.svg)
### Benchmarking Recipes

Deploy ready-to-use benchmarking templates across any cloud platform. Leverage NVIDIA’s performance baselines to compare expected performance across precisions and scales.

[View Recipes](https://github.com/NVIDIA/dgxc-benchmarking)

* * *

## NVIDIA DGX Cloud Benchmarking Starter Kits

See how well your own environment performs: log into NGC; select a model; download the container, benchmarking recipes, and dataset scripts; and launch to obtain throughput results.

### Benchmark Recipe for Meta Llama 3.1 70B

Understand end-to-end platform performance and learn best practices for cluster sizing and workload optimization with DGX Cloud Benchmarking.

- 

[Review DGX Cloud Benchmarking Artifacts](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/dgxc-benchmarking/collections/dgxc-benchmarking/artifacts)

- 

[Select DGX Cloud Benchmarking Entity for Meta Llama 3.1 70B](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/dgxc-benchmarking/resources/llama31-70b-dgxc-benchmarking-b)

- 

[Access DGX Cloud Benchmarking Service](/dgx-cloud/benchmarking/join)

### Benchmark Recipe for NVIDIA NeMo Megatron

This recipe contains information and scripts to produce training performance results of the NVIDIA NeMo Megatron workload.

- 

[Review DGX Cloud Benchmarking Artifacts](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/dgxc-benchmarking/collections/dgxc-benchmarking/artifacts)

- 

[Select DGX Cloud Benchmarking Entity for NVIDIA NeMo Megatron](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/dgxc-benchmarking/resources/nemo_megatron175b-dgxc-benchmarking-b)

- 

[Access DGX Cloud Benchmarking Service](/dgx-cloud/benchmarking/join)

### Benchmark Recipe for xAI Grok-1 314B 

This recipe contains information and scripts to produce performance results for the Grok1 314B training workload.

- 

[Review DGX Cloud Benchmarking Artifacts](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/dgxc-benchmarking/collections/dgxc-benchmarking/artifacts)

- 

[Select DGX Cloud Benchmarking Entity for Grok1 314B](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/dgxc-benchmarking/resources/grok1-314b-dgxc-benchmarking-b)

- 

[Access DGX Cloud Benchmarking Service](/dgx-cloud/benchmarking/join)

* * *

## NVIDIA DGX Cloud Benchmarking Learning Library 

Techblog 

### NVIDIA-Optimized Code for Popular LLMs

**NVIDIA AI Foundation Models and Endpoints**  
  
Learn tips to generate code, answer queries, and translate text on Llama, Kosmos-2, and SeamlessM4T with NVIDIA AI Foundation Models.

Tutorial 

### How to Deploy a NIM in 5 Minutes

**NVIDIA NIM**  
  
NVIDIA NIM™ is a set of easy-to-use inference microservices for accelerating the deployment of foundation models on any cloud or data center.

Model 

### Try NVIDIA NIM APIs

**NVIDIA Build**  
  
Explore leading open models built by the community, optimized and accelerated by NVIDIA’s enterprise-ready inference runtime.

Video 

### NVIDIA DGX Cloud Create

**NVIDIA DGX Cloud**   
  
NVIDIA DGX Cloud Create is a high-performance, fully managed AI training platform that provides optimized accelerated computing clusters on any leading cloud and access to NVIDIA experts.

Techblog 

### NVIDIA DGX Cloud Serverless Inference

**NVIDIA DGX Cloud**  
  
NVIDIA DGX Cloud Serverless Inference simplifies deploying AI workloads across multiple regions with seamless auto-scaling, load balancing, and event-driven execution.

* * *

## More Resources

 ![Decorative image representing forums](https://developer.download.nvidia.com/icons/m48-misc-question-faq.svg)
### NVIDIA DGX Cloud Benchmarking FAQ

 ![](https://developer.download.nvidia.com/images/isaac/m48-certification-ribbon-2-256px-blk.png)
### Get Training and Certification  

 ![](https://developer.download.nvidia.com/icons/m48-developer-1.svg)
### Join the NVIDIA Developer Program

## Ethical AI   

NVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.   
  
For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety &amp; Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

**Get started with DGX Cloud Benchmarking today**

[Try Now](/dgx-cloud/benchmarking/join)