NVIDIA Dynamo is an open source, low-latency, modular inference framework for serving generative AI models in distributed environments. It scales inference workloads across large GPU fleets using intelligent resource scheduling and request routing, optimized memory management, and seamless data transfer. Dynamo supports open source inference engines including SGLang, TensorRT LLM, and vLLM, and simplifies distributed serving by disaggregating inference phases across GPUs, routing requests to avoid redundant computation, and extending GPU memory through data caching to cost-effective storage tiers. It builds on the NVIDIA Triton Inference Server and is intended for developers and ML platform teams running large-scale model serving. Access is via the open source GitHub repository and official documentation.
How to use NVIDIA Dynamo
1
Review the documentation
Read the official NVIDIA Dynamo documentation to understand architecture, deployment options, and supported backends.
2
Download from GitHub
Get the open source Dynamo project from the official ai-dynamo/dynamo GitHub repository.
3
Choose an inference engine
Select a supported open source inference engine such as SGLang, TensorRT LLM, or vLLM for your serving workload.
4
Deploy across GPUs
Configure distributed serving with disaggregated prefill and decode phases, request routing, and memory management across your GPU fleet.
NVIDIA Dynamo Features
Open source, low-latency, modular inference framework
Intelligent resource scheduling and request routing across GPU fleets
Optimized memory management and seamless data transfer
Supports SGLang, TensorRT LLM, and vLLM inference engines
Disaggregated prefill and decode phases across GPUs
GPU memory extension through data caching to cost-effective storage tiers
Builds on NVIDIA Triton Inference Server
NVIDIA Dynamo Use Cases
Serving generative AI models in distributed environments
Scaling inference workloads across large GPU fleets
Running mixture-of-experts reasoning models at scale
Deploying open source LLM inference with vLLM, SGLang, or TensorRT LLM
NVIDIA Dynamo FAQs
It is an open source, low-latency, modular inference framework for serving generative AI models in distributed environments.
It supports open source inference engines including SGLang, TensorRT LLM, and vLLM.
It disaggregates inference phases across GPUs, routes requests to avoid redundant computation, and extends GPU memory through data caching to cost-effective storage tiers.
The open source project is available from the official ai-dynamo/dynamo GitHub repository linked from the NVIDIA Dynamo page.
Official documentation is available at docs.nvidia.com/dynamo/latest/.
It builds on the NVIDIA Triton Inference Server, an open-source software that standardizes AI model deployment and execution.
NVIDIA Dynamo itself is open source and free to download from GitHub. NVIDIA AI Enterprise, which will include Dynamo for production inference in a future release, is a separate commercial offering with a 90-day free trial license; no specific price for it is stated on this page. Checked 2026-11-19.
NVIDIA Dynamo (Open Source)
0
Open source, low-latency, modular inference framework for serving generative AI models across distributed GPU fleets, available on GitHub.
Open source software on GitHub with end-to-end examples
Supports SGLang, TensorRT LLM, and vLLM backends
SLO Planner, KV-aware Router, NIXL, KV Block Manager, Grove, AI Perf