✳ainav.
NVIDIA Dynamo screenshot
NVIDIA Dynamo website preview

PRODUCT OVERVIEW

What is NVIDIA Dynamo?

NVIDIA Dynamo is an open source, low-latency, modular inference framework for serving generative AI models in distributed environments. It scales inference workloads across large GPU fleets using intelligent resource scheduling and request routing, optimized memory management, and seamless data transfer. Dynamo supports open source inference engines including SGLang, TensorRT LLM, and vLLM, and simplifies distributed serving by disaggregating inference phases across GPUs, routing requests to avoid redundant computation, and extending GPU memory through data caching to cost-effective storage tiers. It builds on the NVIDIA Triton Inference Server and is intended for developers and ML platform teams running large-scale model serving. Access is via the open source GitHub repository and official documentation.

How to use NVIDIA Dynamo

  1. 1

    Review the documentation

    Read the official NVIDIA Dynamo documentation to understand architecture, deployment options, and supported backends.

  2. 2

    Download from GitHub

    Get the open source Dynamo project from the official ai-dynamo/dynamo GitHub repository.

  3. 3

    Choose an inference engine

    Select a supported open source inference engine such as SGLang, TensorRT LLM, or vLLM for your serving workload.

  4. 4

    Deploy across GPUs

    Configure distributed serving with disaggregated prefill and decode phases, request routing, and memory management across your GPU fleet.

NVIDIA Dynamo Features

Open source, low-latency, modular inference framework

Intelligent resource scheduling and request routing across GPU fleets

Optimized memory management and seamless data transfer

Supports SGLang, TensorRT LLM, and vLLM inference engines

Disaggregated prefill and decode phases across GPUs

GPU memory extension through data caching to cost-effective storage tiers

Builds on NVIDIA Triton Inference Server

NVIDIA Dynamo Use Cases

  • Serving generative AI models in distributed environments

  • Scaling inference workloads across large GPU fleets

  • Running mixture-of-experts reasoning models at scale

  • Deploying open source LLM inference with vLLM, SGLang, or TensorRT LLM

NVIDIA Dynamo FAQs

It is an open source, low-latency, modular inference framework for serving generative AI models in distributed environments.

It supports open source inference engines including SGLang, TensorRT LLM, and vLLM.

It disaggregates inference phases across GPUs, routes requests to avoid redundant computation, and extends GPU memory through data caching to cost-effective storage tiers.

The open source project is available from the official ai-dynamo/dynamo GitHub repository linked from the NVIDIA Dynamo page.

Official documentation is available at docs.nvidia.com/dynamo/latest/.

It builds on the NVIDIA Triton Inference Server, an open-source software that standardizes AI model deployment and execution.

NVIDIA Dynamo Official Information

Company
NVIDIA
Website
https://developer.nvidia.com/dynamo
Documentation
https://docs.nvidia.com/dynamo/latest/
Contact
https://developer.nvidia.com/contact

NVIDIA Dynamo Pricing

Free

NVIDIA Dynamo itself is open source and free to download from GitHub. NVIDIA AI Enterprise, which will include Dynamo for production inference in a future release, is a separate commercial offering with a 90-day free trial license; no specific price for it is stated on this page. Checked 2026-11-19.

NVIDIA Dynamo (Open Source)

0

Open source, low-latency, modular inference framework for serving generative AI models across distributed GPU fleets, available on GitHub.

  • Open source software on GitHub with end-to-end examples
  • Supports SGLang, TensorRT LLM, and vLLM backends
  • SLO Planner, KV-aware Router, NIXL, KV Block Manager, Grove, AI Perf

View official pricing ↗

COMMUNITY REVIEWS

NVIDIA Dynamo User Reviews

Reader reviews

No reviews yet.

Share your experience with NVIDIA Dynamo

Sign in to share your experience. Your display name, rating and review will be public after approval. One review per account per tool.

Your rating