Note to developers: NVIDIA Cosmos-Dreams was formerly named NVIDIA OmniDreams. OmniDreams was the research name for this model. It has been renamed Cosmos-Dreams to align with NVIDIA Cosmos family of world foundation models. The model weights, repository, and URL are unchanged. References to OmniDreams in the research blog, technical report, and model card refer to Cosmos-Dreams.

Resources: Research blog | Technical Report | Model weights

NVIDIA Cosmos-Dreams is a world model that generates multi-camera photorealistic video for autonomous-driving simulation in real time.

The model consumes:

• A single input frame
• Initial text prompt
• Per-frame coarse HD map image and trajectory poses

It produces photorealistic video frames in chunks.

Rollout starts from one real RGB frame. That frame anchors the appearance of the scene. The text prompt describes the driving context, while the per-frame HD map image and trajectory poses provide the structured conditioning for each generated chunk. The world model autoregressively generates the next video chunk, then feeds that chunk back into the next step so the scene can continue over time.

Interactive inference and live driving demos now live in the companion flashdreams project. This repo owns the post-training sample and release tree.

This repository contains the following samples for demonstration:

• samples/post-training - launchers and configs for fine-tuning the Cosmos2 SV-HDMap world model on a single 8-GPU node or a Slurm cluster (student-init, bidirectional teacher, and self-forcing distillation). See samples/post-training/QUICKSTART.md for the four-step out-of-box flow.

For interactive driving and offline, reproducible batch mp4 video generation, see the companion flashdreams project.

Join us on the NVIDIA Omniverse Discord to share your results, attend office hours, and take part in technical discussion with the NVIDIA Cosmos-Dreams team and community.

If you are new, start with the server invite to on-board.

Channels:

• #omnidreams
• #flashdreams
• #world-models-chit-chat

Use a recent NVIDIA driver compatible with the CUDA stack in the selected workflow. The post-training quickstart was validated with driver 570.148.08 and CUDA 12.8 on 8x H100 80 GB HBM3.

The post-training sample stages both checkpoints and gated Hugging Face sample data on first run. The sample dataset repo and branch are:

• nvidia/PhysicalAI-Autonomous-Vehicles-NuRec — PAI-NuRec 26.01 sample scenes for post-training

You also need access to the OmniDreams model checkpoint repo under your authorized Hugging Face org. By default the post-training scripts look under nvidia; set OMNI_DREAMS_HF_ORG=<YOUR-HF-ORG> if your access lives under a different org.

To allow automated download, create a Hugging Face token at https://huggingface.co/settings/tokens/new.

export HF_TOKEN=<YOUR-HF-TOKEN>    # optional: used by quick access checks only

The post-training runtime reads the token from $OMNI_CACHE_DIR/huggingface/token, not directly from HF_TOKEN. Follow the post-training quickstart's staging step to write the token file before running setup_env.sh.

If any download fails with 401, 403, or a gated-repo error, verify both the token and the repo access above before debugging anything else.

Use the FlashDreams OmniDreams runner for inference and interactive driving. The inputs are the same pieces described above: an initial RGB frame, a text prompt, and HD-map / trajectory conditioning frames. Batch inference produces a reproducible mp4 sequence; the interactive-drive integration provides the live driving experience.

The interactive driving sample that previously lived in this repository has moved to FlashDreams:

• FlashDreams OmniDreams model docs
• FlashDreams interactive-drive sample

For fine-tuning the Cosmos2 SV-HDMap world model on a single 8-GPU node (student-init, bidirectional teacher, or self-forcing distillation), see samples/post-training/QUICKSTART.md. It covers HuggingFace auth setup, checkpoint + sample-dataset staging, and the launcher invocations for all three experiments. Slurm wrapper included for multi-tenant clusters.

Cosmos-Dreams is licensed under the Apache License, Version 2.0. Third-party runtime dependencies are fetched by package managers or companion projects such as flashdreams.

• LICENSE — repository-wide Apache-2.0 grant
• CONTRIBUTING.md — DCO sign-off and PR conventions
• REUSE.toml — per-path / per-file license metadata
• THIRD_PARTY_NOTICES.txt — upstream attribution for vendored third-party code
• NOTICE — third-party-fetch notice for runtime / install-time downloads