CES 2025: NVIDIA launches Cosmos world foundation model, expands Omniverse

At CES 2025 in Las Vegas, NVIDIA made two major announcements geared toward the development of physical and generative AI.

NVIDIA announced Cosmos, a platform comprising state-of-the-art generative world foundation models, advanced tokenizers, guardrails and an accelerated video processing pipeline built to advance the development of physical AI systems such as autonomous vehicles (AVs) and robots.

The company also announced generative AI models and blueprints that expand NVIDIA Omniverse integration further into physical AI applications such as robotics, AVs and vision AI. Global leaders in software development and professional services are using Omniverse to develop new products and services that NVIDIA said will accelerate the next era of industrial AI.

Cosmos available under open model license

NVIDIA said that Cosmos models will be available under an open model license to accelerate the work of the robotics and AV community. Developers can preview the first models on the NVIDIA API catalog, or download the family of models and fine-tuning framework from the NVIDIA NGC catalog or Hugging Face.

Robotics and automotive companies, including 1X, Agile Robots, Agility, Figure AI, Foretellix, Fourier, Galbot, Hillbot, IntBot, NEURA Robotics, Skild AI, Virtual Incision, Waabi and XPENG, along with ridesharing giant Uber, are among the first to adopt Cosmos.

“The ChatGPT moment for robotics is coming. Like large language models, world foundation models are fundamental to advancing robot and AV development, yet not all developers have the expertise and resources to train their own,” said Jensen Huang, founder and CEO of NVIDIA. “We created Cosmos to democratize physical AI and put general robotics in reach of every developer.”

Open world foundation models to accelerate next wave of AI

NVIDIA Cosmos’ suite of open models means developers can customize the WFMs with datasets, such as video recordings of AV trips or robots navigating a warehouse, according to the needs of their target application.

NVIDIA Cosmos world foundation models can enable reinforcement learning to accelerate AI agent learning. Source: NVIDIA

Cosmos WFMs are purpose-built for physical AI research and development, and can generate physics-based videos from a combination of inputs, like text, image and video, as well as robot sensor or motion data. The models are built for physically based interactions, object permanence, and high-quality generation of simulated industrial environments - like warehouses or factories - and of driving environments, including various road conditions.

In his opening keynote at CES, Huang showcased ways physical AI developers can use Cosmos models, including for:

• Video search and understanding: enabling developers to easily find specific training scenarios, like snowy road conditions or warehouse congestion, from video data.
• Physics-based photoreal synthetic data generation: using Cosmos models to generate photoreal videos from controlled 3D scenarios developed in the NVIDIA Omniverse platform.
• Physical AI model development and evaluation: whether building a custom model on the foundation models, improving the models using Cosmos for reinforcement learning or testing how they perform given a specific simulated scenario.
• Foresight and “multiverse” simulation: using Cosmos and Omniverse to generate every possible future outcome an AI model could take to help it select the best and most accurate path.

Advanced world model development tools

Building physical AI models requires petabytes of video data and tens of thousands of compute hours to process, curate and label that data. To help save enormous costs in data curation, training and model customization, Cosmos features:

• An NVIDIA AI and CUDA-accelerated data processing pipeline: powered by NVIDIA NeMo Curator, that enables developers to process, curate and label 20 million hours of videos in 14 days using the NVIDIA Blackwell platform, instead of over three years using a CPU-only pipeline.
• NVIDIA Cosmos Tokenizer: a visual tokenizer for converting images and videos into tokens. It delivers 8x more total compression and 12x faster processing than today’s leading tokenizers, according to NVIDIA.
• NVIDIA NeMo: a framework for highly efficient model training, customization and optimization.

Cosmos availability

Cosmos WFMs are now available under NVIDIA’s open model license on Hugging Face and the NVIDIA NGC catalog. Cosmos models will soon be available as fully optimized NVIDIA NIM microservices.

Developers can access NVIDIA NeMo Curator for accelerated video processing and customize their own world models with NVIDIA NeMo. NVIDIA DGX Cloud offers a way to deploy these models, with enterprise support available through the NVIDIA AI Enterprise software platform.

Expanded Omniverse opens new AI opportunities

“Physical AI will revolutionize the $50 trillion manufacturing and logistics industries,” Huang said. “Everything that moves - from cars and trucks to factories and warehouses - will be robotic and embodied by AI. NVIDIA’s Omniverse digital twin operating system and Cosmos physical AI serve as the foundational libraries for digitalizing the world’s physical industries.”

Creating 3D worlds for physical AI simulation requires three steps: world building, labeling the world with physical attributes and making it photoreal.

NVIDIA offers generative AI models that accelerate each step. The USD Code and USD Search NVIDIA NIM microservices are now generally available, letting developers use text prompts to generate or search for Universal Scene Description (OpenUSD) assets. The new NVIDIA Edify SimReady generative AI model can automatically label existing 3D assets with attributes like physics or materials, enabling developers to process 1,000 3D objects in minutes instead of over 40 hours manually.

NVIDIA Omniverse, paired with new NVIDIA Cosmos world foundation models, creates a synthetic data multiplication engine - letting developers easily generate massive amounts of controllable, photoreal synthetic data.

Developers can compose 3D scenarios in Omniverse and render images or videos as outputs. These can then be used with text prompts to condition Cosmos models to generate countless synthetic virtual environments for physical AI training.

Omniverse blueprints speed up industrial, robotic workflows

During the CES keynote, NVIDIA also announced four new blueprints that make it easier for developers to build OpenUSD-based Omniverse digital twins for physical AI. The blueprints include:

• Mega: powered by Omniverse Sensor RTX APIs, for developing and testing robot fleets at scale in an industrial factory or warehouse digital twin before deployment in real-world facilities.
• AV simulation: also powered by Omniverse Sensor RTX APIs, that lets AV developers replay driving data, generate new ground-truth data and perform closed-loop testing to accelerate their development pipelines.
• Omniverse spatial streaming to Apple Vision Pro: helps developers create applications for immersive streaming of large-scale industrial digital twins to Apple Vision Pro.
• Real-time digital twins for computer-aided engineering (CAE), a reference workflow built on NVIDIA CUDA-X acceleration, physics AI and Omniverse libraries that enable real-time physics visualization.

New free Learn OpenUSD courses are also now available to help developers build OpenUSD-based worlds faster than ever.