NVIDIA has introduced four specialized Cosmos 3 Nano variants for interactive simulations and synthetic data. They generate driving scenes from multiple cameras, LiDAR observations and images for robotics tasks. The model weights are available, with a license permitting commercial and non-commercial use under its terms.
The models are aimed primarily at autonomous driving and robotics developers. They can create extra situations for training and evaluation, such as changes in road lighting or the visual result of a robot action. A realistic image alone, however, does not guarantee accurate physics or make a simulation sufficient for safety evaluation.
Four models for different tasks
In a Hugging Face article, NVIDIA describes four post-trained versions of Cosmos 3 Nano. Each has 16 billion trainable parameters.
Transfer-Auto generates synthetic driving data offline. It can produce synchronized video from up to 11 camera views, including fisheye cameras, and LiDAR data. Developers specify scene structure and traffic motion, while text guides appearance, such as the weather and time of day.
Sim-Auto extends that approach to streaming simulation. It generates short successive chunks of video and LiDAR observations using prior history and updated controls. A complete closed loop also needs a simulation runtime that updates those controls in response to driving inputs.
Sim-Transfer turns information from an existing simulator and a text prompt into color video. The simulator remains responsible for motion, contact and physical state; the model generates the visual observations.
Sim-Bimanual predicts how a scene will look after a two-armed robot acts or a camera moves. It takes an initial image and control inputs. A policy or a person using joysticks can supply the actions.
Why synthetic scenes matter
A robot or autonomous driving system needs more than one road recording or one attempt to pick up an object. Developers need varied conditions and rare situations that can be expensive or difficult to collect at scale. Synthetic data can expand the examples available and help teams study how systems respond when the environment changes.
Consistency also matters: camera images and LiDAR observations should describe the same scene. NVIDIA proposes generating them together and adapting models to a developer's cameras, sensors and operating domain through post-training.
The four sets of weights are available through links in NVIDIA's article. The company specifies the OpenMDW 1.1 license; users should read its terms before deployment. Open weights allow researchers to adapt the models, although running them and checking their results require computing resources and technical expertise.
The authors also say further work is needed to measure how faithfully these models support policy learning and evaluation, and to understand their limitations. Generated scenes are therefore a development tool alongside other testing methods, rather than a replacement for real-world validation.