NVIDIA to Present Robotics, Simulation Research at ICRA 2023

Paper on the physics of fabric behavior will receive an IEEE Best Paper Award in London.


NVIDIA says simulation can help robots become more capable.
NVIDIA will present more than a dozen research papers on how robots can be trained in simulation to work in the real world at the annual IEEE International Conference on Robotics and Automation.

NVIDIA Corp. today said it will present more than a dozen robotics research papers at this year's IEEE International Conference on Robotics and Automation, or ICRA, which will be held in London from May 29 to June 2.

The Institute of Electrical and Electronics Engineers (IEEE) will recognize one of the company's papers, titled “Geometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior,” with an IEEE Best Paper award at ICRA.

Papers examine training robots in simulation to work in reality

Many of this year’s submissions demonstrate robotics capabilities trained in simulation that were then shown to work in the real world, noted NVIDIA.

Members of the NVIDIA Robotics Research Lab will also present papers at ICRA, including:

  • “DeXtreme: Transferring Agile In-Hand Manipulation From Simulation to Reality” (Read paper, watch video)
  • “MVTrans: Multi-View Perception of Transparent Objects” (Read paper, watch video)
  • “Self-Supervised Learning of Action Affordances as Interaction Modes” (Read paper, watch video)
  • “nerf2nerf: Pairwise Registration of Neural Radiance Fields” (Read paper, watch video)
  • “DexGrasp-1M: Dexterous Multi-Finger Grasp Generation Through Differentiable Simulation” (Read paper, watch video)

Large language models among paper topics

In addition, NVIDIA will present research on machine learning:

  • “ProgPrompt: Generating Situated Robot Task Plans Using Large Language Models” (Read paper, watch video)
  • “DefGraspNets: Grasp Planning on 3D Fields With Graph Neural Nets” (Read paper, watch video)
  • “CuRobo: Parellelized Collision-Free Robot Motion Generation” (Read paper, watch video)
  • “RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control” (Read paper, watch video)
  • “FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments” (Read paper, watch video)
  • “Planning for Multi-Object Manipulation With Graph Neural Network Relational Classifiers” (Read paper)
  • “CabiNet: Scaling Neural Collision Detection for Object Rearrangement With Procedural Scene Generation” (Read paper, watch video)
  • “Global and Reactive Motion Generation With Geometric Fabric Command Sequences” (Read paper, watch video)

Many of the papers above have GitHub code links for those who want to explore, explained the company.

The NVIDIA Robotics Research Lab is a Seattle-based center of excellence focused on robot manipulation, perception and physics-based simulation. It is part of NVIDIA Research, which includes more than 300 leading researchers around the globe focused on topics spanning artificial intelligence, computer graphics, computer vision, and self-driving cars.

NVIDIA partners to exhibit at ICRA

NVIDIA partners such as Ahead.io, Cogniteam, Connect Tech, Mathworks, and Silicon Highway will also be at ICRA. They plan to demonstrate robotics research using the NVIDIA Jetson Orin platform for edge AI and robotics as well as the NVIDIA Isaac simulation platform.

In other partnership news, Numurus LLC last week announced that it has joined the NVIDIA Metropolis partner program for vision AI applications. The Seattle-based company incorporates NVIDIA CUDA-X software libraries, the JetPack development environment, Nsight developer tools, and Jetson system-on-modules within its product line.

Future integration with the NVIDIA TAO Toolkit and DeepStream SDK will enable developers to build AI models and manage datasets even more rapidly, said the companies.

RoboSense, a developer of solid-state smart lidar systems, said it has integrated its second-generation model into Omniverse for physics-accurate, high-fidelity sensor simulation. The Shenzhen, China-based company will use DRIVE Sim to test and validate new lidar systems, as well as provide developers of advanced driver-assist systems (ADAS) access to highly accurate simulation models, said NVIDIA.

NVIDIA’s Isaac Sim and Gym tools allow robots to be trained entirely in simulation before their AI brains are transferred into real robots. This result has been made possible by simulating thousands of robots in parallel using NVIDIA GPUs and by generating millions of training images using our Omniverse Replicator tools.

Email Sign Up

Get news, papers, media and research delivered
Stay up-to-date with news and resources you need to do your job. Research industry trends, compare companies and get market intelligence every week with Robotics 24/7. Subscribe to our robotics user email newsletter and we'll keep you informed and up-to-date.


NVIDIA says simulation can help robots become more capable.

Robot Technologies