As developers build robots that adapt to their environments, handle diverse items, or interact safely with people, they need tools that facilitate the training of machine learning systems. NVIDIA Corp. today announced the availability of the 2022.1 release of NVIDIA Isaac Sim.
“With Isaac Sim, developers can generate production-quality datasets to train AI perception models,” wrote Gerard Andrews, senior product marketing manager for robotics at NVIDIA, in a blog post. “Developers will also be able to simulate robotic navigation and manipulation, as well as build a test environment to validate robotics applications continually.”
“This follows our strategy for end-to-end robotics,” Andrews told Robotics 24/7. “We segment the intersection of robotics and AI into four phases of development: No. 1, train with Isaac Replicator and synthetic datasets. No. 2, simulate with digital twins, which are the focus of today's release.”
“No. 3, build using Jetson Orin, Isaac ROS, SDKs [software development kits], and conversational AI, and No. 4, deploy and manage using our suites for fleets of robots in warehouses and other industrial settings,” Andrews said. “It's really about safer simulation for hazardous environments, faster training in parallel and with GPU-accelerated GEMs, and lower cost than testing in the real world.”
Why don't more roboticists use simulation? “Often, it doesn't model the behavior of a robot closely enough,” replied Andrews. “Once the reality gap is too large, developers would rather go to the lab, even if that's slower, smaller-scale, and more expensive.”
“With this release of NVIDIA Isaac Sim, we're delivering on incrementally closing that reality gap,” he said.
Isaac Sim 2022.1 release features
Isaac Cortex is a decision framework for training collaborative robots. “Program cobot tasks as easily as programming game AI,” said Andrews.
“Leverage this decision framework for cobots to develop task-aware and adaptive skills,” he said. “Using its belief representation of the world, analogous to the robot’s brain, real or simulated data can be used as inputs, and the resulting actuations will be generated.”
Andrews cited the example of working with Festo on a picking system (see video below) and said it's now available.
Isaac Gym is a reinforcement learning framework accelerated by NVIDIA's graphics processing units (GPUs; see video above). It keeps the entire learning workflow on the GPU to reduce training time, Andrews wrote.
“ETH Zurich/Swiss Mile created a wheeled version of ANYbotics' ANYmal and trained a new locomotion policy in Gym,” he said. “We had a standalone version for reinforcement learning, but it's now available as part of Isaac Sim.”
Omnigraph is a visual programming and scripting environment intended to simply application development and debugging.
“Connecting nodes with different inputs and outputs to drive compute tasks translates well to robotics applications, which already tend to be modular,” noted Andrews. “As with passing ROS [Robot Operating System] nodes, this works well with visual programming.”
Isaac Sim/Gazebo Connector is a feature to allow ROS developers to move between Gazebo and Isaac Sim for tasks such as generating synthetic datasets or high-fidelity rendering, said NVIDIA. In addition, users can keep multiple Gazebo simulations synchronized by connecting to Omniverse's nucleus server.
“The community can use any simulator as needed,” Andrews said. “The option of the high-fidelity Isaac Sim gives flexibility to developers.”
Isaac Replicator is a set of synthetic data-generation (SDG) tools, application programming interfaces (APIs), and workflows. It allows robots to be trained in minutes instead of weeks, claimed NVIDIA.
In its latest release, the company added the SceneBlox SDG feature to Replicator to generate industrial environments procedurally. It includes modules for conveyors and adds examples on how to generate synthetic data and train a pose-estimation model.
Developers to get more support
In addition, NVIDIA Isaac Sim 2022.1 includes limited support for Windows. It previously supported only Linux, but some customers asked for Windows, explained Andrews.
“Fraunhofer's AMR [autonomous mobile robot] is omnidirectional, can move fast, and can pick up heavy loads,” said Andrews. “It has plans to open-source the platform.”
“Our panel on digital twins at Automate last week was well-received,” he recalled. “The panelists from companies like Kroger and our newly announced partnership with READY Robotics demonstrate that the technology is becoming really important to diverse users.”
“Our discussions with them and people at our booth showed that it's not just the technology providers—end users are starting to think about how digital twins and simulation can help them improve efficiency, save money and energy, and make a smaller climate impact,” said Andrews.
“The overall theme of this release is that NVIDIA is providing a rich toolbox for developers to close the simulation-to-reality gap,” he said.
ICRA 2022 workshop on "Releasing Robotics into the Wild" with Liila Torabi.
About the Author
Eugene DemaitreEugene Demaitre is editorial director of Robotics 24/7. Prior to joining Peerless Media, he was a senior editor at Robotics Business Review and The Robot Report. Demaitre has also worked for BNA (now part of Bloomberg), Computerworld, and TechTarget. He has participated in numerous robotics-related webinars, podcasts, and events worldwide.
Follow Robotics 24/7 on Facebook
Stay up-to-date with news and resources you need to do your job.
Research industry trends, compare companies and get market intelligence every week with Robotics 24/7.
Subscribe to our robotics user email newsletter and we'll keep you informed and up-to-date.
NVIDIA demonstrates Isaac Sim 2022.1 with partners ANYbotics, Festo, and the Fraunhofer Institute in a simulated warehouse.
NVIDIA is the pioneer of GPU-accelerated computing. The company said its invention of the graphics processing unit (GPU) in 1999 redefined computer graphics and gaming, ignited the era of modern AI, and fueled the creation of the industrial metaverse. The GPU now acts as the brains of robots,...
Festo is a leading global manufacturer of pneumatic and electromechanical systems, components and controls for process control and factory automation solutions, with more than 55 national headquarters serving more than 180 countries.