NVIDIA Sees Simulation Continuing to Add Value for AI, Robotics Developers Heading Into GTC 2023

NVIDIA Corp. will be putting on GTC, a global conference for developers of artificial intelligence, computer graphics, autonomous machines, and more.

In advance of GTC, Robotics 24/7 caught up with Gerard Andrews, senior product marketing manager at NVIDIA. He discussed the accelerated computing pioneer's innovations in simulation, the economic outlook, and ways in which NVIDIA can help robotics developers.

Isaac Sim already changing robot design, training

What were some of NVIDIA's accomplishments of the past year that you're proudest of?

Andrews: It was a great year for NVIDIA as AI and robotics continue to converge. We released important new technologies in Isaac, our end-to-end robotics platform that is being used by the greater robotics ecosystem to build, test, train, and deploy intelligent robots.

Gerard Andrews, NVIDIA

We announced our latest Isaac Sim release, which has gained traction with customers and partners worldwide. The video [below] demonstrates the breadth of robotics applications where our robotics simulator is proving beneficial.

One of the hottest use cases is for synthetic data generation, with roboticists building more robust ML [machine learning] models using the training data generated and automatically labeled in Isaac Sim.

Moreover, the Isaac ROS Humble release was a key accomplishment in partnership with Open Robotics. We improved the framework to make it easier to build hardware accelerated ROS 2 applications.

We launched a new webinar series for ROS developers that has been extremely well received and available for on-demand viewing.

We’re extremely proud of the release of the NVIDIA Jetson Orin edge AI platform, the highest-performance, energy-efficient computer for roboticists currently available (per MLPerf benchmarks). It’s the brains of a new generation of autonomous robots that can sense and perceive the world better and faster.

Can you give some examples of how developers are using NVIDIA Omniverse and the Isaac robotics platform in creative ways?

Andrews: The end-to-end Isaac robotics platform provides advanced AI and simulation software and accelerated computing capabilities -- with corresponding hardware from the Jetson family -- to the entire robotics ecosystem.

More than 1,000 companies rely on one or many parts of NVIDIA Isaac, including companies that deploy physical robots developed and tested in the virtual world using Isaac Sim on Omniverse. Public examples of Isaac Sim users include:

• Amazon Robotics uses NVIDIA Omniverse Enterprise and Isaac Sim to simulate warehouse design, train robots with synthetic data generation (via Isaac Replicator), and gain operational efficiencies before physically implementing them in warehouses. This helps the e-commerce giant fulfill thousands of orders in a cost- and time-efficient manner [see video below].
• Telexistence has deployed beverage restocking robots across 300 convenience stores in Japan.
• Deutsche Bahn is training AI models to handle very important but unexpected corner cases that happen rarely in the real world — like luggage falling on a train track, to improve safety.
• Festo uses Isaac Cortex to simplify programming for cobots and transfer simulated skills to the physical robots.
• Fraunhofer is developing advanced autonomous mobile robots (AMRs) using the physically accurate and full-fidelity visualization features of Isaac Sim.
• Soft Robotics is harnessing NVIDIA Isaac Sim to generate synthetic data to train their robotic gripping systems to handle delicate foods like raw meat or bakery items.
• Sarcos Robotics is developing robots to pick and place solar panels in renewable energy installations.
• Flexiv is using Isaac Replicator for synthetic data generation to train AI models for complex pick-and-place tasks.

Some challenges ease, but labor shortages still drive robotics demand

How much have economic headwinds affected NVIDIA's customers? Are labor or processor shortages easing?

Andrews: Labor shortages and the re-engineering of supply chains are continuing to drive interest in robotics and automation.

Companies will leverage more of our Isaac platform as it allows them to incorporate cutting-edge AI and simulation technologies while maintaining ultimate focus on the unique problems that they are trying to solve.

Overall, the supply chain situation is better than last year.

What product is the biggest game-changer for robotics developers -- and users?

Andrews: Simulation is the biggest game changer. All aspects of robotics development will leverage simulation. As the robots become driven by more complex software stacks, the simulators will be required to develop the software and validate its functionality over the life of the robots.

Simulation provides a way for everyone to do more work, and given capital constraints, Isaac Sim is becoming essential.

Simulation also helps with planning and deployment, where the game changer is robot fleet deployment. Robotics business scaling and fleet management for AMRs gets easier and faster with Isaac Sim, so companies can serve even more customers with existing resources.

What should robotics startups know as they try to differentiate themselves in maturing markets with little venture capital?

Andrews: Robotics startups should stay laser-focused on the problems they are trying to solve for the specific verticals they are targeting. They should have experts on their teams within that vertical space to help them understand everything they can about their prospective customers' challenges.

It is very easy for all of us to fall in love with the tech and then inadvertently lose focus on the business problems that we are trying to solve!

And, as mentioned earlier, startups should leverage platforms and the ecosystem to build their solutions. There's definitely no need to reinvent the wheel, and, ultimately, no one is going to pay much for a re-invented wheel.

NVIDIA shares 2023 outlook

Can you give us a sneak peek at what NVIDIA is working on for the coming year?

Andrews: Stay tuned for all of the new tools and capabilities that we will be releasing in our robotics simulation platform, Isaac Sim. We recently added people simulation to the platform. That just came out and we can’t wait to see how the ecosystem embraces this new capability and does more cool simulation.

We will be adding more simulation-ready assets to the Omniverse platform to reduce the cost and time to building meaningful robotic simulations, especially in warehouse and manufacturing environments. In the near future, multi-robot simulation will be a major new feature upgrade. Our customers simulating the performance of fleets of robots are clamoring for this.

We have also started the process of making Isaac Sim available in the cloud. We will continue to make it easier and as frictionless as possible for customers and partners to leverage the power of cloud computing to scale their synthetic data generation operations and to build out their continuous integration and continuous delivery (CI/CD) testing and verification infrastructure.

And of course, with over 1 million developers, 6,000+ customers, and 150 ecosystem partners worldwide, we’re continuing to build momentum behind the NVIDIA Jetson platform. We're launching a full range of Orin-based developer kits and production modules for companies to scale their edge AI and robotics applications.

Looking ahead, where do you think robotics applications will grow the most?

Andrews: From an application area, I am really excited to see the continued growth and innovation of robotics technology in the agtech industry. What better problem could automation address than helping with sustainable and more efficient food production?

Companies like John Deere, Monarch Tractor, Verdant, Agrist, FarmWise, and many others are creating unique and compelling solutions.

Logistics and manufacturing will continue to leverage robotics as they have for years. I am particularly interested to see the emergence of mobile manipulators in this space.

We have always had mobile robots that moved things from Point A to Point B and manipulators that have done picking and sorting. But mobile manipulators will be able to move to a location and then do more complex manipulation tasks. This will allow more processes and tasks to be automated in warehouses and factories.

What roboticists should look for at GTC

What are some things to look for at GTC next week?

Andrews: Make sure to attend our featured sessions on NVIDIA Isaac and Jetson platforms at GTC from March 20 to 23. Learn how edge AI, simulation, and synthetic data generation are accelerating robotics research and deployment.

Noteworthy sessions include "Training Highly Dynamic Robots for Complex Tasks in Industrial Applications" and "Design a Complex Robot Architecture on NVIDIA Isaac ROS."

Anyone interested in the future of AI and robotics should register for free today, which will also open access to replay the on-demand sessions.

Will you be speaking at the event?

Andrews: Yes, I'll be hosting "Connect with the Experts: Development and Simulation of Autonomous Robots," a Q&A session with NVIDIA robotics engineers and product leaders.