Developers have been striving to improve robotic gripping for the past several years, since securely grasping and transferring fast-moving items on conveyor belts holds vast promise for multibillion-dollar industries. Soft Robotics Inc. is using NVIDIA Isaac Sim to close the simulation-to-reality gap in pick-and-place operations for packaged food.
“We’re selling the hands, the eyes, and the brains of the picking solution,” said David Weatherwax, senior director of software engineering at Soft Robotics.
Unlike other industries that have adopted robotics, the $8 trillion food market has been slow to develop robots to handle variable items in unstructured environments, said Bedford, Mass.-based Soft Robotics. Founded in 2013, the company recently secured $26 million in Series C funding from Tyson Ventures, Marel, and Johnsonville Ventures.
Getting a grip with synthetic data and mGripAI
Companies such as Tyson Foods and Johnsonville are adopting automation to help improve safety and increase production in their facilities. Both companies rely on Soft Robotics technologies.
Food packaging and processing companies are using the startup's mGripAI system, which combines soft grasping with 3D vision and artificial intelligence to grasp delicate foods such as proteins, produce, and bakery items without damage.
Soft Robotics is a member of the NVIDIA Inception program, which provides companies with GPU (graphic processing unit) support and AI platforms guidance. The company develops unique models for every one of its gripping applications, each requiring specific data sets.
Picking from piles of wet, slippery chicken and other foods can be a tricky challenge. With Omniverse and Isaac Sim, Soft Robotics can create 3D renderings of chicken parts with different backgrounds, like on conveyor belts or in bins, and with different lighting scenarios.
The company taps into Isaac Replicator to develop synthetic data, generating hundreds of thousands of images per model and distributing that among an array of instances in the cloud. Isaac Replicator is a set of tools, APIs, and workflows for generating synthetic data using Isaac Sim.
It also runs pose-estimation models to help its gripping system see the angle of the item to pick. NVIDIA A100 GPUs on site enable Soft Robotics to run split-second inference with the unique models for each application in these food-processing facilities.
Meanwhile, simulation and training in Isaac Sim offers access to NVIDIA A100s for scaling up workloads.
“Our current setup is fully synthetic, which allows us to rapidly deploy new applications,” said Weatherwax. “We’re all in on Omniverse and Isaac Sim, and that’s been working great for us.”
Simulation solves for occlusion, lighting
A big challenge for Soft Robotics was understanding of how different pieces of chicken stack up and overlap one another when dumped into a pile. In addition to occlusion, glare on wet chicken can throw off detection models.
“How those form can be pretty complex,” Weatherwax said. “A key thing for us is the lighting, so the NVIDIA RTX-driven ray tracing is really important.”
Soft Robotics uses NVIDIA's technology to build 3D models and figure out in a split second which item in a pile is the most accessible for a robot gripper to pick and place. Building synthetic data sets with physics-based accuracy, Omniverse enables Soft Robotics to create such environments.
“One of the big challenges we have is how all these amorphous objects form into a pile,” said Weatherwax.
Production lines in food-processing plants can move fast. But robots deployed with application-specific models promise to handle as many as 100 picks per minute, noted Soft Robotics.
Still a work in progress, success in such tasks hinges on accurate representations of piles of items, supported by training data sets that consider every possible way items can fall into a pile. The objective is to provide the robot with the best available pick from a complex and dynamic environment.
If food items fall off the conveyor belt or otherwise become damaged, then they are considered waste, which directly impacts yield, said the company.
Omniverse and soft grippers boost accuracy, productivity
Meat-packing plants still rely on lines of people for processing chicken, but like many other businesses, they have faced employee shortages. Some food processors that are building new plants can’t even attract enough workers at launch, said Weatherwax.
“They are having a lot of staffing challenges, so there’s a push to automate,” he said.
The Omniverse-driven work for food-processing companies has delivered a more than 10X increase in its simulation capacity, accelerating deployments times for AI picking systems from months to days, claimed Soft Robotics. The company said that enables its customers to get a grip on more than just deploying automated chicken-picking lines.
Soft Robotics reported that Isaac Sim and robotics also help its customers can face employment challenges, which are particularly acute in industries with increased injury and health risks. “Handling raw chicken is a job better suited for a robot,” Weatherwax said.
Article topics
Email Sign Up
Related Simulation News
Related Companies
