NVIDIA, USC Researchers Present DiSECt Simulator for Robotic Cutting

Cutting of fruits, surgical resections, and other robotic knife operations could become more intelligent, adaptive, and generalized with the DiSECt algorithm.


DiSECt is a Differentiable Simulation Engine for Autonomous Robotic Cutting.
NVIDIA and USC said DiSECt, their differentiable simulation engine for autonomous robotic cutting, could predict forces on knives cutting through soft natural materials.

Adaptive, generalized cutting behavior is a challenge for robotics applications such as food processing, household automation, and surgical manipulation. At the 2021 Robotics: Science and Systems Conference (RSS), researchers from NVIDIA Corp. and the University of Southern California presented DiSECt, which they described as the first differentiable simulator for cutting soft materials. 

The researchers said DiSECt can accurately predict the forces acting on a knife as it presses and slices through natural soft materials, such as fruits and vegetables. The work was nominated for a Best Paper Award at RSS 2021.

The simulator augments the Finite Element Method (FEM) with a continuous contact model based on signed distance fields (SDF), as well as a continuous damage model that inserts springs on opposite sides of the cutting plane and allows them to weaken until zero stiffness, enabling crack formation.

From fluid controllers to differentiable simulation

As in other areas of robotics, simulators can facilitate controller verification, policy learning, and dataset generation. Moreover, differentiable simulators can enable gradient-based optimization, which is invaluable for calibrating simulation parameters and optimizing controllers, said the scientists.

The process of cutting with feedback requires adaptation to stiffness of the objects, applied force during the cut, and often a sawing motion to cut through. To achieve this, researchers use a family of techniques that use feedback to guide the controller adaptation.

However, fluid controller adaptation requires very careful parameter tuning for each instance of the same problem, said NVIDIA. While these techniques are successful in industrial settings, no two cucumbers or tomatoes are the same, rendering this family of algorithms ineffective in a more generic setting.

In contrast, recent research has focused on building differentiable algorithms for control problems, which in simpler terms means that the sensitivity of output with respect to input can be evaluated without excessive sampling. Efficient solutions for control problems are achievable when the simulated dynamics is differentiable, but the process of simulating cutting has not been differentiable so far, said the researchers.

Differentiable simulation for cutting poses a challenge, since naturally cutting is a discontinuous process where crack formation and fracture propagation occur that prohibit the calculation of gradients. The NVIDIA and USC researchers proposed a novel way of simulating cutting that represents the process of crack propagation and damage mechanics in a continuous manner.

DiSECt calculates cutting dynamics

DiSECt implements the commonly used FEM to simulate deformable materials, such as foodstuffs. The object to be cut is represented by a 3D mesh which consists of tetrahedral elements. Along the cutting surface, the NVIDIA and USC team sliced the mesh following the Virtual Node Algorithm.

This algorithm duplicates the mesh elements that intersect the cutting surface and adds so-called virtual vertices on the edges where these elements are cut. The virtual nodes add extra degrees of freedom to accurately simulate the contact dynamics of the knife when it presses and slices through the mesh.

Next, DiSECt inserts springs connecting the virtual nodes on either side of the cutting surface. These cutting springs allowed the researchers to simulate damage mechanics and crack propagation in a continuous manner, by weakening them in proportion to the contact force the knife exerts on the mesh.

This continuous treatment allows us to differentiate through the dynamics in order to compute gradients for the parameters defining the properties of the material or the trajectory of the knife.

For example, given the gradients for the vertical and sideways velocity of the knife, the researchers can efficiently determine an energy-minimizing yet fast cutting motion through gradient-based optimization.

DiSECT deformable objects with FEM

Deformable objects are simulated via the Finite Element Method (FEM). To simulate crack propagation and damage, the researchers insert springs (shown in cyan) between the cut elements of the mesh which are weakened as the knife exerts contact forces. Source: NVIDIA
NVIDIA said it used reverse-mode automatic differentiation to efficiently compute gradients for hundreds of simulation parameters. The simulator used source-code transformation, which automatically generated efficient CUDA kernels for the forward and backward passes of all our simulation routines, such as the FEM or contact model.

This approach allows the researchers to implement complex simulation routines that are parallelized on the GPU, while the gradients of the inputs with respect to the outputs of such routines are automatically derived from analyzing the abstract syntax tree (AST) of the simulation code.

Through gradient-based optimization algorithms, the NVIDIA and USC researchers automatically tuned the simulation parameters to achieve a close match between the simulator and real-world measurements. In one experiment, they used an existing dataset of knife force profiles measured by a real-world robot while cutting various foodstuffs.

The scientists set up the simulator with the corresponding mesh and its material properties, and optimize the remaining parameters to reduce the discrepancy between the simulated and the real knife force profile. Within 150 gradient evaluations, the simulator closely predicted the knife force profile, as demonstrated on the examples of cutting an actual apple and a potato.

As shown in the figure, the initial parameter guess yielded a force profile that was far from the real observation, and the approach automatically found an accurate fit. In their paper, the researchers demonstrated that the found parameters generalized to different conditions, such as the downward velocity of the knife or the length of the reference trajectory window.

NVIDIA knife simulation

Evolution of the knife force profile while the simulation parameters are calibrated via gradient-based optimization. The ground-truth trajectory has been obtained from a real force sensor attached to a robot cutting a real apple. Source: NVIDIA

NVIDIA potato data

Source: NVIDIA

NVIDIA cutting data

Source: NVIDIA

NVIDIA and USC also generated additional data using a highly established commercial simulator, which allows us to precisely control the experimental setup, such as object shape and material properties. Given such data, they could use the motion of the mesh vertices as an additional ground-truth signal.

After optimizing the simulation parameters, DiSECt is able to predict the vertex positions and velocities, as well as the force profile, much more accurately.

NVIDIA knife simulation predictions

Source: NVIDIA

Aside from parameter inference, the gradients of the differentiable cutting simulator can also be used to optimize the cutting motion of the knife. In the full cutting simulation, the researchers represent a trajectory by keyframes, where each frame prescribes the downward velocity, as well as the frequency and amplitude of a sinusoidal sideways velocity. At the start of the optimization, the initial motion is a straight downward-pressing motion.

NVIDIA and USC optimized this trajectory with the objective to minimize the mean force on the knife and penalize the time it takes to cut the object.

Studies have shown that humans perform sawing motions when cutting biomaterials in order to reduce the required force. Such behavior emerged from the researchers' optimization as well.

NVIDIA knife position and force

Source: NVIDIA

After 50 iterations with the Adam optimizer, the average knife force was reduced by 15%. However, the knife slices sideways further than its blade length. The researchers adds a hard constraint to keep the lateral motion within valid limits and perform constrained optimization. Thanks to the end-to-end-differentiability of DiSECt, accurate gradients for such constraints are available and led to a valid knife motion that requires only 0.3% more force than the unconstrained result.

Cutting food items multiple times results in slightly different force profiles for each instance, depending on the geometry of such materials. NVIDIA and USC also presented results to transfer simulation parameters between different meshes corresponding to the same material.

The approach uses optimal transport to find correspondences between simulation parameters of a source mesh and a target mesh (e.g., local stiffnesses) based on the location of the virtual nodes.

As shown in the following figure, the 2D positions of these nodes along the cutting surface allow us to map simulation parameters (shown here is the softness of the cutting springs) to topologically different target geometries.

NVIDIA optimal transport

Source: NVIDIA

“In our ongoing research, we are bringing our differentiable simulation approach to real-world robotic cutting,” said Eric Heiden, an NVIDIA intern and Ph.D. candidate at the Robotic Embedded Systems Lab at USC. “We investigate a closed-loop control system where the simulator is updated online from force measurements, while the robot is cutting foodstuffs. Through model-predictive planning and optimal control, we aim to find time- and energy-efficient cutting actions that apply to the physical system.”

DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting

Email Sign Up

Get news, papers, media and research delivered
Stay up-to-date with news and resources you need to do your job. Research industry trends, compare companies and get market intelligence every week with Robotics 24/7. Subscribe to our robotics user email newsletter and we'll keep you informed and up-to-date.


DiSECt is a Differentiable Simulation Engine for Autonomous Robotic Cutting.

Robot Technologies