Collaboration Between Humans, Robots Could Improve With Context-Aware System From KTH

A new transfer learning system could provide robots awareness of their environments and human collaborators for greater efficiency and safety.

Hongyi Liu, KTH

KTH researcher Hongyi Liu tests a robot arm by placing his hand in its path.

For safe collaboration, robots must not only be integrated into processes; they must also must also understand the production environment and be able to predict what humans might do next. Researchers at the KTH Royal Institute of Technology last week said they have developed a system that provides context awareness so that robots can work alongside people more safely and efficiently.

Instead of judging distance between a robot and human co-workers, the new system can identify each worker it works with, said Hongyi Liu, a researcher at the Stockholm, Sweden-based institute. The system uses a skeletal model as an abstract of a person's body volume to recognize the worker's pose and predict the next pose. This enables the robot to have contextual awareness while interacting, he said.

KTH's system uses artificial intelligence that requires less computational power and smaller datasets than traditional machine learning methods, Liu said. It relies instead on a form of machine learning called transfer learning—which reuses knowledge developed through training before being adapted into an operational model.

The research was published in the latest issue of Robotics and Computer-Integrated Manufacturing, and Lihui Wang, another KTH professor, was co-author.

KTH researcher Hongyi Liu tests a robot arm by placing his hand in its path.

KTH researcher Hongyi Liu tests a robot arm by placing his hand in its path. Source: Hongyi Liu, KTH

Context awareness said to be better than ISO standard

Liu noted that the new technology came out before the latest International Organization for Standards (ISO) requirements for collaborative robot safety, so implementation would require industrial action. But the context awareness offers better efficiency than the one-dimensional interaction workers now experience with robots, he claimed.

“Under the ISO standard and technical specification, when a human approaches a robot, it slows down, and if he or she comes close enough, it will stop,” he said. “If the person moves away, it resumes. That's a pretty low level of context awareness.”

“It jeopardizes efficiency,” Liu asserted. “Production is slowed, and humans cannot work closely to robots.”

He compared the context-aware robot system to a self-driving car that recognizes how long a stoplight has been red and anticipates moving again. Instead of braking or downshifting, it begins to adjust its speed by cruising toward the intersection, thereby sparing the brakes and transmission from further wear.

Experiments with the system showed that with context, a robot can operate more safely and efficiently without slowing down production. In one test, a robot arm's path was blocked unexpectedly by someone's hand. But rather than stop, the robot adjusted—it predicted the future trajectory of the hand, and the arm moved around the hand.

“This is safety, not just from the technical point of view in avoiding collisions, but [also] being able to recognize the context of the assembly line,” said Liu. “This gives an additional layer of safety.”

The research was an extension of the Symbiotic Human Robot Collaborative Assembly project, which was completed in 2019.

Email Sign Up

Get news, papers, media and research delivered
Stay up-to-date with news and resources you need to do your job. Research industry trends, compare companies and get market intelligence every week with Robotics 24/7. Subscribe to our robotics user email newsletter and we'll keep you informed and up-to-date.

Hongyi Liu, KTH

KTH researcher Hongyi Liu tests a robot arm by placing his hand in its path.

Robot Technologies