Advancing human-robot collaboration: Predicting operator trajectories through AI and infrared imaging

In industrial applications demanding close interaction between humans and robots, accurate perception of the robot's surroundings is vital to adjust its behavior appropriately. This need becomes especially pertinent when manipulators operate autonomously and execute both separate and common tasks with human operators. To ensure operator safety in such Human-Robot Collaboration (HRC) environments, the present paper proposes an innovative approach that leverages advanced collaborative robotic systems capable of sensing the operator's position to effectively prevent or mitigate potential contact. Our solution combines infrared-thermal imaging technology with a workspace monitoring system, utilizing thermal and depth cameras, to track the operator's current location and predict their future trajectory using a multi-layer Long Short-Term Memory (LSTM) neural network. We also introduce a technique for simulating both human and robot movements to generate a synthetic dataset. This data is shared with the motion planning system to generate trajectories that do not intersect with the operator's future paths, minimizing downtime due to safety stoppages. The solution is integrated and evaluated within an industrial high payload collaborative robot context, and the results have shown improvements in cycle time efficiency and operator approval. This approach offers a promising direction for enhancing safety and productivity in HRC workspaces.