Augmented Reality Based Control of Autonomous Mobile Robots

A new approach to control and interact with autonomous mobile robots via a head-mounted Augmented Reality device is presented. Such robots are often used for highly repetitive tasks, however, in small and medium-sized enterprises, tasks are often non-repetitive. In such cases, the programming of mobile robots is neither time nor cost-efficient. Hence, a direct human-machine interaction was developed, connecting an Omron LD-90 with a Microsoft HoloLens 2 via a cloud-based communication. Users can send inputs to the mobile robot using voice commands or select from a pre-defined set of actions. The robot will execute the task and send feedback to the HoloLens about its position and other status information. The potential of this new communication approach is exemplified using a small shop floor where a forming machine operator has to fulfill a wide range on tasks, where the mobile robot can support the worker.