Mobile projective augmented reality for collaborative robots in construction

Augmenting virtual construction information directly in a physical environment is promising to increase onsite productivity and safety. However, it has been found that using off-the-self augmented reality (AR) devices, such as goggles or helmets, could potentially cause more health, safety, and efficiency concerns in complex real-world construction projects, due to the restricted field of view and the non-negligible weight of those devices. To address these issues, we propose a mobile projective AR (MPAR) framework in which the AR device is detached from human workers and carried by one or more mobile collaborative robots (co-robots). MPAR achieves glassless AR that is visible to the naked eye using a camera-projector system to superimpose virtual 3D information onto planar or non-planar physical surfaces. Since co-robots often need to move during the operation, we design algorithms to ensure consistent mobile projection with two major components: projector pose estimation and projection image generation. For planar surfaces, MPAR is achieved by a homography-based pose estimation and image warping. For non-planar surfaces, MPAR uses iterative closest point (ICP) for pose estimation and common graphics pipelines to generate projection images. We conducted both qualitative and quantitative experiments to validate the feasibility of MPAR in a laboratory setting, by projecting 1) as-planned building information onto a planar surface, and 2) as-built 3D information onto a piece-wise planar surface. Our evaluation demonstrated centimeter-level projection accuracy of MPAR from different distances and angles to the two types of surfaces.