Geo-Referenced Occlusion Models for Mixed Reality Applications using the Microsoft HoloLens

Emergency responders or task forces can benefit from outdoor Mixed Reality (MR) trainings, as they allow more realistic and affordable simulations of real-world emergencies. Utilizing MR devices for outdoor situations requires knowledge of real-world objects in the training area, enabling the realistic immersion of both, the real, as well as the virtual world, based on visual occlusions. Due to spatial limitations of state-of-the-art MR devices recognizing distant real-world items, we present an approach for sharing geo-referenced 3D geometries across multiple devices utilizing the CityJSON format for occlusion purposes in the context of geospatial MR visualization. Our results show that the presented methodology allows accurate conversion of occlusion models to geo-referenced representations based on a quantitative evaluation with an average error according to the vertices' position from 1.30E-06 to 2.79E-04 (sub-millimeter error) using a normalized sum of squared errors metric. In the future, we plan to also incorporate 3D reconstructions from smartphones and drones to increase the number of supported devices for creating geo-referenced occlusion models.