Computer Vision-Based Geometry Mapping and Matching of Building Elements for Construction Robotic Applications

Robotic automation of construction tasks is a growing area of research. For robots to successfully operate in a construction environment, sensing technology must be developed which allows for accurate detection of site geometry in a wide range of conditions. Much of the existing body of research on computer vision systems for construction automation focuses on pick-and-place operations such as stacking blocks or placing masonry elements. Very little research has focused on framing and related tasks. The research presented here aims to address this gap by designing and implementing computer vision algorithms for detection and measurement of building framing elements and testing those algorithms using realistic framing structures. These algorithms allow for a stationary RGB-D camera to accurately detect, identify, and measure the geometry of framing elements in a construction environment and match the detected geometry to provided building information modeling (BIM) data. The algorithms reduce identified framing elements to a simplified 3D geometric model, which allows for robust and accurate measurement and comparison with BIM data. This data can then be used to direct operations of construction robotic systems or other machines/equipment. The proposed algorithms were tested in a laboratory setting using an Intel RealSense D455 RGB-D camera, and initial results indicate that the system is capable of measuring the geometry of timber-frame structures with accuracy on the order of a few centimeters.