Implementation of an FPGA-Based System to Process Images and Match Keypoints on High-Resolution Pictures

Processing scenery and finding points of interest is crucial for applications in robotics and aerospace missions. Those areas require efficient and reliable visual input processing. Here, field programmable gate arrays (FPGAs) offer essential advantages, like low power consumption compared to CPUs, performing a large number of calculations simultaneously, and having compact hardware. This paper presents an FPGA system that processes incoming camera data, finds points of interest, and matches them across different images on high-resolution images (2048 x 1088). It is a novel approach to implement the complete image processing pipeline on high-resolution images within the FPGA fabric without additional hardware. For keypoint detection and matching, our work uses a modified SIFT algorithm optimized for FPGA implementation processing and a nearest neighbor-based matching method. It was implemented on a Xilinx Kintex-7 FPGA and partially on a NanoXplore NG-Ultra to evaluate a radiation-hardened FPGA for space applications. On the Kintex-7, the keypoint detection achieves a speed of 33 ms per image, and its features are matched on up to 5 images per second. Judging by the resource utilization of one image processing module on the NG-Ultra, porting the entire system on a radiation-hardened FPGA appears feasible.