An accurate approach to real-time machine-readable zone detection with mobile devices

In this article we consider a problem of machine-readable zone (MRZ) detection in document images on mobile devices. MRZ recognition is actively used for fast and reliable automatic personal data extraction from passports, IDs and visas. However, due to the low computing power and limited battery life of most mobile devices, the requirements for the complexity of the used models increase significantly. We present a state-of-the-art MRZ detection approach based on YOLO-MRZ—extremely fast, compact and accurate deep learning model. We consider the MRZ as a graphical object and use the object detection approach to find it. Proposed YOLO-MRZ is 83 times faster than Tiny YOLO v3, weights only 1 MB and well suited for embedded systems and mobile devices: It achieved 62 FPS on the Apple iPhone SE (2020). We address the small-scale MRZ detection problem with two-stage approach in which the YOLO-MRZ model is run twice: If the detected MRZ bounding box is too small or does not meet geometric criteria, we construct the ROI image based on it and run the same detector in the ROI. To assess the quality, we have tested it on 4 public datasets: SyntheticMRZ, MIDV-500, MIDV-2019 and MIDV-2020. Our approach outperforms all other solutions by a wide margin.