A two-stage deep neural network for multi-norm license plate detection and recognition

Abstract

Abstract In this work, we tackle the problem of multi-norm and multilingual license plate (LP) detection and recognition in natural scene images. The system architecture use a pipeline with two deep learning stages. The first network was trained to detect license plates on the full raw image by using the latest state-of-the-art deep learning based detector namely YOLOv2. The second stage is then applied on the cropped image to recognize captured license plate photographs. Two recognition engines are compared in this work: a segmentation-free approach based on a convolutional recurrent neural network where the recognition is carried out over the entire LP image without any prior segmentation and a joint detection/recognition approach that performs the recognition on the plate component level. We also introduced a new large-scale dataset for automatic LP recognition that includes 9.175 fully annotated images. In order to reduce the time and cost of annotation processing, we propose a new semi-automatic annotation procedure of LP images with labeled components bounding box. The proposed system is evaluated using two datasets collected from real road surveillance and parking access control environments. We show that the system using two YOLO stages performs better in the context of multi-norm and multilingual license plate. Additional experiments are conducted on the public AOLP dataset and show that the proposed approach outperforms over other existing state-of-the-art methods.