Research On License Plate Recognition Algorithm For Complex Environments

With the acceleration of urbanization and the improvement of people’s living standards in recent years,the number of car owners has increased dramatically,and the development of intelligent transportation systems has become increasingly important.License plate recognition is a key part of obtaining vehicle information,and has wide application in city monitoring,road inspection,parking fee,and other scenarios.Quick and accurate access to vehicle license plate information can improve vehicle management efficiency.Existing license plate recognition algorithms perform well in simple scenarios such as indoor parking lots and close gate access.However,license plate recognition tasks in complex environments such as city roadside parking and road inspection are still very challenging.The main difficulties are: 1)the recognition algorithms based on deep learning require a sufficient number of training samples,strong diversity,and balanced character labels,while currently available Chinese license plate datasets are challenging to meet the above requirements;2)the presence of negative factors such as large inclination angle,long distance,uneven luminosity,partial occlusion,motion blur and other harmful factors of license plate in complex environments such as urban roadside greatly aggravate the difficulty of license plate recognition task.Therefore,this paper researches license plate recognition in complex environments.The main research contents are as follows:1)A license plate generation algorithm based on improved Cycle GAN is proposed.Currently available license plate datasets have defects of insufficient quantity,poor diversity,and unbalanced label distribution.Generated samples of existing license plate generation algorithms can hardly meet the above requirements.In this paper,a large number of virtual license plates are generated according to the standards.Then,affine transformation is used to embed the virtual license plates into background images.Finally,Cycle GAN is used to achieve the migration from virtual image to real image.Based on vanilla Cycle GAN,Weight Demodulation(WD)is introduced to solve the white spot defects in generated samples.Spatial and Channel Attention Mechanisms(CAM & SAM)are added and Significant Aera Loss(SALoss)is designed to ensure the realism of the license plate in the generated samples while preserving the diversity of the background.Least Squares Loss(LSLoss)is used to make training more stable.Experimental results show that the generated samples are similar to the real license plate images and have a balanced character distribution,which also outperform other generation algorithms in terms of realism and diversity.A dataset containing more than 20,000 generated license plate images is constructed,which is categorized into various scenes including large inclination angle,long distance,blur,complex lighting,rain,snow,fog,etc.The dataset has been publicly released to the address:https://github.com/DGUT-Io T/DGUT＿LPR.2)A license plate recognition algorithm called RD-LPRNet based on image correction and denoising is designed.The existence of negative factors in complex scenes such as large inclination angles,long distance,and uneven illumination of license plate targets greatly aggravates the difficulty of license plate recognition.Firstly,an improved spatial transformation network called RNet is designed to rectify license plates.Then a denoising algorithm DNet is adopted,which can effectively eliminate irrelevant noises in license plates and retain effective features of characters.Finally,a new license plate recognition is designed based on LPRNet.The input part of the algorithm is redesigned to enrich the initial features and accelerate the convergence.CAM and SAM are introduced in the feature extraction part to suppress irrelevant information and enhance effective features.Depth-wise separated convolutional layer is used to strengthen feature extraction capability.Experimental results show that the proposed RD-LPRNet algorithm enables the license plates to be recovered to near its standard forms in the pre-processing stage,which greatly eliminates the influence of negative factors and effectively improves the accuracy of license plate recognition.3)A license plate recognition algorithm based on video temporal information compensation is proposed.Negative factors in complex scenes such as partial occlusion and motion blur on license plates could lead to the partial absence of character features in single-frame images.Redundant information exists in the video temporal information,which can be used to compensate for the missing features.A video temporal information compensation algorithm is designed based on the previously proposed RD-LPRNet algorithm in this paper.Firstly,the algorithm adopts the Non-local Attention mechanism to assign weights to the feature maps of the current image frame and multiple preceding reference frames to enhance the non-interference regions and suppress the interference regions.Then,the weighted feature maps of the current frame and the reference frames are summed to aggregate useful features.Additionally,a memory module M is constructed to store the aggregated features for use in subsequent frames,saving a large amount of computation used for reference frames feature extraction.Ablation experiments and comparison experiments are conducted on multiple commonly used single-frame license plate datasets and a manually collected multi-frame license plate dataset.Results show that the proposed algorithm can effectively compensate for the missing information and improve the accuracy of the license plate recognition task in partial occlusion and motion blur scenes.