Research On Ship Identification Characters Detection And Recognition Based On Image Analysis And Deep Learning

Waterway transportation is not only a bridge of communication between ancient civilizations,but also the foundation of today's Economic Globalization.Nowadays,more than two-thirds of international trade goods are transported by water.Ships are the most important and basic conveyances in waterway transportation.Canals,ports,harbors and waterways are becoming increasingly congested with the expanding use of ships.As unique identities,Ship Identification Characters(SICs)have been playing a significant role in ship traffic behavior control.Automatic SICs recognition is an important part of waterway intelligent transportation systems.In recent years,with the use of image processing,deep learning and other computer vision-related technologies,many impressive achievements,e.g.,car license plates recognition,vehicles tracking and autonomous vehicles,have been made in the domain of landway intelligent transportation systems.However,there is a dearth of published papers concerning the applications of these technologies in the field of waterway intelligent transportation.In this dissertation,we are committed to accurately detecting and recognizing SICs by employing the progressive image processing and deep learning technologies.This dissertation is expected to provide some insights and new methods on the following key tasks of waterway intelligent transportation system,such as ship-to-ship communication,shore-to-ship identification and waterway traffic monitoring.The main contributions of this dissertation are concluded as follows:(1)We propose two ship identification characters benchmark datasets—ZJUSHIPS950 and ZJUSHIPS60K.Also,some reasonable evaluation protocols of the detection and recognition algorithms of SICs are presented.ZJUSHIPS950 and ZJUSHIPS60K are expected to solve the problem of dataset insufficiency of SICs.ZJUSHIPS60K contains 67873 ship images and 39053 SIC images.ZJUSHIPS950 is a multi-style SICs detection dataset with 950 images of different kinds of ships.All the images were captured and selected according to various criteria(different backgrounds,different seasons,different hours,different ships,different SIC styles,different illuminations,different capturing angles and different resolutions).All the valid SICs were manually labeled.Finally,statistics prove the proportionality,representativeness and rationality of the proposed benchmark datasets.In addition,we have released these two SICs datasets to publics for further research.(2)We present a Gestalt Theory-based multilingual ship identification characters detection method.The characters of SICs from different countries or ships are various.Characters contained in different SICs may be different in color,angle,size,aspect ratio,and amount.To handle all these varieties at the same time,we detect SICs from a more abstract level of "Gestalt Theory".As an important psychology theory,Gestalt Theory reveals not only the essence of human perception,but also the fact that we design and make things by following the Gestalt laws.SICs usually show some Gestalt features.In our method,the geometry similarity,projections proximity,characters continuity and low-level features similarity of SICs are first summarized;Then,based on the summarized Gestalt features of SICs,text coarse detection,SICs fine detection,fake SICs elimination and missed characters compensation processes are conducted respectively.Finally,experimental results on ZJUSHIPS950 dataset prove the effectiveness of the proposed method.(3)We present a ship identification characters detection method based on transferred deep convolutional neural network.To use deep convolutional neural networks to detect SICs,the difficulties mainly lie in the various styles,data insufficiency,multi-line printing and tiny size of SICs.Transfer learning is a kind of learning framework that helps to accomplish the interested task by training the models with more easily obtained data from other similar domains.The training data insufficiency of SICs is thus tackled by employing transfer learning.In addition,a mechanism of pyramid of models is presented to detect multi-scale SICs.After a simple post-processing,the final positions of SICs in the input image are detected.Experimental results on ZJUSHIPS60K dataset suggest that our proposed method can accurately detect SICs with a higher precision and a lower false positive rate.(4)We present a multi-style ship identification characters layout normalization and non-segmentation recognition method.A layout normalization algorithm is firstly designed to convert the multi-line-printed characters into one nearly horizontal line,which makes SICs more recognizable.Then,based on the connectionist temporal classification model,we propose a deep convolutional recurrent neural network to recognize the characters of SICs in a non-segmentation way.Our SICs recognition deep model is comprised of four main modules,i.e.,convolutional neural network module,spatial transformation module,bidirectional LSTM module and connectionist temporal classification module.At last,the effectiveness and efficiency of proposed SICs recognition method are tested on ZJUSHIPS60K dataset.