Vehicle Type Recognition With Deep Learning

Vehicle type recognition is that computers classify vehicles intelligently based on vehicle’ s types.It is usually the most basic part of many real applications such as intel-ligent traffic,surveillance tracking and unmanned driving.According to the granular-ity of classification,there are coarse-grained and fine-grained vehicle type recognition tasks,which are generally applied to various scenarios independently.Vehicle type recognition has been studied for more than twenty years.But the challenge of multiple viewpoints has never been solved well for either coarse-grained or fine-grained vehicle type recognition.One vehicle has different planar appearances from different viewpoints,which are of a wide variety and cause much difficulties for recognition.In coarse-grained level,the multi-view problem hasn’ t been studied deeply.In fine-grained level,though the influence of multiple viewpoints included,the performance of vehicle type recognition isn’ t good enough.Considering these draw-backs of historical researches,we sum up two unsolved problems about vehicle type recognition:1.Overcome the influence of multiple viewpoints in coarse-grained vehi-cle type recognition;2.Improve the performance of fine-grained vehicle type recogni-tion.Based on the two points,our works are as follows:1.Create a coarse-grained multi-view vehicle type recognition(MVVTR)database.Existing coarse-grained vehicle databases are all for methods which use feature extrac-tion or geometry estimation.These methods don’ t consider the influence of multiple viewpoints.So we can’ t study the influence of multiple viewpoints on vehicle type recognition with theses databases and must create a new one.Therefore,we collect ve-hicle images of seven types from the internet with web crawling technique.And these images are taken from various viewpoints,e.g.front,rear,side,front side and rear side.2.Design a feedback-enhanced multi-branch convolution neural network(FM-CNN).We first train from scratch or fine-tune an existing neural network for vehicle type recog-nition but find that it is unable to learn efficient features in both situations and doesn’ t work at all.Then inspired by traditional image descriptors like SIFT and HOG,which combine multi-scale features,we expand the convolution part of the neural network to multiple branches.These branches take as inputs images of different scales.And to alleviate the interference among branches during training,we add a local classifier after each branch.These local classifiers enhance the feedback from classification results to parameters within branches.3.Fine-tune high convolution layers in FM-CNN to avoid over-fitting.Databases for coarse-grained and fine-grained vehicle type recognition used in this paper are both small.Deducing from the results of training from scratch or fine-tuning the existing single-branch neural network,training from scratch or fine-tuning all the parameters in FM-CNN on these two databases is easily prone to over-fit.Considering the similarity between the features learned by convolution layers in neural networks and the patterns of the respondences of visual neurons in human brains,we assume low convolution layers learn low-level semantic features which are generally applied for different tasks,and high convolution layers learn high-level semantic features which are task-specific.So we initialize FM-CNN with pre-trained weights of the existing neural network and fine-tune only high convolution layers and fully-connected layers.By decreasing the amount of parameters to be updated,we lower the opportunity of over-fitting.After the above works,FM-CNN surpass the state-of-the-art methods in both coarse-grained and fine-grained vehicle type recognition.In coarse-grained vehicle type recog-nition,it achieves 94.9%Top-1 accuracy.In fine-grained vehicle type recognition,it achieves 91.0%Top-1 and 97.8%Top-5 accuracy.