The Research And Implementation Of Handwriting Recognizing System Based On Cudaaccelerated Deep Belief Network

In recent years, deep learning has developed rapidly in many fields of theindustry, while it has not been used widely in the area of handwriting recognizing.More could be done to enhance the recognizing rate of handwriting characters byusing deep belief network. However, the computation complexity of deep beliefnetwork is very high. So if deep belief network is used to train huge number of data,it will take a long time to run the code by just using single thread. For this reason,the possibility of parallel computing need to be analyzed and multiple threads shouldbe used in order to accelerate its operation and save most of the time.This dissertation will implement the acceleration of deep belief network basedon the architecture of CUDA and will optimize the code with multiple kinds oftechniques of CUDA. After implementing the acceleration of deep belief network,this dissertation has a deeper research on it mainly to enhance the recognizing rate ofhandwriting characters, minimize the size of the training model, minimize thedecoding and training time.This dissertation firstly analyzes the parallelism of the deep belief networkfrom the prospect of data and tasks. Then runs the algorithm parallel by using themulti-thread of GPU with some optimizing techniques, such as shared memory,stream, merge access and so on. Given the different back-propagation methods,CUDA C is used to implement the Conjugate Gradient Decent and CUDA Python toStatistical Gradient Decent. After ensuring the efficiency of training data, thisdissertation will improve the deep belief network based on the training ofhandwriting characters by using several methods, such as Rarefaction, Low-Rank,Sparse Rectify Layer and the optimization of Conjugate Gradient Decent.Now, training the deep belief network based on the architecture of CUDA is15times faster than before which meets the requirement of the project. Besides, themethod of Rarefaction could shrink the size of the model and could even enhance therecognizing rate at some certain raring rate. The Low-Rank method could extract theprinciple component of the model which will not only save the training time but alsothe decoding time. Sparse Rectify Layer the third method used makes the neuronnodes sparse that enhance the recognizing rate and reduce the decoding time. Thelast method used is the optimization of Conjugate Gradient Decent which makes theconvergence of deep belief network faster.