Bayesian-based Tensor Decomposition For Deep Convolutional Network Compression Methodology And Implementation Study

In recent years,with rapid development of software algorithms and hardware computing power,various network models have been widely applied.However,as network performance continues to improve,increasing dept h of networks has led to issues such as space occupancy and computational complexity,which cannot be ignored.This greatly limits deployment of models in resource-constrained devices.Network compression,which effectively compresses redundant parameters in networks to reduce parameter quantity or computational complexity,has become a research hotspot for many scholars both domestically and internationally.After an in-depth analysis of existing compression techniques in field of networks compression,this thesis conducts thorough research on key issues such as difficulty of ranks selection in low-rank decomposition,combined with tensor decomposition theory.Considering practical needs of model s deployment,two network compression methods based on tensor decomposition and an application system based on model encapsulation are proposed,achieving following innovative research results:1.Firstly,a systematic review and analysis of deep convolutional networks and optimization methods,tensor decomposition theory,compression frameworks based on constrained optimization,and rank selection methods are conducted.Then,further summarizing issues that there are difficulties in rank selection for BTD,possibility of traditional learning rates getting trapped in local minima,limitations in TT and Tucker decompositions that hinder effective trade-off between accuracy and compression ratio,inability to obtain optimal compressed models through indirect compression,complexity in compression steps due to pre-training process,unreasonable rank selection,difficulties in code operations and environment reconstruction for model migration.Relevant research motivations are listed to provide directions for future research.2.In order to address difficulties in rank selection of BTD and possibility of traditional learning rates falling into saddle points and local optima,a thorough analysis of mechanism of BTD decomposition and impact of existing rank selection methods on BTD decomposition and learning rate optimization methods on model accuracy is conducted.Drawing on Bayesian rules for obtaining global analytical solutions and cyclic learning rate methods to avoid local optima,a Bayesian optimization-based block term decomposition method for compressing deep convolutional networks is proposed.This method utilizes Bayes Opt to obtain global analytical solutions and adopts Cos cyclic learning rate to ensure model performance while enhancing decomposition and generalization capabilities.Results demonstrate that the proposed method can achieve better accuracy and higher compression ratio in image classification tasks.3.In order to address limitations of TT and Tucker decompositions,which compromise a balance between accuracy and compression ratio,inability of indirect compression to achieve optimal compressed models,complexity of pre-training process,and unreasonable rank selection,a thorough analysis of advantages of TT and Tucker decompositions is conducted.Drawing on methods such as constrained optimization for obtaining optimal models and Bayesian rules for obtaining global analytical solutions,a TT-Tucker-based LC compression convolutional neural network method without pre-training is proposed.This method utilizes the TT-Tucker composite decomposition method,removes pre-training from LC framework,introduces EVBMF and Bayes Opt for rank selection of TT and Tucker.In addition,this method utilizes an exponential decay cyclic learning rate optimization method to ensure that model can achieve good accuracy and compression ratio while simplifying compression process.Results show that the proposed method simplifies steps of compression process compared to other methods and achieves superior accuracy and compression ratio in image classification tasks.4.In order to address issues such as complexity of code operations and environment reconstruction in model migration,an in-depth study of model encapsulation and GUI development is conducted.Pyinstaller is utilized to package dependencies and Tkinter is used for designing visual interfaces,resulting in development of an encapsulation system for compressing CNN models.This system includes a parameter configuration manual,a login interface,an interactive main interface for model encapsulation,and a parameter configuration interface.It simplifies environment configuration and operation difficulty in model deployment,making process of model migration to different devices convenient and efficient.Effectiveness of the system is validated through system debugging and experimental runs,laying a solid foundation for deploying models on mobile devices.