Research On Deep Multi-Task Learning Method Based On Attention Mechanism

The essence of multi task learning is to obtain a reasonable knowledge sharing structure through the joint training between related tasks in the context of deep learning,so as to express the features effectively.The core of multi task learning is to optimize multiple objective functions at the same time and summarize the feature information extracted from different tasks.In the research process of multi task learning methods,this paper notes the following two problems in this field:first,most of the existing multi task learning methods are predefined network training structures,and this kind of solidified prior knowledge sharing structure between training tasks is only suitable for some specific application scenarios,which has strong limitations;Secondly,in the simple deep learning task scenario,the introduction of too many fusion layers will lead to the disappearance of the deep network gradient.However,there are few methods to solve such problems in this field.Secondly,in the simple deep learning task scenario,the gradient disappearance problem of deep network caused by too many fusion layers is introduced.However,there are few methods to solve this problem in this field.This paper introduces one-way connection on the proposed deep multi task learning method to transfer the shallow features to the deep network,letting the key features of the task express normally.To solve the above problems,this paper first proposes a multi task learning method based on attention mechanism.The innovation of this method is to combine spatial attention and channel attention,so that the attention mechanism no longer focuses on single task,but more on the joint feature expression of multi task.Through the attention mechanism training process based on joint features,knowledge sharing information can be extracted well,Avoid the problem of negative knowledge transfer to the greatest extent and improve the generalization performance of the model.Secondly,based on the proposed deep multi task learning method,this paper introduces one-way connection to transfer the shallow features to the deep network,so that the key features of the task can be expressed normally.Finally,the effectiveness of the proposed method is verified by comparative experiments on three public data sets.