Research On Temporal Action Detection Based On Neural Network

Temporal action detection is a fundamental research task in the field of video content analysis.Temporal action detection aims to locate the temporal boundaries of action instances and determine action categories in original untrimmed videos.The action classification task is relatively mature,so the difficulty and bottleneck of the temporal action detection task lie in the temporal positioning of actions,i.e.,temporal action proposal generation.This thesis will focus on exploring efficient temporal action proposal generation methods to further improve the results of temporal action detection task.This thesis analyzes the problems that have not been solved by existing boundary matching-based temporal action proposal generation methods.Firstly,the common and superior boundary matching methods only consider the local information of the proposal interval when modeling the proposal features,while ignoring the global information of the video,which is considered as an important complement to the local features of the proposal,proposal modeling features that contain global information are better for fitting confidence scores; secondly,the current methods do not explore the potential tacit knowledge among the candidate proposals,and the knowledge information contained in the categories of action instances is not effectively utilized,which is helpful for generating high quality proposals.This thesis has designed solutions to the problems identified.The solutions designed in this thesis is as follows:1)This thesis proposes a dynamic global feature supplementation method(DGFS)for the feature modeling of the proposal,which is significantly different from the previous boundary matching methods.The DGFS method fuses the proposal interval local features and dynamic global features to generate more efficient proposal modeling features,and uses the features to fit the corresponding confidence scores of the proposals.DGFS method can generate more accurate and reliable proposal confidence scores.2)This thesis proposes a proposal knowledge modeling method(PKM),which first designs a loss function to explore the potential tacit knowledge among candidate proposals to make the proposals obtain more reliable confidence scores.And then a new branch of action classification is designed to assist the training process by exploring the tacit knowledge information of action categories and passing the tacit knowledge of action classification to the candidate proposals.The PKM method further improves the performance by exploring tacit knowledge.This thesis has conducted extensive experiments on general large and challenging datasets,and experimental performance proves that the method designed for this thesis is efficient and achieves excellent performance on both the temporal action proposal generation task and the temporal action detection task.