Research And Implementation Of Temporal Behavior Detection Method Based On Semi-supervised Learnin

With the development of multimedia technology,video analysis has gradually become an important research direction in the field of computer vision.Temporal action detection needs to determine action categories and temporal boundaries of action instances occurring in videos.To reduce the burden of labeling caused by fully-supervised learning,semi-supervised learning has gradually attracted the attention of researchers.However,due to the limited labeled data under the semi-supervised setting and the complexity of the task,the existing methods have the problem of imbalance and inaccuracy in the pseudo-labels generated by unlabeled data.This is one of the main reasons affecting the performance of semi-supervised temporal action detection.Aiming at the above two problems,this thesis proposes a spatiotemporal perturbation based dynamic consistency for semi-supervised temporal action detection,and a pseudo-label fusion method based on temporal self-ensembling,and implements a temporal action detection system based on semi-supervised learning.The specific work is as follows:(1)A spatiotemporal perturbation based dynamic consistency for semi-supervised temporal action detection is proposed.The framework is based on a teacher-student model and can utilize both labeled and unlabeled data for model learning.Among them,the dynamic consistency algorithm utilizes the spatiotemporal feature perturbation to enhance the generalization ability of the model.At the same time,the attention mechanism is used to solve the problem of imbalance of pseudo-labels.In the training process,the teacher and student models form a cyclical learning relationship.Experiments show that adding spatiotemporal perturbation can improve the generalization ability of the model,and the dynamic consistency algorithm can improve the accuracy of model detection.(2)A pseudo-label fusion method based on temporal self-ensembling to further improve the accuracy of pseudo-labels is proposed.With the advantage of ensemble learning,we integrate the pseudo-labels predicted by the teacher model for unlabeled data in different training epochs,so as to obtain pseudo-labels with better quality,and take them as the goal of student model learning.Experiments prove that more accurate and stable pseudo-labels can generated through the integration of pseudo-labels,and the accuracy of model detection can be further improved.(3)A temporal action detection system based on semi-supervised learning is designed and implemented.The system can visualize the results of temporal action detection in unedited video,so as to facilitate the evaluation of the effectiveness of the model.