Research On Internet Topic Detection Based On Deep Learning And Model Interpretability Methods

In recent years,along with the rapid development of Internet technology,people are keen to use online networking platforms to express their lives.The emergence of a certain viewpoint or unexpected event may arouse great interest among users and release information on related topics.The mining of these topics can help researchers understand real-time information and assist the government in public opinion early warning and public opinion guidance.Therefore,this thesis combines a deep learning pre-training model based on natural language processing and a model interpretability approach to achieve target topic extraction of information released on Internet platforms.This thesis takes statistical theory and combines the latest progress of natural language processing,and adopts various research methods to investigate the keyword detection method of messages posted by users of the social platform "Twitter",and conducts a thorough research on the traditional topic detection models and methods.On the other hand,this thesis introduces pre-training model to train the target topic detection through fine-tuning and other methods and achieves good results in the low resource situation;on the other hand,this thesis combines the model interpretability method with deep learning,through SHAP,LIME and other model interpretable methods and finally complete the recognition of the target keywords.After the overall recognition method is completed,this thesis uses hypothesis testing methods to compare the advantages and disadvantages of verifying the traditional methods with the target keyword detection method proposed in this thesis,and summarizes and analyzes the results.