| Text classification is a foundational technology in natural language understanding,which automatically divides texts into specific categories based on pre-defined topic categories.It is a cornerstone for advancing deep language understanding and has important application value in many natural language processing tasks such as retrieval,information extraction and search engines.With the continuous progress and rapid development of information technology,Tibetan information processing technology has also made rapid progress.As most of the information on the internet is textual data,automatic text classification has become an important foundation for effective management and organization of textual data.Due to limited data resources and technical level,research on Tibetan text classification has not made significant breakthroughs.At present,traditional machine learning methods based on words are mainly used for Tibetan text classification.This method is restricted by Tibetan word segmentation technology and requires complex manual feature engineering.In order to solve this problem,this article focuses on the relevant technologies of Tibetan text classification from the aspects of Tibetan text classification corpus construction,phrase feature extraction and classification methods.(1)Constructing a dataset for Tibetan text classification.In view of the scarcity of Tibetan text classification datasets,this article collected Tibetan text corpus through web crawlers based on the characteristics of Tibetan text and the basic requirements of text classification.The collected corpus was preprocessed by cleaning,document splitting and text segmentation,and a dataset of 1600 Tibetan texts for phrase feature extraction using multinomial naive Bayes was constructed,which laid the foundation for the technical research of Tibetan text classification.(2)Phrase Feature Extraction for Tibetan Text ClassificationDue to the constraints of Tibetan word segmentation technology,words as features have a significant impact on the classification performance of Tibetan text.Based on the analysis of the text classification process and the structure of Tibetan text,this article selected phrase elements and used TF-IDF weighting to extract phrase features of Tibetan text,which laid the foundation for Tibetan text classification based on phrase features and multinomial naive Bayes.(3)Classification Method for Tibetan Text ClassificationBased on the existing technical conditions for hidden language natural language processing technology,this article studied Tibetan text classification methods,and proposed a phrase feature-based multinomial naive Bayes classification method for Tibetan text classification.This method uses phrase features as the basic elements,constructs a classifier for Tibetan text classification using multinomial naive Bayes model and verifies its effectiveness through experiments. |
