Semantic Enhanced Text Representation Modeling Techniques And Applications Based On Bayesian Latent Variable Models

With the arrival of the information age,text data exists widely in various scenarios in multiple forms,such as emails,news reports,research articles,and job descriptions.Although the explosive growth of text data enriches people’s information life,it has brought significant challenges for machine learning algorithms to understand text semantics.To cope with complex,unstructured,and multi-source text data and simplify the design process of machine learning algorithms,text representation modeling has become a hot research topic in machine learning.However,existing algorithms still face several problems and challenges,such as text representation collapse,insufficient interpretability,and difficulty in capturing the correlation between multi-source text data,resulting in the lack of semantic information in text representation vectors.To address these challenges,it is necessary to extract discriminative information from high-dimensional and complex discretized text inputs to form low-dimensional continuous representation vectors that downstream task models can understand and apply.On the other hand,it is also necessary to capture different interpretable factors that control text generation and give the dimensions of representation vectors a semantic meaning that humans can understand.At the same time,when addressing real-world application scenarios,it is also necessary to explore and exploit the correlation of multi-source text datasets.To this end,this thesis intends to conduct in-depth research in three sub-tasks of text representation modeling,i.e.,text representation learning,text representation explanation,and practical application of text representation.In particular,Bayesian latent variable models with various semantic enhancement strategies will be used as the primary frameworks due to their powerful capabilities of data representation and uncertainty evaluation.Specifically,for these three sub-tasks in text representation modeling,this thesis will carry out semantic-enhanced variational inference methods for representation learning tasks,semantic-enhanced topic modeling for representation interpretation tasks,and semantic-enhanced representation associating mining for practical applications under personality analysis scenarios and talent recruitment scenarios.First,for the task of text representation learning,in order to avoid the convergence of representation vectors and improve the discrimination of text representations,this thesis proposes a novel variational inference method for learning a more diverse and uncertain latent representation space.When using the Bayesian hidden variable model to model text data representation,they may fail to diversify the posteriors of difference text samples by sampling using the single posterior distribution component to model all samples.It leads to uninformative and indiscriminative latent representations.This phenomenon is particularly well-known as posterior collapse.Therefore,this thesis takes a classic Bayesian latent variable model,Variational Autoencoder(VAE),as a representative case.Based on the theoretical analysis of geometric properties of the latent variable space,the close relationship between the distribution of posterior parameters and posterior collapse will be explored in depth.Furthermore,this thesis proposes to exploit Dropout on the variances and Batch-Normalization on the means simultaneously to regularize their distributions implicitly.This strategy will enhance the diversity of representation vectors and enrich the corresponding semantic information among them.Extensive experiments on multiple benchmark datasets clearly show the proposed novel model and its variants achieve the best performance in likelihood estimation and classification tasks with semantic-enhanced representation vectors.Secondly,for text representation explanation,this thesis proposes a novel method to enhance the interpretability of text representation vectors based on text topic modeling,which aims to mine the interpretable factors behind the text data and enrich the semantic information of the representation vector.As a representative of Bayesian latent variable models,the text topic model totally ignores the semantic dependency among words and results in the lack of semantic comprehension and model interpretability.Therefore,this thesis revisits topic modeling by transforming each document into a directed graph with word dependency as edges between word nodes and develops a novel method,namely Graph Neural Topic Model(GNTM).Specifically,this thesis defines a new topic concept by introducing the distribution of word dependency edges as the part,which enhances the semantic information of the text topic and improves the interpretability of the representation vector.A well-defined probabilistic generation story is designed to model the document graph structure,and the word set jointly.Theoretical analysis shows that the traditional topic model is a special case of the graph-structured topic model proposed in this thesis.Experimental results on multiple benchmark datasets have clearly demonstrated the effectiveness and interpretability of GNTM in text representation modeling compared with state-of-the-art baselines.Thirdly,for the practical applications of text representation,to mine the correlation among text data sources,this thesis takes the personality analysis as a representative case,and proposes a text representation correlation mining method to model the relationship between employee personality and work behavior based on joint representation modeling.In the personality analysis scenario,employee personality is considered to be one of the dominant factors of work behavior,but the quantitative relationship between the two is rarely explored.Therefore,this thesis explores the correlation between the employee’s personality description and the work behavior record.Combined with the joint representation modeling strategy,a joint Bayesian latent variable model,called Joint-PJB,is proposed by assuming that employee personality,job position,and work behavior share the same data representation vector space.In addition,this thesis further designs an algorithm based on the Joint-PJB model to assist talent recruitment and retention from the perspective of employee personality.Case analyses and quantitative experiments show that Joint-PJB effectively captures the correlation between multiple data sources,providing a new possibility for understanding people’s personality traits in different job contexts and their impact on work behaviors.Fourth,this thesis turns to modeling the hierarchical relationships among multiple text sources in practical application scenarios.Specifically,taking the typical scenario of talent recruitment interviews as an example,this thesis proposes a series of methods for mining hierarchical relationships in text representation vectors based on collaborative representation modeling.The candidate’s resume is designed based on the job descriptions,and the interview assessments are conducted based on the resume and the job position.However,there is little research on mining hierarchical relationships among multiple text sources in the existing literature.Therefore,taking the talent recruitment interview scenarios as an example,this thesis first designs a basic model(JLMIA)by using a joint representation modeling method to model the correlation between the candidate’s resume and interview assessment report and using the collaborative representation modeling method to model the hierarchical relationship between the job description with the resume and interview assessment reports.Subsequently,an enhanced model,named Neural-JLMIA,is designed to improve the representative capability by applying neural variance inference.Finally,R-JLMI A is proposed to model multiple hierarchical relationships among those three text collections,which further enhances the semantic interaction among different text sources.In addition,this thesis exploits the proposed approaches for two real-world applications,i.e.,person-job fit and skill recommendation for interview assessment.Extensive experiments conducted on real-world data clearly validate the effectiveness of the proposed models,which provide an interpretable understanding of job interview assessment.