Representation Learning For Entity Disambiguation

Entity disambiguation is an important problem in the filed of natural language processing.Entity disambiguation is important for the tasks such as understanding the entities in unstructured text,knowledge base population,and understanding the entity queries in search engines.A key problem in entity disambiguation is how to effectively represent entity mentions and candidates.In recent years,neural network models based on representation learning have been successfully applied in multiple tasks in natural language processing,such as question answering,text summarization,and sentiment analysis.In this paper,we study the entity disambiguation task on the basis of representation learning and neural networks.In consideration of the semantic compositionality of natural language,we conclude our study into the following four aspects:1.We study the problem of learning character embedding with character structure.Character or word is the basic unit in text,therefore,learning the embedding of character or word is the basis of representation learning for natural language processing tasks.Traditional embedding learning algorithms are typically based on the context information of words.We believe that the structure of words/characters(such as morpheme for English word,and radical for Chinese character)contains rich semantic information.In this work,we take Chinese character as the example to study the effectiveness of our approach in integrating the character structure.We present a neural network model which incorporates radical information for Chinese character embedding learning.The model not only uses context information but also utilizes character’s radical information.We apply the learned Chinese character embeddings into word segmentation task.Experimental results show that our model performs better than standard context-based approaches.2.We present an entity disambiguation model based on semantic composition.Traditional entity disambiguation methods usually depend on feature engineering,which is time-consuming.The neural networks based on word embeddings can learn effective semantic representations of entity mentions and candidate entities,model the semantic similarity between mention and candidate,and avoid feature engineering.We study the task of entity disambiguation for the entity mentions in unstructured documents.Specifically,we make use of the context information of entity mention as its description information,and propose a general architecture based on semantic composition.Mentions,contexts,and candidate entities are represented separately in the semantic vector space in our model.We apply convolutional neural network and recurrent neural network in this architecture respectively,and compare their performance for entity disambiguation.We conduct experiments on a public data set for entity disambiguation.The experimental results show that the neural network proposed for entity disambiguation achieves promising results,and avoids using hand-crafted features.3.We present an entity disambiguation model with memory network.In many scenarios,different word in the context of mention plays different roles for the modeling of the semantic representation of entity mention and entity disambiguation.In order to distinguish the different importance of each context word,and make use of the more important context clues,we propose a neural network model based on memory network for entity disambiguation.We study the task based on unstructured documents.Experimental results show that the memory network based model can automatically find and utilize important clues in context words.4.We study the task of entity-type query disambiguation for search engine.Entitytype query is the kind of query that only contains an entity name,for example,“Apple” is an entity-type query which may indicate “Apple(company)” or “Apple(fruit)”.A large amount of entity-type queries exist in search engine.If the search engine can understand the meanings of entity-type queries well,then it can return search results with higher quality or recommend related entities that users may be more interested to.Therefore,to improve user experience,we study the problem of entity-type query disambiguation for search engine.Due to the lack of description information for entity-type queries,we make use of history search information of users in limited time as the context information of current entity-type query.Specifically,we utilize recurrent neural networks based on character or word embedding to model the semantic representations of entity-type query and its candidate entities.The optimized objective of the neural network is to correctly classify an(entity-type query,candidate entity)pair.Experimental results on real data in search engine show that history search information is crucial for the disambiguation of entity-type query.In summary,in this paper we study the task of entity disambiguation based on representation learning.The contributions of the thesis are as follows:(1)On the aspect of representation learning,we propose a character embedding learning algorithm which incorporates morphemic structure.Experimental results on Chinese word segmentation show that our method performs better than the embedding learning algorithms which only utilize context information.(2)We present an entity disambiguation model based on semantic composition.Experimental results show that our method performs better than baseline methods.(3)We present an entity disambiguation model based on deep memory network,which has the ability to automatically find important evidences from contexts and explicitly leverage them for entity disambiguation.Experimental results show the effectiveness of our method.