Multi-Linguistic Acronym Disambiguation Based On Crossed Pre-trained Language Model

Acronyms,which are shorter combinations of individual characters or initials to replace frequently occurring or more complex and redundant phrases.Acronym disambiguation requires extensive background knowledge and contextual understanding.At the same time,acronyms exist in any language in the world.The research on this issue is still weak at home and abroad.In this case,we extend the acronym disambiguation to the zero-shot inference on multilingual data for the first time.We propose various strategies based on the Fine-Tuning and Prompt Tuning method for investigation.We first introduce fine-tuning.Acronym disambiguation is transformed into a classification through the construction of suitable positive and negative samples.The input of the text and the sentence vectors are optimized.The optimization goal of classification is not a direct recognition of the correct disambiguation words for the acronyms due to the dataset.Then we also propose a model based on entity recognition,which avoids the impact of constructing positive and negative samples.Meanwhile,two types of contrastive learning tasks are added to the training process,which are used to facilitate the learning of the model for the acronym disambiguation and the learning of the model in the multilingual case,respectively.We design a variety of templates to construct the prompt tuning models,which also optimize for text input and sentence vector.We also propose Lizards,which greatly reduces the training time consumption by freezing the language model parameters,while obtaining 92%performance maintenance.Furthermore,we apply soft prompt for the extension of prompt tuning methods on new language data.The experiments are first conducted on monolingual data.Then we use the same experiment setting in multilingual data sets to verify the effectiveness and the comparison between different methods.Eventually,the experimental results show that the methods designed in this paper achieve better results in inference on new language datasets,which greatly promotes the research of acronym disambiguation and extends the research boundary.