Construction Of Multimodal Metaphor Dataset For Metaphor Detection

Metaphor is ubiquitous in human daily expressions.According to statistic,there is an average of one metaphorical phenomenon in every three sentences.Metaphor is not only a common way of linguistic expression,but also an important cognitive means.Human beings are used to depicting abstract concepts with the help of concrete concepts.Detecting metaphorical phenomena can help us better understand complex abstract concepts and understand the deep meaning of language.At present,metaphor detection has become an important issue in the field of natural language processing,which is widely used in information extraction,opinion mining,machine translation,sentiment analysis and other tasks.With the vigorous development of social media,language expressions also show the trend of modal diversity.On the other hand,the cognitive structure of metaphor determines its multimodal characteristics.At present,multimodal metaphor is still in its infancy,and the large-scale and high-quality semantic resources available are very scare,which hinders the research process.In order to solve the above problems,the following work has been carried out in this paper:(1)This paper constructs a large-scale,high-quality multimodal metaphor dataset.Based on the theory of conceptual metaphor,the definition of generalized multimodal metaphor and the characteristics of multimodal discourse,this paper proposes three categories of multimodal metaphor.This paper follows the principle of dataset construction,make use of distant supervision,and narrow the scope of data retrieval by building a set of retrieval keywords.Crawler technology is used to obtain data,and specific processing is carried out according to the characteristics of the data.A standardized annotating scheme is developed,and crowdsourcing platform is used for annotating.A quality control mechanism is set up to select reliable annotators,two inter-annotator agreements are used for evaluation,and baseline experiments are set up to verify the reliability of the dataset.(2)This paper proposes a framework for multimodal metaphor detection and multimodal sentiment analysis based on multitask learning.In order to explore the diversity of dataset in application,this paper annotates the tasks related to multimodal metaphor phenomenon and shows the relationship between multimodal metaphor and it's associated language phenomena from the perspective of statistical analysis.This paper uses the cross-modal attention mechanism to capture the dynamic interaction between the modalities,allocate weights reasonably according to the contribution of each modal,and update the information of each modal at the same time.Furthermore,The idea of multitask learning is further introduced,and the interdependence between two tasks is used to improve the performance of the model.