Research On The Method Of Alignment Between Face And Person Names

With the explosive growth of multimodal news data available both on-line and off-line, how to integrate multimodal information sources to achieve more accurate cross-media news retrieval becomes an important research issue. Usually multimodal news is exhibited with the form of captioned news images, which mostly describe stories about people. Thus the situations that a user needs to use a retrieval system to perform querying multimodal news related to a specific person are becoming increasingly emerging and common. The general solution is to search news associated with a particular person by utilizing his/her name in the news image caption. However, a simple textual query is sometimes inadequate and it’s very likely to yield incorrect results, since in a certain captioned news image there may exist the weak correspondence between the textual information (i.e., person names in the caption) and the visual information (i.e., detected faces in the image). Automatic name-face alignment has been a challenging task for supporting more effective cross-media retrieval of large-scale captioned news images.In this paper, a new framework is developed to support more precise automatic name-face alignment for cross-media news retrieval. We first focus on analyzing text and image contents associated with captioned news images and developing the related techniques to extract valuable information from texts and images. Our multi-level analysis of image-caption pairs is established for characterizing the meaningful names with higher salience and the cohesion between names and faces more accurately. To remedy the issue of lacking enough related information for rare names, Web mining is introduced to acquire the extra multimodal information, which is very helpful for discriminating the association relationships between rare names and their related faces. We also particularly emphasize on establishing an efficient measurement and optimization mechanism by our Improved Self-Adaptive Simulated Annealing Genetic Algorithm (ISSAGA) to verify the feasibility of alignment combination of names and faces. For enhancing the whole alignment performance of our approach, a novel model is developed by integrating Name Salience Ranking (NSR), Name-Face Cohesion Measure (NFCM), Web-based Multimodal Information Mining (WMIM) and ISSAGA to effectively exploit the correlations between names and faces. Our experiments on a large number of official public data from Yahoo! News have obtained very positive results.