Research On Semantic Representation Of Eye Movement Behavior And Application In Image Retrieval

With the development of internet and information technologies,the e-commerce industry is increasingly developing.It promotes the vigorous development of consumers' online shopping,which brings e-commerce enterprise unprecedented opportunities and challenges.E-commerce provides consumers a huge amount of online commodity options,meanwhile,it also brings them enormous cognitive load.In order to reduce the blindness of their decision-making in the ocean of goods,many e-commerce websites offer consumers retrieval function based on keywords.Merchandise pictures play an important role in consumers' purchasing behavior during online shopping,because image information has such traits as figure,intuition,understandability,and enormous information,and it is the most important and the most effective way of information acquisition and communication.Traditional content-based image retrieval is compute and comparison of the low-level image features,namely "visual similarity".Actually,consumers' understanding on merchandise pictures are not only based on visual similarity,but also semantic similarity,and we only use image processing algorithm to extract the low-level image features not to express people's understanding image semantics abundantly.People's understanding image combines experience,knowledge,and personal preference accumulated in our daily life,and they understand the image through cognitive thinking model of semantic perspective.This causes easily causes "semantic gap" between human understanding to image and the low-level image features.To overcome "semantic gap"of commodity image retrieval,this dissertation uses eye tracking technique to record individual eye movement behavior during browsing images,which discusses on the inherent relation among eye movement behavior,choice behavior and visual attention computational model as principle of image semantics to gain human understanding,for further study on semantic representation of eye movement behavior and image retrieval,the main contents are as follows:1.Study on inherent relation among eye movement behavior,choice behavior visual,and attention computational model.This dissertation mainly discusses whether eye movement behavior and choice behavior are influenced by both bottom-up factors and up-down factors.Comparing with visual attention computational model,we examine if eye movements tend to reflect better with individual choice behavior.Therefore,we design and complete the related eye movement experiments and click experiments.Finally,the results of experiments indicate that eye movement behavior and choice behavior are influenced by both bottom-up factors and up-down factors.Besides,comparing with visual attention computational model,eye movement behavior appears to highly correlate with choice behavior,and accurately carries the inclusive semantics of individual choice behavior.2.Based on perceptual properties of the human visual system,this dissertation proposes a method of semantic representation based on eye movement behavior.According to the degree of attention on eye fixation points as the same as the strength of eye-interface connection,this method use the kernel-based fuzzy clustering algorithm to calculate the degree of eye fixation points belonging to different components,and then combine the degree of attention and membership on eye fixation points to present two computational models of eye movement behavior:KFCM-A model and KFCM-U model.Therefore,the related eye-movement experiments are developed to verify the accuracy of the eye movement behavior computational model.The results of experiments indicate that the proposed models can describe subjects' attention allocation strategies explicitly.Besides,the proposed method effectively avoids using image segmentation algorithms which are accurately difficult to divide object area,and not describes individual choice semantic of visual attention allocation.3.Concerning the unsupervised image retrieval method is difficult to express users'individualized requirement,this dissertation proposes an approach of image re-ranking based on semantics of eye movement behavior.This dissertation begins by taking eye movements as a kind of implicit rating information to explain high-level semantic concept of individual needs.Next,this approach extracts the low-level visual features of merchandise pictures to describe their feature attribute.Finally,we use ranking support vector machine algorithm to bridge the gap between the feature attribute of merchandises and the high-level semantics.Therefore,the related eye-movement experiments are developed to verify the validity of our proposed approach.The results of experiments indicate that the proposed method uses eye movements that individual browses a few merchandise pictures to reorder all merchandise pictures,and merchandises preferred by consumers are the top few ones of the retrieval results.4.According to visual search process of real human,this dissertation proposes an approach of image classification based on semantics of eye movement behavior.This dissertation mainly studies that the different areas of image are given different weights of visual attention by analyzing the effect of human fixation,and extracting the different areas of image low-level features one by one,then proposes two methods to construct weighed visual feature based on eye movements,and at last,compare the accuracy of different image classification methods.Therefore,the related eye-movement experiments are developed to verify the validity of our proposed approach.The results of experiments indicate that the proposed method can served as a new solution to overcome "semantic gap",and well improve the accuracy of retrieval results.