The Reading Cognitive States Recognition And Assistant System Based On Smart Glasses

The advancement of science and technology and the rapid development of artificial intelligence technology have made it possible for human to explore human brain.Scientific research on the brain is the biggest challenge for human beings which has always been at the forefront of science.Recognition of reading cognitive states is an important part of brain science research which has been included in research programs in many countries.At present,existing methods of recognizing reading cognitive states can be mainly divided into three categories: smart glasses-based,electroencephalography-based and functional magnetic resonance imaging-based.The cognitive state recognition method based on smart glasses-based can only recognize coarser granular reading cognitive states;the cognitive state recognition method based on electroencephalograms requires the user to place many brain wave sensors on the brain,which is very inconvenient to wear;Functional magnetic resonance imaging recognition method for reading cognitive state is achieved by using expensive functional magnetic resonance imaging equipment,which is costly and unfavorable for users.Secondly,with the development of the information age,people's reading needs are increasing,and reading content is becoming more and more abundant.Increasing reading efficiency has become one of the most urgent needs for everyone.Therefore,how to design a reasonable,user-friendly reading assistant system is also one of the focuses of researchers.In order to solve these problems,this paper designs a reading cognitive states recognition and assistance system based on commercial smart glasses.The main research work of this article includes the following five parts:(1)In order to obtain the user's eye tracking data in real time,we modified part of the code of Pupil smart glasses software.In order to remove the outliers and saccades in the user's reading trajectory,we designed an outlier and saccade removal algorithm based on k-means.(2)We extract user's reading characteristics in order to find the location of the user's reading interest or difficulty in reading text.Then,we use the OCR technique to recognize the reading text.After that,we use the spelling correction techniques to correct the misrecognized text.We extract features from user's eye-tracking data,and then using a classifier to classify the user's reading cognitive states.(3)We design a reading assistant app based on i OS system.This app can provide users with different reading auxiliary functions according to users' reading cognitive states and the recognized reading content.Experiments show that our system has an accuracy rate of 84% for reading cognitive states recognition of users;after removing difficulties that do not affect reading results,the system's recall ratio of reading content recognition is 88.9%.Finally,we designed a reading assistant app based on the i OS system.The reading assistant app provided different auxiliary functions for users according to different reading cognitive states or reading contents.Experiments show that the reading assistant system can improve users' reading efficiency and experience.