Semantic Information Acquisition Mechanism For Family Objects Based On Deep Learning And Ontology Technology

In the family environment,the tasks performed by the service robot,such as grabbing objects and searching for objects,are closely related to the semantic information like the category,position,and weight of the target object.The semantic information of objects is not only the basis for robots to complete service tasks,but also the key factor to improve their intelligent service level.At present,robots usually use the object detection models based on deep learning to obtain the category and position of objects,but most of them are difficult to apply to the family scene directly.Meanwhile it is not enough for robots to recognize objects to obtain their categories and positions with detection models.In addition,it is difficult for methods based on deep learning to obtain the semantic information such as the weight and volume of the object.At the same time,further research is also needed on how to represent and store the semantic information into unified efficiently.In response to the above problems,it carries out a deep research on the acquisition of object semantic information in this paper.Firstly,the semantic information such as the category,position,the positional relationship with other objects and attributes of the target object is obtained based on the deep learning.Secondly,the prior semantic information of the target object is obtained from the Internet using a method combining web crawler and feature matching.Finally,the acquired semantic information of the target object is represented and stored in an integrated manner based on ontology technology.The task of acquiring family object semantic information is completed.The contents and innovation of this paper are as follows:(1)Aiming at the limited accuracy and types of semantic information of family objects acquired by the object detection models,a method for acquiring semantic information of objects based on deep learning in the family environment is proposed.A family object detection model is constructed based on YOLOv5 and attention mechanism,and the model is trained to improve the detection accuracy of household objects using the family object detection dataset to better adapt to the home scene.The positional relationship detection for family object is realized based on Neural Motifs,so that the robot can focus on the positional relationship between the target object and the other objects.Based on the multi-label classification task,an object attribute detection model is constructed to detect the color,shape and other attributes of the object.Finally,the three detection models are tested through experiments.The experimental results show that the three models can be well adapted to the task of extracting semantic information of family objects.(2)In order to obtain the weight and volume of family objects,which are difficult to obtain through deep learning and make full use of the prior semantic information of objects in the Internet that have been tested by humans,a method based on web crawler and feature matching is proposed to obtain the prior semantic information of objects.A web crawler algorithm is designed based on the WebDirver module.It uses keywords to search online and crawl products similar to the target object,extracts the product feature vector to match the target object,selects the most similar product to the target object,and automatically crawls and filters its information as prior semantic information of the target object.In order to make the extracted feature vectors better represent family objects,a feature extraction model is constructed based on residual network,and then the model is fine-tuned using a small self-made family object classification dataset,and finally the fine-tuned model is optimized by using the structure of the twin network.The feature extraction model is evaluated on a self-made family object test dataset,and the average cosine similarity of the feature vectors for the same objects and different objects is 0.926 and 0.754 respectively.Compared with other models,the experimental results show that our model has advantages on the ability of feature representation.(3)For the problem of the need for efficient and unified representation and storage of object semantic information,an object semantic knowledge database is constructed based on ontology technology,which realizes the integrated representation and storage of object semantic information,and facilitates the utilization of object semantic information by robots.At the same time,an object feature vector library is constructed,and the feature vector is used as a guide,so that the semantic information of objects in the knowledge database can be obtained quickly.The interaction of the knowledge database is realized based on the Owlready2 module,and the semantic information storage mechanism and acquisition mechanism are designed,which can automatically complete the storage and acquisition of objects semantic information.Finally,the feasibility of the acquisition method and storage mechanism of object semantic information is verified by experiments.