Multi-Scale Object Detection And Scene Reasoning For Home Environment

In recent years,with the improvement of people’s living standards and the intensification of population aging,service robots have gradually entered the home scene to provide users with a variety of intelligent services.Home object detection and scene recognition are the foundation for service robots to complete various service tasks.However,the complex and diverse home layouts and home objects in different forms pose challenges for the application of robots in the home.The object detection method based on deep learning provides a feasible solution for the actual deployment of robots,but the lack of large-scale home object detection dataset seriously limits the practical application of the detection model.Meanwhile,robots efficiently perform service tasks,which not only require obtaining data about objects in view,but also have a more urgent need for indoor scene perception.In response to the above issues,this article conducts research on the construction of home object dataset,multi-scale object detection and home scene reasoning.The main work as follows:(1)In order to solve the problem of insufficient detection datasets in current home scenarios and the lack of contextual information about scenes and objects,a machine vision based method for constructing home object dataset is proposed,which improves the efficiency of dataset construction.Firstly,the home scene pictures and home object pictures are obtained by network crawling,manual screening and field shooting.Secondly,the object instance pictures and mask pictures are quickly extracted by saliency detection,and the scene parsing files corresponding to the home scene pictures are obtained by using semantic segmentation.Then the matching rules of the scene areas and the home objects are designed in combination with the intelligent space ontology knowledge base.Finally,a home object dataset combined with contextual knowledge is automatically constructed.In addition,the home object dataset is further expanded by manual labeling and screening of public object detection datasets,which provide rich data for model testing and practical robot applications.(2)Aiming at problems such as the wide variety of objects in the home environment,the large scale comparison between the operating objects and the scene objects,and the change of the viewing angle of the target objects during the grasping operation,a multi-scale object detection model for home environment is constructed,which improves the actual detection performance of robots.Based on the YOLOv5 model widely used in engineering projects,firstly,the attention mechanism is combined to make the model focus on the task-related areas.Secondly,the context information module is added to enhance the learning of the target features by the detection model.And then the cross-scale feature connection is introduced to aggregate shallow layer positioning information and deep semantic information to achieve the fusion of multi-scale features.Finally,experiments are carried out on self-constructed dataset and public dataset to verify the effectiveness of different improvement methods,and compared with related detection algorithms to verify the advanced nature of the proposed method.The experimental results show that the model constructed in this paper can improve detection accuracy while maintaining real-time detection speed,meeting the detection needs of robots for objects of different scales.(3)Aiming at the problem of insufficient information of scene categories and target objects association,a home scene reasoning method based on graph convolutional network is proposed by making full use of object detection information,which the robot’s environmental perception ability.Firstly,use the ResNet network to extract the global feature information of the scene pictures.Secondly,combine the object detection results to construct the object co-occurrence relationship graphs,apply the graph convolution network to integrate the information between nodes,and reason to obtain the object feature vector.Then,an information fusion method based on Dempster-Shafer evidence theory and a feature fusion module based on attention mechanism are designed to integrate the features of different information sources to achieve the final scene classification.Finally,the ablation experiment are carried out on the public dataset and compared with other methods.The experimental results show that the proposed method achieves good recognition results for different types of family scenes.(4)A home object detection and scene perception system is built to verify the effectiveness of the proposed method in this paper.Firstly,the system architecture and components are introduced.Then,the scene is set up in the laboratory to verify the detection effect of the detection model in complex situations such as object occlusion and perspective change,and the recognition effect of the scene reasoning method in home scenes.Finally,a test is carried out in the smart home space to verify the actual performance of the method in this paper.The experimental results show that the multi-scale home object detection model proposed in this paper can meet the detection needs of the robot platform in the home environment,and the scene reasoning method can provide the necessary information support for the robot to perform tasks efficiently.