Enabling Perception Intelligence In Home Service Robots

As one of the important research fields of intelligent robot technologies,home service robots provide a good solution to intelligent home-based elderly care,and have received extensive attention due to their much-needed functions such as emotional companionship,daily communication,and health monitoring.However,the limitations of the key knowledge(i.e.,the perception knowledge)required by service robots for their own perception intelligence(i.e.,the ability to perceive the home environment and carry out perception-related tasks)affect their service quality,and restrict the scope and effect of their applications.Therefore,it is necessary to use various sources to develop the perception intelligence of home service robots.To this end,this dissertation introduces three kinds of knowledge sources(i.e.,the heterogeneous distributed multi-sensor fusion,the user’s assistance,and the cloud’s assistance)and develops the perception intelligence of home service robots with the help of these knowledge sources.The main works and contributions of this dissertation are as follows:(1)This dissertation proposes a strategy to develop distributed multi-sensor fusion-based perception intelligence of home service robots.Aiming at the representative case that robots help users find misplaced items,this dissertation develops a robot-based misplaced item finding system that integrates two kinds of heterogeneous distributed sensors in a simulated home environment.This dissertation also proposes a search path planning method that uses the contextual information of the user’s historical trajectory provided by the heterogeneous distributed multisensor fusion.This method leverages the correlation between the location of the misplaced item and the user’s movement.The advantages of the proposed search path planning method compared to traditional methods are verified with multiple experiments.(2)This dissertation proposes a strategy that takes advantage of users’ assistance to help home service robots adapt to users’ initial home environments.Inspired by the learning styles by which humans learn new knowledge,this dissertation proposes the ACSSI(All Class Sampling at Short Interval)learning style for the robot,and gives a probabilistic analysis of the deep neural network training during the continual learning process.Three kinds of error sources that affect the performance of the learning style are determined based on this analysis,i.e.,the hypothesis error,the cumulative approximation error,and the approaching target error.Qualitative comparisons of the performance of the ACSSI,the Batch,and the SSSI(Selective Sampling at Short Interval)learning styles are provided.Based on the comparative experiments with the traditional learning style and the performance-availability trade-off,this dissertation determines that the most suitable learning style for home service robot applications is the ACSSI learning style.(3)This dissertation proposes a difficulty-aware active learning method for home service robots to select data that is not only useful but also easy for elderly users to label.A series of image properties related to annotation difficulty are proposed based on existing medical researches in elderly vision,and a user study is conducted to determine the ground truth of annotation difficulty of images for older adults.Based on that,a robust annotation difficulty predictor is developed using the results of the user study,and the difficulty prediction of an image is combined with its learning value to form the query data selection metric of the proposed active learning method.The advantages of this method are verified through an ablation study.(4)This dissertation proposes cloud-assisted robot perception intelligence.With the help of the cloud platform that contains knowledge learned by other robots,this dissertation designs a mechanism that realizes knowledge transfer among robots via the cloud.Based on equivalent network analysis,this dissertation proposes the BPFusion model fusion algorithm to facilitate the knowledge transfer from the cloud to the robot,and the variant of the algorithm is given when the cloud model and the robot’s local model are heterogeneous.Finally,the advantages of the proposed algorithm compared with other advanced model fusion methods are verified by simulation experiments in both homogeneous and heterogeneous scenarios.In general,this dissertation focuses on the development of the perception intelligence of home service robots and leverages multiple perception knowledge sources to achieve this goal.Both theoretical and practical results are obtained and guidelines for developing the perception intelligence of home service robots are provided in this dissertation.