Establishment Of Robot Knowledge Base And Action Sequence Reasoning For Complex Tasks

In recent years,the problem of self-care of the elderly and the disabled needs to be solved urgently.The protection of their basic life is the focus of our government and all sectors of society.With the rapid development of robotics and artificial intelligence technology,diversified service robots gradually enter various fields to assist human production and life.Among them,robots with autonomous operation ability have good application prospects in helping the elderly and the disabled because they can complete a variety of dexterous operation tasks independently.However,most home service robots can only complete the tasks set by the program,lack the ability to independently plan complex operation tasks,and can not meet the complex and diversified requirements of home service.To solve this problem,this thesis focuses on the establishment of robot knowledge base and action sequence reasoning for complex tasks.The main research contents are as follows:(1)The description of diversified operation skills in the knowledge base: In order to represent the operation skill knowledge,an operation skill primitive composed of three parts is proposed: Action Semantic text,which is used for the retrieval and matching of actions in the knowledge base;The motion features learned based on the dynamic motion primitives method are used to generate the motion trajectory in the new scene;Action function code is used for the reasoning of action sequence.The parametric description of operation skill primitives lays a foundation for the robot to transform the semantic knowledge of complex tasks into executable motion trajectories.Learned 12 basic operation skills as the action knowledge in the robot knowledge base.(2)The description of object operation information in knowledge base: In order to represent the operable mode of object,object operation code is defined,and the relationship between 16 common objects and actions in knowledge base is recorded;Based on the generated residual convolution neural network,the grasping position information of the objects are learned;The knowledge map of objects orientation relationship is established,and the positional relationships between objects are described in the form of four tuples,which provide a knowledge basis for object search in the reasoning stage.(3)Complex task key information extraction based on language vision multimodal interaction: For task input in natural language form,a task key information extraction method based on language vision multimodal interaction is proposed: Analyze the part of speech and dependency syntax of task instructions,and extract key task information such as object object relationship diagram and verb object phrase sequence;Train Yolo V5 model to detect objects in the current scene,analyze and obtain the initial relationship diagram between objects;Combined with the above language and visual information,the semantic layer task sequence is generated and verified by experiments.(4)Robot action sequence reasoning and execution: In order to transform the task sequence of semantic layer into the operation skill primitive sequence of robot,an action sequence reasoning algorithm based on Deep Q Network is proposed.For each subtask in the task sequence,an action sequence reasoning method with the achievement degree of object target state and relationship as the reward function is designed,which solves the problems that the operated object does not appear in the operation scene,the same semantic verb corresponds to multiple operation skills,the current objects’ states do not meet the action execution conditions and the connection of action trajectory.The feasibility of this work is verified by typical complex operation task experiments on the robot experimental platform(aobo-i5 manipulator and robotiq claw),which enhances the generalization ability of knowledge and improves the task completion rate.