Research On Technology Of Human-Computer Interaction And Target Perception For Mobile Augmented Assembly Guidance System

The augmented assembly guidance system uses augmented reality technology to superimpose virtual assembly information into real assembly scene for guiding users in product assembly.It can realize the visualization,information and intelligence of the assembly process to improve assembly efficiency and assembly quality.In "Made in China 2025",augmented reality technology is regarded as a key technology in the core information equipment field of intelligent manufacturing,but the technology still has problems in practical application such as too frequent human-computer interaction,and not ideal virtual reality fusion effect.Aiming at the problems that the human-computer interaction methods are not convenient enough and the changes in the posture of real parts affect the registration effect,the research has focused on the technology of picking perception human-computer interaction and target part perception.An mobile application system has been developd based on the research results.The related work is carried out from the following aspects:(1)According to the demands of augmented assembly guidance,the mobile augmented assembly guidance system architecture is established.The system operation mode is built to compared with the traditional guidance mode for analyzing the advantages of the system.Picking perception human-computer interaction technology and target parts perception technology in the system are given.(2)A picking perception human-computer interaction method based on convolutional neural network is designed.HSV skin color model is used to obtain important regions in the image for accurately selecting key features of image data.A lightweight convolutional neural network model is constructed to realize picking perception with less computation,and then to push the relevant assembly guidance information.An expirement has been conduct to verify the feasibility and the convenience of the proposed method.(3)A target parts perception method based on image matching is proposed.Edge orientation is used to weight the pixel value of distance image to improve the calculation accuracy of image matching degree.The differential evolution strategy and the tabu search stategy are used to modify the genetic algorithm,which improves the accuracy and search efficiency of the target parts perception.Experimental results show that the proposed method can realize the perception of target components without using markers and feature points,and has good accuracy and robustness for messy environment.(4)Based on the above theoretical research,an augmented assembly guidance prototype system for the assembly scene is developed.The main functional of the prototype system has been verified through the assembly task of a cabin in an assembly workshop of Aerospace.