Research On Human-Machine Interaction Recognition And Digital Twin Modeling For Complex Machining Equipment

With the proposal of Industry 5.0 and the rapid development of emerging technologies such as artificial intelligence,virtual reality and the Internet of Things,human-oriented intelligence has become a research hotspot for the deep integration of modern information technology and advanced manufacturing,with more emphasis on human-oriented intelligent manufacturing.Therefore,it is of great significance to study the digitalization,networking and intelligent upgrading of machining equipment empowered by human-machine interaction technology in complex processing environments.This thesis takes complex machining equipment as the research object and focuses on the recognition of static and dynamic gestures in processing operations based on the aggregation of human-machine interaction functions and the definition of operating gestures.As a result,the modeling and interactive control of the digital twin of complex machining equipment were realized.Firstly,to address the issues of human-machine interaction function aggregation and ambiguity in the digital twin of complex machining equipment,this study adopts a modular approach for analysis.The hierarchical clustering method is used to aggregate the humanmachine interaction functions,and the analytic hierarchy process is used to determine the relevant standard weights,thus achieving systematic clustering of the interactive functions of the digital twin of complex machining equipment.Subsequently,based on the gesture definition criteria,the corresponding operational gestures are designed for the interactive functions to complete the association mapping between the interactive functions and operational gestures.Secondly,to address the low accuracy problem in static gesture recognition in practical environments,this study proposes a static gesture recognition method based on the YOLOv5 algorithm.Depth data and color data are collected using the Kinect sensor,and after registration and segmentation,a static gesture dataset with complex backgrounds removed is obtained to avoid background interference and reduce computational complexity.Furthermore,an attention mechanism is introduced to optimize the YOLOv5 algorithm,enhancing the utilization of feature information.Experimental results demonstrate that the proposed method can achieve good detection results and recognize static gestures in processing operations.Thirdly,considering the continuity,diversity,and multidimensionality of dynamic gestures in human-machine interaction,this study proposes a multimodal data fusion strategy for dynamic gesture recognition.The C3 D network model is used to extract information from both spatial and temporal dimensions in videos,and features are extracted from both depth images and color images.The recognition results from both modalities are then fused at the decision layer and the activation function is optimized.Experimental results show that the proposed method achieves high accuracy and robustness in recognizing dynamic gestures in processing operations.Finally,based on the proposed human-machine interaction function clustering and gesture recognition methods,a prototype system is developed using Kinect 2.0 and Unity 3D software,including the establishment of a digital twin model of complex machining equipment and gesture interaction implementation.The feasibility and rationality of the proposed methods and model interaction are validated through a case study.