Research On Workpiece Defect Detection And Behavior Prediction Technology In Human-Robot Collaborative Assembly

Human-robot collaboration is the most effective solution in the intelligent manufacturing process at this stage.The fully automated manufacturing system is exceedingly difficult to achieve when confronted with the unstructured workspace and various flexible tasks.Meanwhile,customized high-value-added products with shorter lifecycles become the core objectives of intelligent manufacturing.The ensuing problems such as multimodal information fusion,human intention inference,human-robot collaborative interaction,and motion planning are urgently needed to be resolved.This thesis conducts a comprehensive study on workpiece quality detection,material transportation,human fault detection,and active human-robot collaborative assembly within the assembly scenario.An active collaborative system for the intelligent assembly process was built.This approach can integrate robots into the intelligent manufacturing environment and provide theoretical guidance for future human-robot collaboration applications in the assembly manufacturing field.To tackle the insufficient training data problem in smart manufacturing scenarios,a subspace-based defect detection algorithm is established to obtain high detection accuracy.The different space-based discriminative criterion functions are constructed to manage the irreversible intra-class scatter matrix.The algorithm achieves high defect detection accuracy under the condition of the small database since it expands the feature extraction space and extracts more hidden nonlinear feature information.After the inspection process is finished,a hybrid machine learning model for material vehicle trajectory inference under unfamiliar scenes is proposed.The hybrid machine learning model combines the computer vision method with the machine learning methods to solve the problem of insufficient samples during trajectory planning.The relationship between training scenes and unfamiliar scenes is derived,and the trajectory inference and control model is studied.Finally,the material vehicle in unfamiliar scenes can automatically realize the trajectory inference and control process under the small training dataset.Video representation is an important prerequisite for human-robot collaboration.It serves as the bridge for knowledge transfer between the human and the robot.Based on the video representation,a behavior analysis model is established.By analyzing the highdimensional spatio-temporal information in the video data,a network configuration fusing the recurrent neural network and encoder is proposed.The model can obtain spatiotemporal features more effectively for further analysis and abstraction.To extract key video representations in video data samples without the label,an unsupervised training method is implemented.Two types of downstream tasks are designed to verify the performance of the proposed model.The applications of video representation learning in subsequent tasks are explored.The human fault in assembly manufacturing occurs randomly in diverse forms.It’s very difficult,sometimes even impossible to detect it in real-time.We proposed an action prediction and human fault detection model based on spatio-temporal feature learning to monitor the human assembly process.The model is trained based on passive learning from demonstration.The end-to-end learning method is tuned to achieve knowledge transfer in complex unstructured manufacturing scenarios without predefined assignments.In order to simultaneously extract single-frame spatial features and long inter-frame temporal features,the Conv LSTM module is embedded in the prediction model to deconstruct the complex temporal information.The proposed model improves the decoupling and learning performance for long video data.The similarity between the predicted action sequences and the testing action sequences is calculated to achieve automatic monitoring of human fault without data labels.A scoring function is designed for real assembly scenarios.The experiment shows that the proposed model can achieve an accuracy of 97% for human fault detection.To improve collaboration efficiency,an active human-robot collaboration model framework is established based on human intention inference.To handle the loss of spatio-temporal information during long sequence learning,the coupling link between input and hidden states in the Conv LSTM model is enhanced.A prediction model based on state enhancement is constructed,and the performance of intention inference and prediction is investigated.A tracking-based intention inference method is built to realize the recognition of human intentions.A verification platform of active human-robot collaboration based on intention inference is constructed to verify the contribution of active collaboration.In terms of assembly efficiency,equipment utilization,and collaboration comfort,the active human-robot collaboration has better performance than the traditional method.A passive collaboration scheme based on the voice-commanded method is designed,and the differences of two collaboration schemes are compared and analyzed.The experimental results show that the proposed model improves the naturalness of collaborative assembly and achieves safe and efficient assembly manufacturing.