Research On Multimodal Natural Interaction Technology Using Projection To Indicate Wall Features

Indoor wall plastering projects play a crucial role in determining wall quality.Traditional acceptance methods typically evaluate the quality of plastering projects through chalk sketches and paper reports,which are time-consuming,labor-intensive,and not sufficiently intuitive.These methods fail to display the exact location of wall defects,making it difficult to reproduce the construction scene and identify the specific causes of problems.This thesis focuses on researching a more convenient and intuitive way to display wall defect locations and designs and implements a device that uses a projector to show wall feature positions and enables natural interaction through various methods to display additional information about a particular feature and the replication of specific construction scenarios.The main contents of this thesis are as follows:1.This thesis employs transfer learning on a collected specific-identity personnel face dataset based on the YOLOv6 algorithm.The final average accuracy for the dataset’s recognition is 95.1%,achieving accurate identification of specific operators and avoiding interference from unrelated personnel.2.This thesis presents an improvement on the YOLOv6 algorithm through structural pruning,making it more lightweight and suitable for deployment on low-performance devices while maintaining smooth operation.The YOLOv6 algorithm was tested against the Hagrid static gesture recognition dataset,with human and facial target boxes being re-annotated.A comparison was made between the performance of the YOLOv6 algorithm and the improved algorithm proposed in this article.The results showed that the improved algorithm was more easily convergent,with an accuracy of 82.8%,and was capable of accurately recognizing three types of target boxes,including human,facial,and hand.3.This thesis proposes an offline recognition method for dynamic gestures based on the Res Ne Xt101 and Mobile Net V2 networks.The standard convolution in the Res Ne Xt101 network is replaced with a depthwise separable structure from the Mobile Net V2 network to make the model more lightweight.Additionally,the HSwish activation function is used to replace the Re LU activation function,improving the accuracy of the model’s classification.Experimental results show that the proposed algorithm greatly improves the actual running speed while maintaining detection accuracy.The Paddle Speech framework’s Transformer algorithm is also used to implement speech recognition tasks,demonstrating its high recognition accuracy in speech interaction commands.4.This thesis designs and implements a natural interaction system for projecting wall feature indicators.A tracking registration algorithm for calculating projection coordinates based on wall feature point world coordinates,face detection,static and dynamic gesture recognition,and voice recognition functions is integrated into the system.Experiments have shown that this system significantly boosts display and interaction capabilities in the realm of building acceptance.