Research On Real-time Mobile Augmented Assembly System Based On Lightweight Object Detection Network

Under the influence of China Manufacturing 2025,the development of high-level and high-quality manufacturing is particularly important.As an essential part of manufacturing,assembly has a significant impact on the development of the manufacturing industry.With the trend toward higher levels of automation in assembly,manual assembly at this stage is still an indispensable part of the process and has become the main factor to restrict the development of assembly.Improving the efficiency of manual assembly can give a major impetus to promotion of the overall assembly level and the development of manufacturing operations.This paper proposes a mobile real-time augmented reality-assisted assembly system based on a lightweight object detection network.It innovatively integrates a recognition module of mechanical parts into the augmented assembly system to reduce the operator’s cognitive burden.And in the assembly module the virtual-real fusion is achieved to be more easily understood by users and to improve the efficiency of manual assembly.This paper deeply studies the object detection algorithm of YOLOv3,and proposes an improved YOLOv3——Mobilenet-YOLOv3-Lite,a mobile real-time algorithm for recognition of mechanical parts.For the mechanical parts dataset collected in this article,Mobilenet-YOLOv3-Lite has a slightly lower m AP(mean average precision)than YOLOv3,but has a even faster inference speed than tiny-YOLOv3 designed for mobile devices.By replacing the feature extraction backbone network Darknet-53 with Mobilenet,reducing the number of network channels and simplifying the fusion feature structure,the proposed Mobilenet-YOLOv3-Lite greatly reduces the amount of network parameters and calculations,and improves the inference speed of network.In addition,in order to improve the accuracy of the network,this paper uses k-means clustering algorithm,which uses k-means++ algorithm to initialize the clustering center,for clustering analysis of the mechanical parts dataset and obtaining the appropriate anchor parameters.Finally,the flops of Mobilenet-YOLOv3-Lite network is only 3.69% that of YOLOv3,the parameter amount is only 4.32% that of YOLOv3,and the model file size is only 4.44% that of YOLOv3.Although the m AP is slightly lower than YOLOv3 by 6.87%,i.e.91.26%,its inference speed on Samsung mobile phones is about 10 frames per second,which is 14.65 times that of YOLOv3 and 1.57 times that of tiny-YOLOv3,which meets the real-time requirements.It can accurately predict a single mechanical part,multiple parts of the same type,multiple parts with different types and small-size parts under the conditions of different backgrounds,occlusion,low lighting,etc.,with strong robustness.After studying the traditional virtual-real occlusion handling methods,this paper proposes a model-based occlusion handling method.The virtual models are augmented into the real world scene to obtain the depth information at each point in the real world scene compared with the depth in the virtual world scene.The experimental results show that the real time display of 30 frames per second can be achieved on mobile phones.Compared with the occlusion handling method based on image analysis,it effectively solves the problem of occlusion in the augmented assembly process and the fusion distortion problem due to the high latency of display.Finally,based on the above system,this article develops a mobile augmented assembly application Assembly AR,which uses augmented reality glasses Epson Moverio BT35 E to display video frames,supplemented by a mobile phone as a computing platform,which has the advantages of easy portability,lower price compared with Holo Lens,hands free and convenient operations.Taking the manual assembly of a reducer as an example,the application can run smoothly in real time to guide and assist the operator to complete the assembly process.