Research And Application On Deep Learning Based Reinspection Methods Of Automated Optical Inspection

Medical electronic devicecs such as Computed Tomography(CT)are indispensable tools in the field of medical diagnosis.Due to the high precision of the equipment,the production requirements and quality control of the internal circuit are very strict.Automated Optical Inspection(AOI)is an important technique in the quality control of the Printed Circuit Board Assembly(PCBA),which is often applied in the assembly lines of Surface Mount Technology(SMT).The AOI systems are usually installed in the key positions of the assembly lines to monitor the product quality through inspecting the component defects caused by mount errors or soldering failures.Current AOI mainly uses an optical CCD camera to acquire the images of each component on the panels and find defects via canonical image processing methods.Due to complexity of detection scenarios and limitations of detection approaches,the false alarm rates of AOI remain high.In term of this,manual reinspections have often been taken on those components with presumed defects afterwards,which leads to extra labor costs and decreases detection efficiency.In collaboration with one AOI system producer,a deep learning based reinspection method of AOI has been researched.The reinspection stage is often involved in inspecting the components with presumed defects(No Good,NG)by the AOIs and the researches are going to reduce the false alarm rate of overall defect detection.However,due to the complexity of diverse components and defects,it is challenging to be accomplished even for AI based approaches.In order to reduce the degree of coupling of different detection contents and highlight the key points of the subject research,in this thesis,the proposed methods focus on the defects of four most common catergories: Chip components,integrated circuits(ICs),Plug-in components and Surface Mount Device(SMD)transistors,and the designated defects are also constrained in the ones such as components missing,wrong components and common soldering defects etc.Here,NG and OK data are referred to normal and abnormal component data respectively.This specific contents of our study comprise four aspects as the following:First of all,all defect detection processes are divided into three steps: 1)locating the component bodies and soldering pads,2)detecting the body defects and 3)detecting the soldering defects.Each step uses one or more convolution neural network(CNN).For feasilitating feature extraction and improving inference efficiency,after comparing with several networks(VGG,Google Net,Res Net,Dense Net,Sequeece Net)with respect to their parameters,floating point of operations(FLOPs)and performance on Image Net etc,the Mobile Net is taken as the backbone network.And then,while comparing the SSD and YOLO,the SSD is chosen as the main detection network for bodies and pads of components of all 4 catorgories.Based on the backbone network Mobile Net and the detection network SSD,the further researches have been conducted on the defect detection over all 4 catorgories of components.For the components in Chip category,there are two detection methods presented: 1)the self-supervised learning method for solder joint defects and 2)a bilinear convolutional network for body color defects.In detection of solder joint defects based on self-supervised learning,this thesis aims at the problem that real NG data is difficult to collect and proposes three specific data augmentations methods(electrode shifting,random missing and electrode morphological transformations)to generate synthetic NG data from real OK data.Synthetic NG data is helpful to improve Mobille Net V2 in case of less real NG data.Mobille Net V2 has achieved the recall rates of 99.78% and 100% on real OK and NG data respectively.The experimental results showed that it outperformed other semi-supervised defect detection networks.In detection of body color defects,a bilinear convolution network was applied to distinguish the body colors of various components in Chip category(i.e.resistors,capacitors and inductors).In this way,the color classification accuracy of capacitors and inductors both achieved up to 99.98% and the accuracy of color classification of resistance resistor achieved up to98.26%.To all components in 3 other categories like ICs,Plug-in components and SMD transistors,as the defects are relatively simple,a SSD network combined with some canonical algorithms is employed for processing.This detection processes comprise three aspects: 1)detection of components missing,2)detection of Plug-in components with wrong direction and 3)detection of ICs with short circuit.In the detection of components missing,the proposed method borrows the confidence threshold of SSD detection boxes to distinguish the missing components from normal data for all components of 3 categories.In the detection of Plug-in components with wrong directions,all four directions of components are regarded as four types and inputed to the SSD network to classify,as a result,an accuracy of 99.78% can be obtained.In detection of ICs with short circuit,this proposed method employs CRAFT network to locate solder joints and further segements each solder via watershed algorithm,to determine whether there exists a short circuit inside.The average accuracy of this method reached 99.60% and the recall rate of short-circuit data was 100%.Finally,all detection processes for components of 4 categories have been integrated into one model,and then,the designated model is translated from python to C++ and encapsulated a dll library to deploy.Through the actual test,the deployed network did not incur significant accuracy loss,the average operating efficiency is 49.17 ms per image,and the total memory consumption of all models is 53.03 M,which meets the needs of practical applications.Currently,the part of the propose model in detection of Chip components has been used in actual production.The straight-through rate of production line increased by 44%.The detection network for all other three are still in the testing stage and hope to be put into production once testing finished.