Research On All-day Foreign Object Debris Detection Method Based On Visible Light Camera And Laser Line Scanning Camera

With the development of aviation technology,air transportation has become more and more convenient.As an important area for aircraft take-off and landing,the airport runway is related to the safety of passengers,crew members and the airport.Foreign object debris on the airport runway generally refer to some foreign substances that may damage the aircraft,which pose a huge threat to the aircraft take-off and landing process.Therefore,the detection of foreign object debris on the airport runway is of great significance for the safe operation of the airport and the safety of aircraft take-off and landing.At present,most domestic airports use manual visual inspection to detect foreign object debris on the runway,which is inefficient and easy to miss.Aiming at the task of foreign object debris detection,this paper proposes a method for detecting foreign object debris based on visible light camera and laser line scanning camera,and realizes the all-day automatic detection of foreign object debris.The main work includes the following aspects:(1)For the foreign object debris detection task,this paper uses YOLOv7 as the baseline network and proposes object detection algorithm FOD-YOLO for foreign object debris,which realizes real-time and high-precision detection of small foreign object debris on the airport runway.To solve the problem of poor detection performance of YOLOv7 algorithm for small targets,FOD-YOLO has improved in three aspects.First of all,FOD-YOLO design four detection heads of different scales,which are respectively detected on the feature map after 4 times,8 times,16 times and 32 times downsampling,which improves the multi-scale detection ability of the model.Then,FOD-YOLO proposes a new pooling module SPD-MP,which retains more local information of the feature map during downsampling.Finally,FOD-YOLO uses the Normalized Wasserstein Distance as the bounding box regression loss function,which overcomes the shortcoming of IoU being too sensitive to small targets,and further improves the small target detection performance of the algorithm.In order to verify the effectiveness of FOD-YOLO,this paper design and produce the airport runway foreign object dataset FOD1200.A large number of ablation experiments and comparative experiments with other algorithms are carried out on the dataset FOD-1200,which verify the superiority of the method proposed in this paper.(2)Aiming at the task of detecting foreign object debris at night,this paper proposes a detection method based on multi-channel information fusion by taking advantage of the advantages of 3D laser line scanning camera that can obtain laser intensity information,laser scattering information and depth information without natural light.All-day detection of foreign objects on airport runways First of all,this paper proposes a contour extraction method based on edge extraction and morphological processing for depth images,which solves the problem that the traditional background subtraction method cannot detect contour accurately due to vehicle bumps and background changes,and obtains accurate foreign object debris contour information.Based on the accurate foreign object debris contour,this paper proposes a threechannel information fusion method NMIF combined with the deep attention mechanism,which uses the contour of foreign object debris in the depth image to enhance the laser scattering information,and fuses the depth image,laser intensity image and laser scattering image as three channels into false color image,which realizes the fusion of multi-channel features and improves the visual effect at the same time,solves the problem of lack of single-channel image information and high misrecognition rate.Finally,this paper uses FOD-YOLO to detect false color images,and finally completes the task of foreign object debris detection at night.In order to verify the effectiveness of the proposed method,this paper uses a 3D laser line scanning camera to produce airport runway foreign object debris datasets FOD-1466 and FOD-470,and experiments on the dataset verify the effectiveness of NMIF and FOD-YOLO.(3)In view of the functional requirements of foreign object debris detection system,such as detection and alarm,this paper built a vehicle-mounted foreign object debris detection system based on the inspection vehicle.First of all,this paper integrates visible light camera and 3D laser line scanning camera and other equipment on the existing inspection vehicle,and selects and installs related hardware to realize all-day runway surface information collection.Then the FOD detection algorithm is deployed in the on-board computer to realize the real-time detection function of runway surface information.According to the working characteristics of visible light camera and 3D laser line scanning camera,the workflow and personnel operation method of the detection system under different lighting conditions are studied and determined.At the end of this paper,the system performance verification is carried out on the simulated airport runway and the real airport runway,which verify the effectiveness and practicability of the detection system.