Precise Geometric Correction And Robust Mosaic In Airborne Photoelectric Imaging System

Airborne photoelectric imaging system is a kind of optoelectronic imaging and measuring device with high precision, which is used to implement air reconnaissance, target acquisition, tracking, positioning, and other functions. The system is usually equipped with representative payloads such as the zoom visible light cameras, which can make the scenery’s imaging magnification change continuously through the focal length’s continuous change. It can search the targets in a wide field and distinguish the targets in a small view field very well, and achieve the targets’ orientation. It is widely employed in aerial reconnaissance, combat terrorism, safeguard stability, border patrol, disaster early warning, emergency response, atmospheric detection, urban planning and many other fields.However, the field of view of a single camera is limited at present. The image resolution and the field of view are two inter-constraint indicators. To satisfy the requirements of high resolution and large field in the aerial reconnaissance tasks at the same time, it need to implement the mosaic of many aerial images of the same scene. In addition, due to the design, manufacture, alignment and environmental control factor of the aerial zoom optical imaging system itself, the zoom cameras have a severe nonlinear distortion in a short focal length with wide field, and the magnitude of the lens’ distortion changes over the focal length. At the same time, the aerial cameras photograph the ground at a certain oblique angle because of the changing of the aircraft’s attitude and the camera’s optic axis pointing direction, and the oblique imaging mode causes the trapezoidal distortion. The trapezoidal distortion and zoom lens distortion are added together to produce a more complex geometric distortion, which directly affects the geometry precision of imaging system and the subsequent image mosaic quality. To improve the precision of such quantitative analysis task as target detection, location, and pattern matching, and the accuracy of real-time image stitching, the distortion must be corrected. In addition, the airborne photoelectric imaging equipments’ operating range is generally in thousands of meters to dozens of kilometers, so they are easily influenced by atmospheric turbulence, motion blur, defocus blur, mechanical vibration and fog environmental factors. Hence, the aerial images acquired by the airborne photoelectric equipments are fuzzy, with low contrast, and exist some brightness and color differences in different views. It caused great difficulties in image registration and mosaic. Therefore, this paper studies the scientific and technical problems in the image precise geometric correction, the target positioning and measurement, and the image registration and robust mosaic of airborne zoom optical imaging system, the purpose is to output a wide field and high resolution composite image with geographic coordinates information, high geometric accuracy and good visual effects. The main work and research results of the thesis are as follows:(1)Concerned with the nonlinear distortion problems varying with the focal length of aerial optical zoom lenses, a rapid geometric correction algorithm based on the division distortion model and the same geometric type principle between straight lines and its ideal projection image is presented. The algorithm adopts the method combining the offline calibration on the ground with the online correction on board. The division correction model is constructed to correct the radial nonlinear distortion of plane template images. According to the perspective projection invariance of collinear points in template images, the HOUGH transform method is used to detect the line edges in the corrected image, and the distances from the edge points to the corresponding ideal lines are calculated to construct the linearity voting function as a distortion measure, and a linear functional using the distortion parameters as the optimal solution is established. Then, an optimization searching method of variant step length is adopted to estimate the optimal distortion coefficients and the distortion center coordinates of the zoom lenses on the conditions of a few discrete focal lengths. The variation laws of the distortion parameters with the focal lengths are discussed by experiments finding. Then, the corresponding empirical formulas between the distortion parameters and the focal lengths are established, or the lookup table(LUT) of distortion parameters is constructed. In the actual flight test, the distortion parameters corresponding to the actual measured focal length value are calculated based on the empirical formulas or obtained through the LUT method, and the lens distortion in the real scenes images are corrected using these parameters. This correction method can significantly improve the correction accuracy, the processing speed and the degree of automation in engineering applications.(2)For geometric distortion caused by high altitude and large quint imaging mode of a zoom aerial camera equipped with array CCD(Charge Coupled Device), an automatic simultaneously correction method for both the squint trapezoidal distortion and the nonlinear distortion of the zoom lens is proposed. The above calibration method is used to obtain the empirical formulas between the distortion coefficients, the distortion center coordinates and the focal lengths of the zoom lenses, or the LUT of distortion parameters is outputed. By introducing the position and attitude of the airplane and the pointing orientation of the camera’s optical axis, a comprehensive correction model of oblique trapezoidal distortion and lens’ nonlinear distortion is established based on the projection transform method and the division distortion model. The quaternion method is introduced to describe the transformation relationship between the relevant coordinate systems in the correction model, and the projection transformation between the camera frame and the mapping frame is constructed. Then the oblique aerial images specified in the camera frame are georectified into the mapping frame. The orthorectified images eliminated the lens distortion and squint trapezoidal distortion are obtained by resampling the pixels’ intensity after coordinate transformation. The algorithm reduces the number of processing times of the pixel data in the image, and can protect the rectifying image quality at the largest degree. It can considerably improve the rectifying accuracy, the PSNR(Peak Signal-to-Noise Ratio) and correction speed, which can satisfy the requirements of the follow-up image mosaicing, target positioning and tracking.(3)To provide the geographical coordinates information of the targets in the stitching image, and determine the roughly overlapping region of adjacent images and the mosaic orders according to the vertex geographic coordinates of each image to register the aerial sequence images preliminary and improve the precision and speed of image mosaic, a multi-target self-determination orientation algorithm based on the line-of-sight vectors in the airborne photoelectric imaging system is proposed. Based on image forming principles of the single-plane array charge coupled device sensors, the line-of-sight vectors of the targets are constructed, and the view angles of pixel between the main target at image center and other targets are calculated. Combining the azimuth angle, elevation angle and the distance between the main target and the platform, the distance and angle relations between other targets and the platform are calculated. Based on the position and attitude information of aircraft obtained by the GPS positioning technology and aviation attitude measurement technology, the homogeneous coordinate transformation method is applied to calculate the geodetic coordinates of multiple targets in a single image. For the orientation error caused by the lens distortion, a method based on the lens’ s distortion ratio is presented to correct it. The lens distortion correction algorithm and RLS(Recursive Least Squares) filtering algorithm are presented to improve the target orientation accuracy even more. The multi-target orientation algorithm has high precision and efficiency and is convenient for engineering application. The geographic information of the targets and the the mosaic image are fusioned together to generate a wild field and high resolution synthetic images with geographic coordinates, which is important for military application.(4)Images acquired by the airborne photoelectric equipment in foggy weather are usually blurred, low contrast, and existing certain color and brightness differences. To this problem, a feature points detecting algorithm based on the improved difference operator of Mexican-hat wavelet function is presented. It has high positioning accuracy of feature points, and high duplicate detection rate in different images. The algorithm is invariant to scale, rotation, translation, and blur, and robust to illumination and noise. Then, the feature description algorithms based on moment invariants are researched deeply. The geometric moments in the neighborhood of feature points are used to construct the affine invariants, and the blur invariants are constructed based on the convolution invariance theory. The affine invariants are fused with the blur invariants to obtain the combined blur and affine invariants(CBAIs). The affine invariants are also constructed based on the relevant principle of graph theory, and are fused with the blur invariants to obtain the merging blur and affine moment invariants based on graphs(GMBAIs). Then, the GMBAIs and CBAIs are fused together to get a more robust affine and blur invariant feature descriptor,and it’s the logarithmic invariants to blur and affine transform based on graphs(GLBAIs). For the color images, GLBAIs can be futher combined with the color invariants based on the color invariant model, and get the logarithmic invariants to color, blur and affine transform based on graphs(GLCBAIs). This feature descriptor are not sensitive to the blur, noise, and illumination changes, and has good invariance to rotation and scale. At last, the Euclidean distance between the feature descriptors are used as the similarity measure to realize the feature matching. The experiment results indicate that the image registration algorithm is invariant to rotation, scale and affine transform, and has good robustness and adaptability to blur, noise, and illumination. Its matching speed and precision is better than the traditional algorithm. It realize the precise registration and robust mosaic of the blurred aerial images.(5)For the mosaic errors accumulation problems with the increase of the number of images, the above methods of obtaining the geography coordinates of four vertexs in every image through the multi-target orientation algorithm, and making the images preliminary registered can reduce the mosaic errors effectively. Besides, an adaptive multistage stratified stitching algorithm is proposed to reduce the mosaic errors further. It can eliminate the splicing gap better, and then get a high-resolution and wide-field composite image with good vision effect.