Multi-camera Matrix Real-time Video Stitching And Intelligent Analysis Based On UAV System

With the development of video technology,the technology and application of video splicing have also developed rapidly.Currently,video splicing algorithms are mostly aimed at scenes where the camera angle is fixed,and for non-fixed viewing angles,they are faced with many problems such as high computational complexity and inability to process in real time.The combination of video applications and artificial intelligence has also become a recent research hotspot.However,combined with low-altitude unmanned aerial vehicles,there is currently no excellent work of detection in aerial video,and the existing algorithms cannot achieve high performance for the detection of small objects,and have a single dimension for information extraction.In order to obtain flexible perspectives,high-quality real-time panoramic video and multi-dimensional target information,this paper creatively combines drone platforms to build instruction interaction control systems,and proposes new online and real-time video splicing algorithms,intelligent target detection and extraction.The algorithm is used to analyze the video data under a large field of view.In this paper,by the research of the real-time video with multi-camera matrix,this paper proposes a feature point extraction using information entropy to optimize the corner points.Based on the feature description of multigranularity operators,AFD(auto-adaptive feature detection)and ROI(region of interest)are combined to extract feature vector.At the same time,for the multicamera panoramic video,the difference of camera angles,and the video jitter caused by the flight difference,an adaptive frame-restraining algorithm and a fixed screen algorithm are proposed,and the flight control information of the drone is used to optimize the video quality.Through the CPU-GPU heterogeneous processing algorithm,the computing speed of the splicing algorithm is improved.For the optimization of the algorithm flow,two stitching algorithms,FrameStitch and SimpleStitch,are proposed.For two channels of 720 p video,the real-time stitching speed of 98 fps can be achieved.In the application of drone panoramic video,this paper introduces a target detection algorithm based on artificial intelligence.Because there are few pixels in the low-altitude perspective,the accuracy of the detection is difficult to guarantee.To solve this problem,combined with the RCNN algorithm,an endto-end depth network based on the VRPN-VAEN pipeline is proposed,and the shallow end pixel information is merged.With deeper high-dimensional feature information,more accurate candidate regions were obtained.In addition,the concept of “cycle learning rate” was also proposed to improve the accuracy of object recognition.And use Postfilter(filtering network)to optimize false positive.Due to the lack of top-view structural datasets,datasets from top view angles for training and testing were collected and established.In order to compensate for the limited training data,the data is rotated and croped with random noise and lighting compensation to prevent overfitting.Through the experimental comparison,the test results are finally performing well in the detection and classification.