Software Design Of Multi-Camera Video Stitching Based On AM5728 Processor

With the rapid development of electronic information technology,new multimedia applications are constantly emerging,and the requirements for video fidelity and clarity are becoming higher and higher.As an important research direction in the field of video information processing,multi-cameras video stitching technology can obtain high quality wide perspective experience,which single camera cannot obtain.During it' s research,improving the quality of images after the video stitching from different point of view,and enhancing the processing speed of multi-camera stitching at the same time are the main difficulties to be solved at present.Based on the TI-AM5728 multi-core processor,the software of multi-camera video stitching is developed in this paper.The main tasks are as follows:(1)The function and performance of video splicing software are analyzed,and video splicing software is designed globally based on AM5728.Besides,the software and hardware development environment is built.(2)Multi-channel video image splicing module design.In the image matching phase,an ORB feature matching method based on image overlapping area is designed by comparing the experimental results of common registration algorithms.Only within the restricted areas,this method complete the image registration and transformation matrix calculation according to the ORB method,two-way FLANN algorithm and RANSAC algorithm.It not only effectively reduce the error rate of the feature points on registration,but also has good robustness for light transformation,affine transformation.In greatly time-consuming image fusion phase,an OpenCL heterogeneous parallel programming framework is used to call AM5728 DSP kernel for accelerated processing in this paper.Firstly,the optimal suture line is calculated by the dynamic programming search criterion in the multi-path image after the projection transformation.Then,in each layer of the Laplace pyramid,the image is fused according to the suture template.Finally,the panoramic image can effectively avoid blurring and splicing gap,and the whole transition is natural.(3)Design of encoding and transmission module of video splicing.To meet the requirements of video stitching software for the network data transmission with low latency and low bandwidth,in this paper,GStreamer framework is used to handle spliced video data,and AM5728 ' s video encoding hardware accelerator IVA-HD is used to complete h.264 video compression encoding.Then,stream data interaction protocol based on TCP/IP protocol is designed,and cross-platform network transmission realizes with the Ethernet interface.(4)Video decoding client design.The client software for remote video decoding,which consist of modules such as starting,data communication and decoding display,is completed on the Qt platform.The developed video splicing software has realized the whole process from video collection,splicing,coding and transmission to decoding display.The average delay is about 320ms,which basically achieves the real-time effect.