Hardware Implementation Of Real-Time Video Super-Resolution Processing Algorithm Based On FPGA

Super-resolution technology,which converts images from low resolution to high resolution,is widely used in video and image processing fields such as remote video interconnection,medical image analysis,earth remote sensing and meta-universe.Super-resolution technology can reduce the requirements on the accuracy of video acquisition equipment and the transmission bandwidth of video data in the process of acquiring ultra-high-definition video,so it is widely used in the application scenarios that require high definition,high fluency and high sensitivity video.At present,the performance of the super resolution algorithm based on Convolutional Neural Network(CNN)is far superior to the traditional methods.However,because of the complexity of the network,deploying CNN algorithm on terminals faces the hardware constraints of computing resources,storage resources and memory access bandwidth.Therefore,this thesis proposes a hardware implementation of deploying CNN-based super resolution algorithm on Field-Programmable Gate Array(FPGA)to complete a complete set of end-to-end real-time video processing display hardware implementation from video input,video processing to video output.The main tasks of this article are as follows:(1)Select an appropriate super-resolution algorithm based on hardware adaptability and super resolution reconstruction performance.The performance of the CNN-based super-resolution algorithm is far better than that of the traditional superresolution algorithm.However,in the calculation process,the CNN algorithm will introduce huge bandwidth consumption for parameter transmission,huge storage resource consumption for recording the intermediate process quantity and a large amount of computing resource consumption for basic operations,resulting in the shortage of transmission bandwidth,storage resources and computing resources of the hardware platform.In this thesis,the performance and hardware resource constraints are comprehensively investigated,and the super-resolution algorithm based on lightweight CNN structure is selected for hardware platform deployment.(2)In order to realize the hardware acceleration of the super-resolution algorithm,the FPGA with low power consumption,high parallelism and the ability to flexibly configure its own hardware circuit according to the downloaded bit-stream file is selected as the hardware platform of the super-resolution algorithm acceleration.In addition,a high-speed real-time video processing circuit board based on Xilinx FPGA is designed for real-time processing of ultra-high Definition video.The hardware resources include dual FPGA core chip,16-channel High Definition Multimedia Interface(HDMI),eight 10-gigabit Ethernet ports and 12 pieces of Double Data Rate Synchronous Dynamic Random-Access Memory(DDR SDRAM)with 40 Gb of dynamic memory space.The platform provides a set of high-performance hardware solutions for the real-time transmission,storage and processing of ultra-high-definition video,and can complete the real-time processing and display of ultra-high-definition video at the highest level 4K@30HZ.The hardware platform provides high storage space,high access bandwidth and sufficient computing resources for the deployment of ultra-resolution algorithms.(3)Design the hardware accelerator of super-resolution algorithm on the independently designed real-time video processing platform based on FPGA.In addition,in the design process of hardware accelerator,the parameters and architecture design are optimized to reduce delay,and the processing time of super resolution algorithm is significantly shortened.With the logic control code of video input and output,a complete set of hardware implementation of ultra-high-definition video input,super resolution processing and display is completed,which can support real-time super resolution processing of 2K@30HZ video.The processing delay is 31.174 ms,the overall power consumption is 19.159 W,and the Peak Signal to Noise Ratio(PSNR)is32.17 d B.Finally,the high-resolution video of 4K@30HZ is obtained.The test results show that the performance of the hardware can meet the requirement of obtaining ultra HD video.