Research On Video Retargeting Algorithm Based On Smart Cropping

With the rapid development of communication networks and video compression technology,people can watch video on various media terminal devices.Since the display screens of terminal devices are not the same,the size of the original video needs to be scaled to fit the size of the device’s screen.Traditional video scaling methods,such as traditional cropping,uniform scaling and letterboxing,directly adjust the video size without considering the important content of the video,which is easy to cause deformation and distortion of the important content of the video and the video looks bad.Therefore,how to adjust the size of the video adaptively according to the video content,while protecting the important content information of the video to the greatest extent,and ensuring the fluency of the video playback is a challenging problem.The existing video retargeting method based on smart cropping is to determine the important content area of the video by detecting the salient content of the video frame,calculate the relevant parameters of the cropping window adaptively according to the size of the retargeting video,and conduct retargeting operation on the original video according to the parameters.At present,video retargeting methods based on smart cropping have the problems of visual content loss and low video temporal coherence.The main reason is that the current relevant algorithm fails to detect the salient contents and moving objects in the video well,which leads to the wrong information of important contents being cut out of the cropping window,and the insufficient coherence constraint on the video in time domain leads to the discontinuity of the video after retargeting,resulting in jitter.In this thesis,the video retargeting algorithm based on smart cropping is deeply studied,which makes full use of visual salient information and strengthens the constraint in time domain,so as to improve the quality of retargeting video and the performance of redirection algorithm.The results are as follows:(1)A smart cropping video retargeting algorithm based on relative displacement constraint between frames is proposed.By calculating the salience map and motion history map of the video frame,the algorithm generates the final importance map by adaptive fusion,and calculates the total visual information loss;Relative displacement parameter and the maximum inconsistent limit coefficient that human can tolerate constitute the temporal constraint term,then the optimal scaling factor and cropping window are obtained by joint optimization solution of the system,and the retargeting result is obtained by smart cropping of the original video.The results of experiments show that when comparing with the traditional method and other video retargeting algorithms based on smart cropping,this algorithm can protect the important content and important information of the video better subjectively,the video is smooth and coherent,and there is almost no jitter,and the retargeting video has a better visual perception.Objectively,TID and SSM indexes were used for comparative analysis,and the performance of the retargeting video obtained by this algorithm was also significantly better than other algorithms,which meant that this algorithm could retain more visual salience information and better maintain the temporal consistency of the video.(2)A multi-operator video retargeting algorithm based on cropping is proposed.When processing videos with fast movement speed,more important objects and scattered distribution of important objects,it may cause the problem that the cropping window can’t be accurately determined,too much unimportant information is retained and some contents of important objects are missing.In order to solve the above problems,the algorithm proposed in this thesis modified motion history map,improve the important degree of the movement salient information,combined with seam carving algorithm adaptively removing seam to the important content of the original video in pixels,after calculating the overall consistency constraints in spatio-temporal information with minimum energy loss optimal scaling parameter and retargeting video.The results of experiments show that the algorithm is effective and feasible for video retargeting with complex backgrounds,many important objects and fast motion.Compared with traditional methods,other video retargeting algorithms based on smart cropping and the algorithm proposed in the second chapter of this thesis,subjectively the video has no jitter and can better protect the information of important moving objects in the video with good visual effect.Objectively,TID and SSM indexes are used for comparative analysis,and the performance of this algorithm is better than other algorithms.Retargeting video can better maintain temporal coherence and important content.