Image Resizing Based On Energy Transferring And Pyramid Pooling

With the increasing number of display devices with different aspect ratios,the mismatch between images and display devices becomes increasingly prominent.Image resizing technology has become an important research topic in image processing to solve the above problem.Content-aware image resizing methods make image deformation occur as far as possible in relatively unimportant areas in the resizing process,but when the image resizing is too large or the background is complex,there will be serious object deformation or the global semantic loss.The image resizing method based on depth learning can extract the high-level semantic features to guide the image resizing process,which can alleviate the global semantic loss,but will result in the low-level semantic loss.To solve the above problems,two image resizing methods are researched in this thesis: one is to propose an energy transferring image resizing method ET-CAIR(Energy Transferring for Content-Aware Image Resizing)to solve the problem of image resizing based on content-aware;the other is to propose an improved image resizing method ISPP-IR(Improved Spatial Pyramid Pooling for Image Resizing)to improve the pyramid pooling model,which canenhance the low-level semantics of image resizing in depth learning.The main work of this thesis are as follows:We researched methods on content-aware image resizing and proposed the ET-CAIR method.The ET-CAIR method improves Seam Carving method,and combines with Uniform Scaling to form the multi-operation resizing.In Seam Carving,firstly,gradient map,visual saliency map and face map are integrated to form the importance map;then,an energy transferring rule is proposed to protect the neighborhood pixels of the optimal seam and avoid a large number of seams passing through the same area when removing or copying the optimal seam;next,an objective similarity evaluation method is proposed to detect the similarity between the retargeted image and the original image.If the deformability of the retargeted image exceeds the threshold value,it will be changed to the Uniform Scaling method to reach the target size,by this way,the retargeted image will not undergo obvious deformation and retain the global semantics to the greatest extent.We researched image resizing methods based on deep learning and proposed the ISPPIR method.The ISPP-IR method is based on encoder and decoder,and can be embedded into end-to-end image resizing frame.The coder is a pre-trained Res Net classification model,and the decoder is an improved pyramidal pooling model,i.e.a multi-layer cascade of the pyramidal pooling model and the upper sampling model.The upper sampling model gradually upsampled to the desired size of the target image.The reverse propagation of the ISPP-IR network is controlled by the sum of the content loss function and the structure loss function.The ISPP-IR method can effectively extract the local and global semantic information,and retain the local low-level semantics of the original image.The ET-CAIR algorithm is validated in MIT Retarget Me and MSRA databases,and the experimental results of ET-CAIRand four state-of-the-art image resizing algorithms show that ET-CAIR method has better resizing effect than other methods in subjective evaluation and objective quality evaluation.ISPP-IR network is trained in VOC2007 database,and ISPP-IR resizing results are compared with image resizing method based on deep learning,image resizing methods based on content-aware and ET-CAIR algorithm,the results show that ISPP-IR method has better visual effect than other methods.