Fast Scene Layout Estimation Via Deep Hashing

Scene layout estimation has been a focus of image processing and computer vision research,its goal is to estimate the geometric structure of the scene according to the given scene image,and automatically identify the specific area.Whether indoor or outdoor scenes,layout estimation benefit many real-life tasks such as automatic drive,domestic robot and 3D reconstruction.Recently,there has been significant work focused on solving for the spatial layout of indoor or outdoor scenes.These methods adopt a common procedure for estimating layout: 1)detect long straight lines use manual design features,2)generate candidate layouts,3)select the best layout.Step 1)is very sensitive to clutter in the scene.However,in the scene,there are often a large number of objects which are related to the scene category,although these can provide rich context information.The structural boundaries are often occluded,and the observed lines are instead generated by the clutter of people,cars,buses,chairs,tables and other objects.So cluster can lead to a poor set of long straight lines,which lead to a poor set of candidate hypotheses,from which even the best layout choice is still wrong.Secondly,the inference in above methods need to select best layout by solving the optimal problem among multiple candidate layouts.This process is usually very time consuming,which makes the algorithm difficult to meet the real-time requirements.In this work,we address the above two problems by using deep learning algorithm and hashing methods.For each image,a number of image patches are first cropped from the image and feed into a convolution neural network which is trained for layout estimation problem.Then,we extract feature patches from the feature maps in multiple layers of the CNN model.The deep features are then coded into binary codes using hashing algorithms.Hashing function can map data into compact binary hash code,which can be compared quickly by hamming operation while greatly reducing the storage cost.When testing,we compared the binary codes obtained from the query image with those from the training set.The closest neighbor of the test patch determines layout of this patch to the predicted layout.The complete layout is then estimated via a sophisticated voting stage in which different votes are considered differently according to their importances.In order to verify the effectiveness of our method,experiments are carried out on a number of large set of scene layout estimation datasets.The proposed algorithm outperforms the state-of-the-art methods in accuracy for outdoor scenes while achieves the comparable performance to the best indoor scene layout estimators.Furthermore,the proposed method is real-time speed(up to 25 fps).