Scene Understanding Based On Point Clouds And 2D Images

Automatic scene understanding by computer is the apple of researchers’ eyes,and recent advances in deep learning makes us more confident about it.Point clouds and images are very important to scene understanding,but point clouds can’t import to deep learning algorithms such as convolutional neural networks directly and image based deep learning algorithm can be applied to more areas such as city management.In this article,we transform point clouds to voxels so they can be input of deep learning algorithm,but during the transformation,information may be lost.We put low level features of point clouds such as normal,surfaceness and linearness into point clouds,and these features can maintain some local distribution information of point clouds.In fact,we improved the classification accuracy of point clouds by put local feature into voxel.On the Sydney Urban Objects Dataset,the Surfaceness feature improved the classification accuracy by 3.5% the normal feature improved 1%.We also tested different voxelization setting such as voxel size and voxel resolution and find out how they affect the classification accuracy.There are more and more street view images and images captured by street cameras,it’s of great value to use the images to help city management.We use deep learning algorithms to detected interested objects of city management of urban scene images.Lack of training images is the bottleneck of interested object detection,we use data merge,data augmentation and transfer learning to solve this problem,and finally we can detect interested objects quickly and accurately.At last,we build a portable platform to detect interested objects and it’s very useful.