Research On RGB-D Semantic Segmentation For Indoor Complex Scenes

With the development of computer vision technology,more and more decoration companies utilize deep learning technology to perform indoor environment aided design.The accurate semantic segmentation information provides better basic data support for the implementation of tasks such as the automatic generation of indoor decoration design scheme and 3D decoration preview.There are many problems in indoor scene,such as large amount of furniture causing space occlusion,uneven lighting,difficult to identify edges.To address the aboved problems and improve the accuracy of indoor scene semantic segmentation,we purpose to integrate the semantic with geometric information.This paper focuses on the RGB-D semantic segmentation technology and application practice of indoor complex scenes.The main research contributions and innovations are as follows:(1)We proposed a depth fusion semantic segmentation model with criss-cross attention.To solve the problem of space occlusion and uneven lighting in indoor scene,we proposed that fused the semantic and geometric information extracted from RGB and Depth branches,and then further refining feature through criss-cross attention module.At the same time,a 3D indoor painting system for indoor scenes was developed based on proposed semantic segmentation model.Following experimental settings,the experimental results on the two datasets show that the proposed depth fusion semantic segmentation algorithm with criss-cross attention improved the mIoU by 4.59% and 2.53%separately.(2)We proposed a depth fusion lightweight real-time semantic segmentation model.To solve the problems of large amount of model parameters,high computational complexity and low real-time performance,we introduced depth branch to leveage depth feature based on a lightweight real-time semantic segmentation model and improved the performance of real-time semantic segmentation.At the same time,a real-time painting system for indoor scenes is developed based on the proposed real-time semantic segmentation model.The experimental results show that the proposed depth fusion lightweight real-time semantic segmentation algorithm improved the segmentation accuracy while maintaining the real-time requirement.(3)We proposed two kinds of lightweight semantic segmentation models with complementary attention mechanism.Due to the RGB and depth feature distribution varies significantly,and hard to combine features effectively,we introduced the complementary attention mechanism to give RGB-D feature in different channels with different weights.The proposed network model was more sensitive to the effective feature by the specific weights.Following experimental settings,the experimental results on the two datasets show that the proposed depth fusion lightweight semantic segmentation model with complementary attention mechanism improved mIoU by 0.9% and 0.78%separately.