Research On Terrain Visual And Semantic Recognition Algorithm For Rescue Robot Based On Deep Learning

Using rescue robots to replace soldiers to perform unmanned battlefield rescue and evacuation tasks has become a key research direction for military logistics research and construction in various countries.The battlefield environment is complex and changeable,various terrain structures of which are coupled with each other,posing great challenges to the continuous movement of rescue robots.Rapid and accurate classification of unknown terrain is the key for rescue robots to achieve autonomous movement.Terrain recognition and terrain semantic segmentation algorithms can effectively improve the rescue robot’s ability to perceive the terrain environment,realize terrain prediction,and avoid falling into dangerous environments.Visual features contain rich terrain scene information,which is an important way for robots to perceive the terrain environment at this stage.Combining with deep learning technology,it has important theoretical significance and practical value to carry out research on visual semantic recognition algorithms for rescue robots based on vision features.To improve the terrain adaptability of rescue robots and enhance their ability to perceive complex terrains,this paper carries out research on the terrain recognition algorithms and terrain semantic segmentation algorithm of rescue robots based on deep learning technology.The research results obtained in this paper are as follows:1)The current research progress of terrain recognition algorithms and terrain semantic segmentation algorithms is summarized,and combined with deep learning technology,the research status,deficiencies and trends of terrain recognition methods and terrain semantic segmentation algorithms based on visual features are emphasized.(2)A terrain recognition dataset containing 16 terrains is constructed.According to the characteristics of the complex terrain of the battlefield,this paper builds the Battlground-16 data set based on the complex terrain of the battlefield by modeling with high-precision military simulation software.The dataset contains a variety of special complex terrain,which can meet the needs of most terrain recognition algorithms research needs.(3)The Best Representation Branch Method(BRBM)is designed based on Capsule Net(Caps Net),which provides new guidance for spatial features,feature selection and feature fusion in visual recognition of terrain images.By combining the mainstream CNN architectures VGG19,Inception-v4 and Res Net50 with Caps Net,this paper verifies the existence of the balance point.The image features extracted at this point can consider both local spatial features and global semantic features,which can effectively improve the performance of the method.The attention heatmaps are visualized through the Grad-CAM algorithm,providing a theoretical explanation for the balance point.Combined with Caps Net,the BRBM algorithm can effectively learn to extract local spatial features in image features,provide recognition decisions for global semantic features,effectively solve the problem of local spatial feature loss,and improve the algorithm’s ability to recognize confusing terrain.Based on the BRBM algorithm,the F~2BRBM algorithm performs feature fusion with activation vectors as the core.The fused image features can effectively represent the local spatial features at the center of the semantic subject and the edge of the structure and achieve higher classification accuracy.The BRBM algorithm and the F~2BRBM algorithm have completed the terrain recognition performance verification on benchmark data set and compared with the public algorithm,and achieved99.58%,96.97%and 94.87%in the UCM,AID and NWPU-45 dataset.Subsequently,the F~2BRBM algorithm also achieved a classification accuracy of 97.62%on the self-built Battleground-16 dataset.The time cost analysis of the algorithm shows that the terrain recognition algorithm designed in this paper can realize the recognition speed of 85 ms for 32 batches of terrain image samples and has excellent terrain recognition ability and real-time response speed.(4)A lightweight semantic segmentation algorithm based on dual-branch feature fusion(Feature Fusion Dual Branch Net,F~2DBNet)is designed.The algorithm extracts and enhances the local spatial features of image features through the shallow feature extraction branch and Caps Net,uses the deep feature extraction branch and Improved SENet to extract and enhances the global semantic features of image features,and finally performs feature fusion through the transposed convolution fusion module.Improve the feature extraction ability of the algorithm.The F~2DBNet algorithm designs two feature extraction structures BA and SA based on atrous convolution and depthwise separable convolution,which reduces the parameters required for network training and improves the performance of the algorithm on small mobile platforms.F~2DBNet-BA algorithm and F~2DBNet-SA The algorithms have completed the comparative evaluation of training,validation and segmentation effects on the Cityscapes dataset,the Sift-Flow dataset,and the Semantic Drone dataset.The experimental results show that F~2DBNet-SA and F~2DBNet-BA achieve a good balance of segmentation effect,parameter quantity and response speed.F~2DBNet-BA achieved 72.88%,87.57%and 82.80%of m Io U under three datasets with lower parameters and faster response speed,which provides the algorithm support for the small and light rescue robot platform.(5)The F~2BRBM algorithm terrain recognition and the F~2DBNet-BA algorithm terrain semantic segmentation experiment are carried out on two mobile robot platforms.The terrain video stream collected by the mobile robot platform uses the video stream preprocessing framework to extract key frames for terrain recognition and terrain semantic segmentation,which avoids repeated detection of adjacent similar frames by the algorithm and reduces the burden of the algorithm.The experimental platforms for terrain recognition are quadruped metamorphic mobile robots and Mecanum wheel omnidirectional mobile robots.To the mobile robot,the experimental terrain is street,parking lot and grass.Under the quadruped mobile robot platform,the F~2BRBM algorithm achieved 99.43%(low speed)and 98.56%(high speed)terrain image recognition accuracy respectively;under the Mecanum wheel omnidirectional mobile robot platform,the F~2BRBM algorithm achieved 96.06%(low speed)and 94.42%(high speed)terrain image recognition accuracy respectively.Compared with the Deep Filter Bank(DFB)algorithm previously designed by our research group,the F~2BRBM algorithm has better recognition performance and better robustness to the platform moving speed.The terrain semantic segmentation experiment was carried out on the Mecanum wheel omnidirectional mobile robot platform.The algorithm has a good segmentation effect for terrain semantic categories with clear boundaries,regular shapes,and large pixels.In addition,the cross-platform expansion application experiments of the semantic segmentation algorithm were carried out on the drone platform.The research results of this paper can provide important information reference and algorithm support for rescue robots to perform rescue tasks in complex terrain environments on the battlefield,improve mobile control and autonomous navigation.