Vehicle 6 Degrees Of Freedom Pose Estimation With Deep Learning

With the advent of the artificial intelligence era,Artificial Intelligence?AI?has been applied in various industries to drive its service development.Automatous driving is an important direction for the automotive industry in the future.Therefore,there are many researches concerning vehicle environmental perception such as objects detection.However,the existing algorithms can only detect the class of objects and their bounding boxes in monocular RGB image,which can't meet the requirements of automatous driving.In this paper,we propose a multi-stage vehicle 6-DoF pose estimation algorithm and another end-to-end vehicle 6-DoF pose estimation algorithm to predict category and 6-DoF pose of the vehicles based on a monocular RGB image simultaneously.Multi-stage vehicle 6-DoF pose estimation algorithm is composed of two stages:in the first stage,there is a vehicle detection and 3-DoF rotation estimation network for predicting the class and rotations of vehicles,in the second stage,a special vehicle 3-DoF translation estimation module is used for predicting the translations of vehicles.To this end,the vehicle detection and 3-DoF rotation estimation network extends Faster R-CNN by adding customised heads for predicting vehicle's finer class and 3-DoF rotations in the actual driving environment respectively.Then,the 3-DoF translations of vehicle in the actual driving environment were estimated by the vehicle 3-DoF translation estimation module according to the vehicle's finer class,3-DoF rotations and its bounding box,which are predicted by vehicle detection and 3-DoF rotation estimation network.The experimental results show that the multi-stage vehicle 6-DoF pose estimation algorithm can achieve vehicle 6-DoF pose estimation based on a monocular RGB images.In addition,another end-to-end vehicle 6-DoF pose estimation algorithm was proposed to deal with the problem that multi-stage vehicle 6-DoF pose estimation algorithm could not achieve end-to-end training and has a low mAP for vehicle 6-DoF pose estimation.The end-to-end vehicle 6-DoF pose estimation network extends vehicle detection and 3-DoF rotation estimation network by adding customised heads to replace vehicle 3-DoF translation estimation module in multi-stage vehicle 6-DoF pose estimation algorithm for predicting vehicle's 3-DoF translations in the actual driving environment to achieve end-to-end vehicle6-DoF pose estimation.In order to verify the feasibility and effectiveness of the end-to-end vehicle 6-DoF pose estimation algorithm,we use the dataset of Baidu ApolloScape's 3D Car Instance Understanding Challenge to conduct some related experiments.The results show that the end-to-end vehicle 6-DoF pose estimation algorithm is significantly better than the multi-stage vehicle 6-DoF pose estimation algorithm and reaches the 1 st place in AppolloScape challenge 3D Car Instance task^[56].