Research On Parking Slot Detection And Tracking Method In Automatic Parking System Based On Around View Monitor

As a typical application of autonomous driving in limited scenarios,the development of the APS(Automatic Parking System)will accelerate the commercialization of autonomous driving.The current APS in mass production relies on too many radars and vision sensors,resulting in higher overall costs,so it is usually used in some mid-to-high-end vehicles.Since the AVM(Around View Monitor System)has become a standard configuration for vehicles,the development of APS based on AVM can make full use of existing fisheye cameras without adding other radars and cameras,thereby further reducing the cost of APS,and promoting the popularization and application of APS.Currently,the research on the visual perception for APS is still in its infancy,with a series of problems such as poor robustness,cumbersome steps,and poor real-time performance.Therefore,aiming at the core vision technology(parking slot detection and tracking)of APS,this paper proposes a series of new methods to deal with the problems of around view image generation,vacant parking slot detection,obstacle detection,and parking slot tracking,which improves the robustness,convenience and real-time performance of the system.The work and innovations in this paper are as follows:(1)Generation and acceleration of around view images.A method for generating a well-spliced and seamless around view image based on the lookup table,IPU(Image Processing Unit)acceleration,and GPU(Graphics Processing Unit)acceleration is proposed to increase the running speed by about 16 times,which meets the requirements of real-time on embedded devices.The dynamic look-up table is calculated by combining the articulation angle and the static look-up table to generate a variable-angle around view image that can be applied to articulated vehicles,which expands the application range of AVM.Through the bench test and real vehicle experiment,it is verified that the globally balanced and seamless around view image generated by the proposed method not only achieves a processing speed of 30frames/second on the embedded processors but also can be applied to different types of vehicles such as passenger cars,articulated engineering vehicles,and multi-carriage articulated vehicles.(2)Detection of vacant parking slots in around view images.A method of vacant parking slot detection based on the DCNN(Deep Convolutional Neural Network)is proposed,namely VPS-Net.By converting the vacant parking slot detection into a cascading problem of multi-target detection and classification based on the DCNN,the robustness in the complex visual environment is improved.For parking slot detection,a detector based on YOLOv3 is used to detect the head and marking points of the parking slot at the same time,which simplifies the matching steps of the marking points of the entrance line.For occupancy classification,a customized DCNN model is designed to determine whether the parking space is vacant,which reduces the size of the model and speeds up the occupancy prediction.Through experiments on the public parking slot dataset,it is verified that compared with other vacant parking slot detection methods,VPS-Net not only can obtain a higher precision rate and recall rate but also has better generalization performance.Compared with the recent method,the precision rate and recall rate of VPS-Net is improved by about 0.9% and 7.0%respectively.(3)Improvement of parking slot detection in around view images.Since VPS-Net is based on the existing object detection model,it has the disadvantages of large model size,long detection time,and cumbersome post-processing.Therefore,a method for parking slot detection based on directional entrance line regression is proposed,namely DLPS-Net.By modeling the parking slot as a directional entrance line,a detector for the directional entrance line detection is designed,which reduces the number of model parameters and simplifies the post-processing process.Through experiments on the public parking slot dataset,it is verified that compared with other parking slot detection methods,DLPS-Net not only can obtain better detection results but also has a shorter prediction time.Compared with the parking slot detection of VPS-Net,DLPS-Net shortens the detection time by about 31.6% under the condition of ensuring accuracy.(4)Obstacle detection in around view images.A method of obstacle ground lines regression and mapping from fisheye images to the around view image is proposed.To facilitate the research of obstacle detection in the around view image,an image dataset is established,namely OD＿AVI.To regress ground lines of obstacles in the fisheye image,three detection network models based on YOLOv5(YOLOv5-VGG,YOLOv5-Point,and YOLOv5-Ratio)are designed respectively.The ground lines detected in multiple fisheye images are mapped and fused to the around view image through the around view generation model.Through experiments on the OD＿AVI dataset,it is verified that the proposed method can effectively detect obstacles in the around view image under different visual environment conditions.Compared with the occupancy classification of VPS-Net,this method improves the classification accuracy of parking slots by about 2.8%.(5)Tracking of parking slots in around view video sequences.A parking slot tracking method based on KCF(Kernelized Correlation Filter)during driving and parking is proposed.To facilitate the research on the tracking of parking slots during driving and parking,a video dataset was established,namely PS-Track.During the driving process,the parking slots detected by DLPS-Net+YOLOv5-Ratio are tracked,thereby reducing the missed detection rate of parking slots and improving the accuracy of occupancy judgment.During the parking process,the directional marking points of the target parking slot detected by the fine-tuned DLPS-Net are tracked,so that even if a marking point is occluded,it can be inferred from the prior geometric informati on and the other marking point.Through model quantization and NPU(Neural-network Processing Unit)acceleration,the running speed is increased by about 28 times,which meets the real-time requirements on the embedded processors.Through experiments on the PS-Track dataset,it is verified that the proposed tracking method improves the accuracy of vacant parking slot detection during driving and the continuity of target parking slot tracking during parking.Through the real vehicle experiment,it is verifie d that the proposed method can be combined with automatic parking path planning and control methods to achieve APS based on around view image.