Research On Vision-based Automotive Lane Change Assistance System Under Virtual Environment

Advanced driver assistance systems (ADAS) have gained increasing popularity inrecent years, as traffic safety and efficiency have become two of the major challenges facedby automotive industry. Conventionally the development, test and verification on ADAS aremainly conducted with extensive in-vehicle field testing, which presents great difficulties inguaranteeing test repeatability, and in covering broad variation and complexity of the drivingenvironment, such as road and pavement, traffic conditions, weather and light conditions, etc.More importantly, for some safety critical features, the field testing can be dangerous if notimpossible.Some commercial tools were developed with attempt to address the issues throughmodeling and simulation technologies on vehicles and/or vehicle driving environment.While they were developed with different emphases on specific applications, thetechnologies are in general more mature on conventional vehicle dynamics and control areasthan on ADAS. Firstly, this paper proposes a virtual vehicle driving environment with notonly vehicle dynamics models, but also real-time virtual driving environment. The aim is toenable the high-fidelity design, test and verification of the ADAS systems. The proposedenvironment mainly includes models of vehicles traffic and on-board perception sensors;virtual road, traffic and weather conditions; and a3D high-fidelity virtual drivingenvironment developed based on computer graphics technologies and a real-timehardware-in-the-loop (HIL) system with a driver simulator. Besides, this paper presents anovel approach of developing a vision-based lane change assistance system (LCA) under thisvirtual and real-time driving environment. Comparing with Radar, visual sensors are able toextract lane markings and different types of obstacles through pattern recognition algorithmsas they can offer much more color information. What’s more, they are cheaper than Radarsensors, and as the digital signal processing unit performance improvement, their disadvantages, including low distance resolution, slow processing time and sensitive tochanging environment, can be improved.By summarizing current research achievement of LCA system development, test andverification, this paper proposes to develop the LCA system under the proposed real-timeand virtual driving environment so as to develop, test and verify the whole system under safe,effective, repeatable and high-fidelity conditions. And this paper tries to address thefollowing common problems in the processing of development, including vehicle and lanemarkings detection, lane change intention recognition, prediction of potential collisionposition, time-to-collision estimation. The specific research in this paper is listed as below:1. Establishing the virtual driving environment simulation platform. The proposedplatform can be divided into two sub-platforms, including offline simulation platform andreal-time simulation platform with virtual driving environment. The offline one is runningunder a personal computer, which has MATLAB/Simulink and the virtual environmentsoftwares. And the real-time one contains a nonlinear real-time vehicle dynamics model,running under a dSPACE HIL-simulator, to simulate the host and traffic vehicles, a driversimulator with6degree-of-freedom motion platform, utilized for human drivers to evaluatethe performance of developed ADAS, a dSPACE1401for running ADAS algorithms, and a3D virtual environment. The whole systems of the real-time virtual driving environmentsimulation platform are connected and communicated via CAN buses and Ethernet, whichemulate the real vehicle electric architecture for readily adoption to in-vehicle field test.2. In order to evaluate the fidelity of the virtual environment, in this paper, severalimage processing algorithms are used to compare the differences between the virtual road,vehicles and environment and the real world. the histogram of each color channel andhistogram of gradient are involved to compare the virtual and real road. In addition, threealgorithms are used to evaluate the fidelity of vritual vehicle models in the virtualenvironment, including vehicle shadow detection, horizontal edges detection and Haarfeature recognition.3. Finally, a vision-based lane change assistance system is developed under the proposed real-time and virtual driving environment simulation platform. It contains threemain modules: an image processing module, running under a digital signal processing unit(DSP) to extract the vehicles and lane markings; a lane change assistance algorithm, runningunder a dSPACE1401to estimate the danger level when drivers intend to change lane; ahuman machine interface (HMI), running under an ARM-board to warn drivers by visual andauditory sense. The image processing unit is attempted to detection vehicle position in theimage coordinate and lane markings exactly with much less noise brought by changingenvironment. On the other hand, the LCA module contains four unit: a lane change intentionrecognition unit with the aim to predict the lane change intention without switching turninglight; a vehicle tracking unit to create sequence data for each detected traffic vehicles fromsingle image; a the most dangerous object selection unit with the aim to reduce error warning;a collision probability analysis. In this paper, two contacted circles are used to emulate thelane change trajectory so as to predict the potential collision position. And then atime-to-collision (TTC) estimation is needed to estimate the relative motion between the egovehicle and rear-end vehicles via the shadow position in the image coordinate. A Kalmanfilter with a time-varying observation error is used to smooth the coordinate of vehicleshadow. By comparing the TTC and the space between the two vehicles, a warning signal isdecided to warning the danger of this lane change behavior.