Research On Job Scheduler For Vehicle Networking System Based On Hadoop Platform

With the size of vehicle networking system data increasingly large,the difficulty of the job processing is growing.The traditional data processing model has been unable to adapt to deal with massive data.Therefore,cloud computing has been widely used in vehicle networking system.Hadoop is the most popular open source distributed cloud computing platform.Hadoop 2.0 is the next generation Hadoop platform,and it consists of significant improvements over the previous stable release.Job scheduling is a core component of YARN,and also directly related to the computational performance and resource utilization of the cloud computing platform.Vehicle network system based on Hadoop2.0 with high real time request.Jobs of vehicle network system is memory intensive.However,current research on real-time scheduling of Hadoop concentrates in Hadoop1.0 framework,and these scheduer has low utilization rate of resource.On the basis of YARN and job scheduling algorithm research,this paper proposes the memory aware real-time scheduler for vehicle networking system.The scheduler designs resource allocation center,double preemptive strategy and minimum resource assessment model.Resource allocation center in order to meet the characteristics of resource model and memory-intensive jobs,selects Memory to quantify the cluster resources.Double preemptive strategy selects job by Surplus-RBTree data structure,and selects tasks based on progress of the tasks in the implementation.The minimum resource assessment model combines task execution time speculate model and lagrange multiplier method to get minimum resource that can make sure job completes before the deadline.Finally,by using the method of theoretical analysis and simulation experiments,the effectiveness of the scheduler is validated in the simulation of Hadoop cluster.Through the analysis of experimental data,the scheduler can meet deadline of the job in vehicle networking system,and also show that preemption and resoucre allocation center can balance the resouces among nodes in the cluster and improve resource utilization.