Research On Cooperative Tracking Between Omnidirectional Camera And Active Camera

With the continuous development of the omnidirectional imaging technology and its application, the technology about object detection and tracking for the omnidirectional video has also been more and more researched, it becomes an active research direction in the field of machine vision, which crosses several subjects, and has current or potential applications in fields of security monitoring, driver assistance, on-site monitoring, vehicle inspection, vehicle guidance and space robotics, etc.. In this paper, an in-depth research has been carried out on some key issues concerning cooperative tracking technology between omnidirectional camera and active camera that are to be solved pressingly.(1) Because of variance of the resolution from the periphery to center in catadioptric omnidirectional image, two points in the world with same distance will have different distance in omnidirectional image when they are projected to the periphery or center on the image plane. Thus, traditional definition of the neighborhood can not be appropriate for omnidirectional images. In this paper, make use of characteristic of catadioptric imaging and ratio of resolution between vary positions in an omnidirectional image, we propose a new definition of neighborhood adapted to the omnidirectional images, which can overcome the distortion and non-consistent resolution of omnidirectional image. Contrasting experiment show that this definition of neighborhood for catadioptric omnidirectional images produces a more coincident result than traditional one in application of moving object detection base on Markov random fields.(2) Because of the features of catadioptric imaging, when the object has linear motion in real space, its trajectory on omnidirectional images is not a straight line, but conic. It leads to the big error in prediction of object's next position when applying trajectory prediction method for common perspective image. This paper adopts unifying catadioptric imaging sphere projection model to project object's imaging point in omnidirectional images onto the sphere where the author then establishes omnidirectional kalman filter to predict next position of the object. The result of prediction will project back to omnidirectional image to fulfill position prediction of objects. Experimental results in synthetic and real omnidirectional video demonstrate an obvious improvement in accuracy and stabilization of modified omnidirectional kalman filter in object position prediction.(3) In a hybrid camera system, composed by an omnidirectional camera and a active camera, in order to achieve cooperative tracking between the two cameras, the relationship between position of the pixels in omnidirectional image and the motion parameters of active camera needs to be gotten. This paper, aim at the shortage of polynomial fitting method, proposes to take polar coordinates of the pixels in omnidirectional image as the fitting variables and to divide the whole image into multi-piece and then fit in every piece, so as to solve the jumping problem on the rotation angle near by 0 degree, at the same time to improve the fitting accuracy. Space mapping experiment in a hybrid camera surveillance system shows that the method of using polar coordinates and piecewise fitting has a more consistent fitting accuracy among different pieces and better stability than the single polynomial fitting method.(4) For catadioptric imaging may cause mirror effect between omnidirectional image and perspective image, and scale invariant feature transform algorithm is not invariant to image mirroring. This paper proposes flip horizontally perspective image, and then matches separately the original image and the flipped image with omnidirectional image, takes a better match as the final result to achieve the mirror invariant. For the ring distortion of the omnidirectional image, the perspective image is transformed to the fan-shaped image before the matching, and two methods are provided to transform the perspective image to the fan-shaped image. Experimental results on the real image show, after the perspective image is transformed to the fan-shaped image and then matches with omnidirectional images, the total number of matching points is increased , while the number of the wrong matching points is reduced, matching results are better than one without the transformation.