Research On Methods And Experiments Of Alignment Based On Hybrid Micro Vision Orienting Microinjection

Seeking the solution to life's mystery,humankind pays great attention to modern life science,including genetic engineering and cell engineering.Therein,microinjection is broadly used in the common experimental operations in these fields.Manual microinjection is a timeconsuming process.In addition,the success rate is greatly affected by the inevitable misalignment and trajectory deviation of the micropipettes.Therefore,automatic injection has been the focus.The researches in this interdisciplinary subject include cell posture control and puncture force control.To solve the problem of observing the posture of the micropipettes during microinjection,a hybrid micro vision model has been proposed,consisting of a telecentric stereo camera and an inverse microscope.Collaborating with a manipulator of five degree-of-freedom(DOFs),the system is established based on this vision model.Research on aligning micropipettes automatically has been proposed.The main research contents are as follows:1)Study on the calibration method of the presented hybrid micro-vision system composed of a telecentric stereo camera and an inverse microscope.Compared to an inverse microscope,a telecentric camera has a larger field of view,depth of view,and work distance.The telecentric stereo camera is utilized to deal with sensing the depth information of a particular feature point,and the inverse microscope is used to deal with fine manipulation.In our works on the calibration method,the calibration board traveled through the common vision field of all the cameras.At first,the microscope has been calibrated in a Gaussian-optics-theory-based method.The feature points in the reference coordinate have been computed with images sampled by the telecentric stereo.These correspondences between spatial points in the reference coordinate and image points,as known as the virtual calibration boards,were utilized to reinforce the calibration outcome by non-linear optimization.2)Study on feature extraction methods of the image and spatial domain of the micropipette.Identify the region of the micropipette in the telecentric stereo camera based on the grayscale difference between two frames sampled before and after the pipette moved.After a coarse search of pipette edge being performed by a grayscale morphological approach,the subpixelscale edge detection has been randomly implemented perpendicular to the axis estimation to extract a precise pipette axis,which is inspired by the RANSAC algorithm.With the image feature extracted,the tip locations can be computed by the Optimal Triangulation method,and the spatial axis can be computed by the rule of the projection of lines on a telecentric camera.3)Study on micropipette alignment method based on visual servo.To meet the demand of the posture control of the micropipette,the model of a manipulator of five DOFs has been established by the multi rod method.Then,the Jacobian matrix has been extracted according to the kinematic model of the manipulator.In this article,an open-loop method with two phases,based on the quaternion multiplication,has been proposed.The rotation for adjusting the posture of the injector would be computed,followed by the computation of the translation.To fulfill the goal to adjust automatically,the posture-based vision servo control scheme has been applied.With the output of the open-loop method as estimation,a process of micropipette alignment with assistance to the installation has been built up.After the alignment,the system would transfer to the image-based vision servo with the microscope only.4)Design and construction of the microinjection system and experimental studies.According to the view structure of the hybrid micro vision,a microinjection system has been built up with the manipulator of five DOFs.Furthermore,the vision system and the model of the manipulator have been calibrated by the method proposed in this article.Both the open-loop and the visual servo method for micropipette alignment have been implemented by the system.To evaluate the result of micropipette alignment,a series of zebrafish embryo puncture experiments have been performed.The phenomenon of the deformation of zebrafish embryos reflected the well quality of micropipette alignment.