Research On Individual Segmentation Of Group-piglet Image Based On RGB-D

Large scale pig breeding is the development trend of our country.There is a huge demand for pork consumption market in our country,and people's requirements for highquality pork are more and more clear.To realize modern management and intelligent breeding of pig farms,improving animal welfare and economic benefits,sustainable and healthy pig breeding is a research hotspot.Pig behavior monitoring can reflect pig's physical condition,growth condition and evaluate pig's condition,which is the main reference for improving pig's welfare and raising economic benefit.The traditional manual monitoring method is time-consuming,laborious and inefficient,which is not suitable for intensive large-scale breeding.Due to the living habits of pigs,the effect of contact sensor on monitoring the behavior of pigs is not ideal.Therefore,the non-contact computer vision technology for monitoring the growth of pigs has broad research prospects and practical needs.Using computer vision system to monitor the behavior of pigs can reduce the stress response of pigs and improve the welfare of pigs.In this paper,the group-piglet as the research object,the development of image acquisition system,image preprocessing,combined with deep learning method to build a network model,RGB images and depth images as data training at the same time,to achieve the group-piglet individual segmentation.The main contents and conclusions of this paper include:1)The image data acquisition system is developed,and the preprocessing of RGB images and depth images as test data is completed.In this paper,Kinect V2 depth camera is selected as the research and development equipment.According to the needs of image acquisition,the image acquisition program is developed to store the RGB images and depth images at the same time.Through the Kinect official SDK,the RGB images and depth images registration experiments are carried out,and the actual performance of the image acquisition program is verified.In this paper,Python is used to get the offset of RGB images and depth images,and ROI of Opencv is used to cut the overlapped part of RGB images depth images.Then,resizing function and inter-linear function of Opencv are used to realize the alignment of RGB images and depth images.2)A D_CNNnet network model based on deep learning semantic segmentation is constructed.Based on the method of deep learning,the basic operation operators of convolutional neural network are redefined.Convolutional operations and pooling operations are recompiled to the traditional operators,and RGB images and depth images information are integrated in the model training process,and the overall parameters of the network do not consume too much computing resources.Based on the method of deep learning and the fusion of RGB images and depth images information,a group-piglet segmentation network D?CNNnet using deep depth-aware convolutional operator is proposed.The experimental results show that the group-piglet segmentation network model based on RGB images and depth images information can achieve better segmentation of the group-piglet in the pig house,and the pixel segmentation accuracy on the training set and the verification set is 97.99 % and 98.00 % respectively.3)A Double FPNet model based on RGB-D instance segmentation is developed.It is difficult to segment target group-piglet individuals due to the habit of crowding and adhesion between piglets in actual piggery,which has become a technical bottleneck for visual tracking and behavior detection of group-piglet.And we think accurate segmentation of group-piglet individuals in piggery is an important prerequisite for monitoring and tracking of group-piglet.With the development of deep learning,the instance segmentation framework further divides the same class of objects into different instances,which is on the basis of the semantic segmentation framework.The instance can not only realize the positioning of the target object,but also segment,classify and score the objects in the positioning box,providing theoretical and technical support for the segmentation of the individual adhesion of group-piglet.Mask R-CNN is an excellent instance segmentation algorithm.And this paper introduces an improved instance segmentation method based on Mask R-CNN,named Double FPNet.The features extracted by the network forward process come from the outputs of the second,third,fourth,and fifth residual blocks,which constitute the feature pyramid.The horizontal connection merges the upsampled result and the feature map of the same size generated by the network forward process,and predicts independently in each layer.Then,Double FPNet uses double-pyramid network based on Res Net50,realizing feature extraction and fusion for RGB images and depth images respectively.The result is then entered into the region to generate the network to extract the pre-selected anchor(Region of Interest,ROI),and the changed shared features into the Head network,including three branches.Calculate the three branch outputs of ROI target,including category,regression and mask respectively,and detect the positioning and classification segmentation results,The experimental results show that the improved double-pyramid network in this paper can effectively solve the influences of color similarity and adhesion for individual similar group-piglet,and achieve the complete segmentation of a single piglet area.The training samples of the network model were 2000 sets of images,and the training sets and verification sets were randomly divided according to 4:1 ratio.The experimental results show the segmentation accuracy up to 89.25 %,and the occupancy of training GPU is 77.57 %.Compared with Pig Net and Mask R-CNN segmentation methods,the segmentation accuracy of the proposed segmentation method was 3.85 and 14.25 percent higher,respectively.