Region Detection And Behavior Classification Of Dairy Cows Based On Machine Vision

With the increase in production and consumption demand of livestock production,the global livestock industry has to feed more animals with limited environmental resources and a shortage of livestock labor.In this situation,precision livestock farming plays an important role in achieving high efficiency with low cost in an environmentally sustainable way.To solve the urgent problems in the precision dairy farming and to further explore more efficient methods of capturing dairy cow information,region detection and behavior recognition of dairy cows based on machine learning have been studied.Based on domestic and international research,and aiming at realizing automatic,efficient and accurate dairy cow monitoring,intelligent and efficient methods were studied for cow wrist area detection,key body parts detection,interaction behavior recognition and video behavior recognition.The main research contents and conclusions of the thesis are as follows:(1)Research on region detection of cow wrist based on color edge and bilateral projection.The complexity of cattle environment,mud,excrement and other interference made it difficult to detect wrist.In order to detect wrist area of cows,the region detection method based on color edge and bilateral projection was presented.Firstly,the edge information of color images was got by color edge extraction.Secondly,most of the background was removed and the corbel area was obtained by using open operation,vertical projection and convex hull processing.Finally,the threshold processing and horizontal projection were performed to determine the target area center and obtained the target area.In order to verify the validity of the algorithm,K-means color clustering algorithm and salient region detection were adapted to compare the performances of the presented method.In this research,81 test samples were selected,and the results showed that OR(Overlap rate)increased 6.48% and 17.26% respectively when compared with results obtained by K-means and saliency-based methods,and the proposed method had good robustness in the case of background obstructions or ground reflections in images.The results of the present work implied that the presented method was effective to extract the target region,and could stimulate further testing of cow wrist swelling and scars.(2)Research on automatic detection of dairy cow region based on YOLOv4-SAM.In order to realize automatic,efficient and accurate detection of dairy cow parts,a deep learning network named YOLOv4-SAM was proposed,to achieve high detection precision of cow body parts in long-term complex scenes.The proposed YOLOv4-SAM consists of two components:YOLOv4 is for multiscale feature extraction,while the attention mechanism SAM highlights the key cow biometric-related features.By doing this,visual biometric feature representation ability is enhanced for improving cow detection performance.In order to verify the effectiveness of YOLOv4-SAM,a challenging dataset consisting of adult cows and calves with complex environments(e.g.day and night)was constructed for experimental testing.The proposed YOLOv4-SAM based approach achieved a precision of 92.29%,a recall of 96.51%,an F1 score of 94.37%,and an m AP@0.5 of 93.13%,which outperformed other methods of Faster R-CNN,Retina Net,and YOLOv4.Experimental results show that the proposed YOLOv4-SAM approach could capture key biometric-related features for cow visual representation and improve the performance of cow detection.In addition,the detected height difference between head and leg proved the capability in automatic identification of lame cows.The proposed deep learning-based cow detection approach provides a basis for developing an automated system for animal monitoring and management on commercial dairy farms.(3)Research on a machine vision-based method for monitoring scene-interactive behaviors of dairy calf.Requirements for animal and dairy products are increasing gradually in emerging economic bodies.However,it is critical and challenging to maintain the health and welfare of the increasing population of dairy cattle,especially the dairy calf(up to 20%mortality in China).In response to this problem,a machine learning method(i.e.,an integration of background-subtraction and inter-frame difference)was developed for automatically recognizing dairy calf scene-interactive behaviors(e.g.,entering or leaving the resting area,and stationary and turning behaviors in the inlet and outlet area of the resting area).Results show that the recognition success rates for the calf's science-interactive behaviors of pen entering,pen leaving,staying and turning were 94.38%,92.86%,96.85%,and 93.51%,respectively.The recognition success rates for feeding and drinking were 79.69% and 81.73%,respectively.This newly developed method provides a basis for inventing evaluation tools to monitor calves' health and welfare on dairy farms.(4)Research on the detection of typical behaviors of dairy cows based on Bi GRU-attention.In order to achieve high behavior classification accuracy,in this study,a Bi GRU-attention based approach is proposed to classify some main behaviors such as exploring,feeding,grooming,standing and walking.In our work,1)Inception-V3 network was firstly used to extract Convolutional Neural Network(CNN)features for each image frame in videos;2)Bidirectional Gated Recurrent Unit(Bi GRU)was used to further extract spatial-temporal features;3)the attention mechanism was deployed to allocate corresponding weights on each of the extracted spatial-temporal features according to feature similarity;4)the weighted spatial-temporal features were fed to Softmax layer for behavior classification.Experiments were conducted on two different datasets(i.e.calf dataset and adult cow dataset),and the proposed approach achieved 82.35% and 82.26% classification accuracy on datasets of calves and adult cows,respectively.In addition,by comparison with other methods,it was found that the proposed Bi GRU-attention approach outperformed Long Short-Term Memory(LSTM),Bidirectional LSTM(Bi LSTM),and Bi GRU.Overall,the proposed Bi GRU-attention approach can capture key spatial-temporal features to improve behavior classification ability significantly,which is favorable for automatic animal behavior classification in precision livestock farming.(5)Research on the detection of typical behaviors of dairy cows based on C3 DConv LSTM.Recognizing or classifying different behaviors with high accuracy is challenging due to the high similarity of movements among these behaviors.In this study,we propose a deep learning framework to monitor and classify dairy behaviors,which is intelligently combined with C3D(Convolutional 3D)network and Conv LSTM(Convolutional Long ShortTerm Memory)to classify the five common behaviors included feeding,exploring,grooming,walking and standing.For this,3D CNN features were firstly extracted from video frames using C3 D network;then Conv LSTM was applied to further extract spatial-temporal features,and the final obtained features were fed to a Softmax layer for behavior classification.The proposed approach using 30-frame video length achieved 90.32% and 86.67% classification accuracy on calf and cow datasets respectively,which outperformed the state-of-the-art methods including Inception-V3,Simple RNN,LSTM,Bi LSTM and C3 D.Additionally,the influence of video length on behavior classification was also investigated.It was found that increasing video sequence length to 30-frames enhanced classification performance.Extensive experiments show that combining C3 D and Conv LSTM together can improve video-based behavior classification accuracy noticeably using spatial-temporal features,which enables automated behavior classification for precision livestock farming.