Study On Pig Cough Sound Recognition Based On Muti-Domain Feature Fusion

The modern pig farming industry is developing in the way of intensification,which is likely to induce pig respiratory diseases and to increase the pressure of pig disease prevention and control.As one of the main clinical symptoms of respiratory diseases in pigs,coughing has become a key indicator for monitoring of respiratory diseases.At present,pig cough monitoring is mainly conducted by manual inspection.However,the traditional monitoring method is not only time-consuming and laborious,but also difficult to achieve continuous monitoring and to miss the detection of disease in the early stage.Audio analysis technology has sufficient feasibility and technical advantages in detecting pig coughing sounds.For this reason,it is potential and effective to identify pig coughs from the sounds collected from commercial pig houses.It is of great importance to achieve the auxiliary early warning of herd respiratory diseases and thus reduce the infection rate of respiratory diseases in pig farms.At present,the following issues exist in the application of audio technology to identify pig coughs: inadequate extraction of acoustic features in complex pig housing environments,lack of effective feature selection and fusion methods,and lack of research on pig cough recognition under unbalanced data sets.To overcome the above problems,this dissertation proposed a feature fusion method under different domains based on the extraction of traditional acoustic features and time-frequency representation features.Furthermore,a hybrid ensemble learning algorithm was proposed to achieve high-precision pig cough recognition under imbalanced datasets.The main work is as follows:(1)Pig cough sound recognition method based on the fusion of traditional acoustic features was studied.Firstly,multiple traditional acoustic features in the time domain(root mean square energy and zero-crossing rate),frequency domain(spectral centroid,spectral flatness,spectral bandwidth,spectral roll-off,spectral contrast and spectral flux)and cepstrum spectral domain(Mel frequency cepstral coefficient,MFCC)of different sounds were extracted in pig barns.According to the correlation and difference between multiple acoustic features,a recursive feature elimination method based on XGBoost was proposed to realize acoustic feature selection.At the same time,feature interpretation based on XGBoost-SHAP method was used to deeply explore the correlation between features and its influence on recognition accuracy under different domains.A gridsearch method was used replace the traditional empirical determination of parameters to optimize the classifier and compare the performance of pig coughing sound classification under three different machine learning algorithms: support vector machine(SVM),K-Nearestneighbor algorithm(KNN)and random forest(RF).The results show that the time-domain features and cepstral domain features have stronger discrimination in the recognition of pig coughing sounds compared to the frequency domain features.The Mel frequency cepstral coefficients contributed the most to the model output.The acoustic feature fusion subset combined with support vector machine identified pig coughing sounds with an accuracy of 94.46%,an improvement of 1.14%compared to the Mel frequency cepstral coefficient in a single domain.(2)Pig cough sound recognition based on the fusion of time-frequency representation features was investigated.Considering the compactness of information expression in both time and frequency dimensions,this paper further explores the two-dimensional time-frequency representation features were explored,including short-time Fourier transform(STFT)and constant Q transform(CQT).Then,the visual features and deep features of the time-frequency representations were further extracted.1)To further enhance the description of visual features,the local binary patterns under different image resolutions were extracted by varying the(P,R)values.And the fused LBP under different resolutions was proposed.At the same time,the strategy of fusing the histogram of oriented gradient(HOG)features under three-channel and single-channel was proposed by using the effect of the colour channel of the time-frequency representation.The results show that the fusion of HOG under multi-channel images achieves an accuracy of 93.98%for pig cough recognition.2)To further explore the deep features in the time-frequency domain,shallow convolutional neural network was constructed.Then,two feature fusion architectures based on single and bispectrogram input were proposed in combination with this model with the difference of element-wise method for deep feature fusion.The results show that the accuracy of pig cough recognition based on shallow convolutional neural networks outperforms that of three deep convolutional neural networks,VGG16,VGG19 and Resnet152.And the model computation time is much lower than that of deep convolutional neural networks,which is more suitable for pig cough recognition under small datasets in pig barns.Compared with the single-input spectrogram feature layer fusion method,the bispectrogram deep feature fusion method has better performance in pig cough recognition,with an accuracy of 94.51%.It can effectively capture the recognisable sound features in the barn.(3)Pig cough sound recognition based on multi-feature fusion was investigated.First,for the balanced data set,fusion methods based on traditional acoustic features and time-frequency representation features were investigated to construct multiple fusion strategies based on three fusion architectures.Also,concatenation fusion,canonical correlation analysis,and decision fusion were applied in the three fusion architectures for further comparison and analysis.It is found that the combination of Acoustic-TFRs fusion(A-TF)achieves the best accuracy of 97.57%in identifying pig coughs,with the shortest time.It indicates that this fusion of feature subset can enhance the expression of sound features in pig house and effectively improve the recognition accuracy of pig cough sounds.Secondly,by reducing the number of pig cough sound samples and increasing the ratio of non-cough sounds by cough sounds,the balanced dataset was divided into three degree of imbalanced datasets: mild,moderate and severe degree of datasets.And six sets of imbalanced datasets were constructed under the three degrees.The three feature subset of acoustic features,deep features of bispectrogram and the fused A-TF subset were adopted for comparative analysis of recognition performance on the unbalanced dataset.The results show that as the imbalanced degree deepens,the A-TF features achieve the best recognition results on all six imbalanced datasets.In particular,the F1 score of A-TF reaches 90.36%,which is 8.89% and8.31% higher than that of acoustic features and deep features of bispectrogram under severe imbalanced degree,respectively.It shows that the A-TF fusion features exhibit strong robustness under data skew,which provides technical support for the study of data imbalance problems in pig houses.(4)A hybrid ensemble learning algorithm for pig cough recognition was proposed.Considering the differences between heterogeneous and homogeneous models among features,ensemble learning could be used to combine them together.Firstly,the method designed differentiated and multi-layer base classifiers for keeping the stable performance of the base classifiers.Secondly,it performed secondary sampling and final decision on the output results of base classifiers.Comparative experiments were conducted on six imbalanced datasets.The method shows different degrees of recognition performance could be improved under six imbalanced datasets.And the F1 score improved to 92.19% when the data was heavily imbalanced,outperforming both the single SVM algorithm and the conventional data balancing algorithm.It shows that the method can overcome the recognition limitations of certain algorithm,fully exploit the advantages of the ensemble algorithm,and effectively improve the overall recognition performance of the model under imbalanced data sets.In summary,the extraction of multi-domain features and the combination of hybrid ensemble learning algorithms can effectively improve the performance of pig cough recognition in unbalanced datasets.To some extent,this study has overcome the problem of accuracy degradation caused by imbalanced datasets.Also,it is of academic significance and application value for achieving high accuracy pig cough recognition in complex pig housing sound environment.And it provides a theoretical basis and technical support for early monitoring of herd respiratory diseases in pigs.Overall,it provides a new idea for healthy pig breeding and precision management.