A Study Of The Fusion Technology Of Sound And Visual Features For Fish Behavior Analysis

Precision farming is a new trend in the development of aquaculture field,precision feeding is an important element of precision farming,and precision feeding based on fish behavior is a new direction of precision feeding.In the real aquaculture environment,in order to solve the problem of low accuracy and recall rate of single-modal fish behavior recognition under complex conditions such as dim light,sound and visual noise interference,research on fish behavior model recognition based on sound features and fusion technology of sound and visual features was carried out,and the specific research contents and innovation points are as follows:(1)Fish behavior recognition based on MFCC and Res Net.Aiming at the problem of low accuracy of fish behavior recognition by means of computer vision under extremely dark or no light conditions,we proposed a method to recognize fish behavior by using sound signals.Mel frequency cepstral coefficient(MFCC)feature coefficients were used to represent fish behavioral acoustic signal features;to effectively learn fine-grained acoustic features of different fish behaviors,Residual Neural Network(Res Net)was used to integrate low-dimensional detail features with high-dimensional semantic features,so as to better ensure feature integrity and improve recognition effect.In order to verify the effectiveness of the proposed method,contrast experiments were designed,and the data collected by the Fish Behavior Laboratory of Dalian Ocean University were used to verify the effectiveness of the algorithm,and the experimental results showed that the accuracy,recall and F1 value of fish behavior recognition reached 99%;the fish behavior recognition method based on MFCC and Res Net can effectively.The fish behavior recognition method based on MFCC and Res Net can effectively recognize the swimming,feeding and other behaviors of fish.(2)U-Fusion Net,a fish behavior recognition model with multi-level fusion of sound and visual features,is proposed to solve the problem of low accuracy and recall of single-modal fish behavior recognition under complex conditions such as dim light,sound and visual noise interference,etc.The fish behavior recognition model with multi-level fusion of sound and visual features,U-Fusion Net,was proposed to exploit the complementarity of sound and visual features of fish behavior.To ensure the integrity of the fused features,a Skip-Concat module was proposed to obtain the potential information between the modalities under different perceptual fields and to fully fuse the modal features under different perceptual fields.To enhance the recognition effect after fusion,SENet was introduced to form a fusion network of attention channel information features,which condenses the fused information while retaining the local information to the maximum extent.To verify the effectiveness of the proposed method,a comparison experiment was designed to verify the effectiveness of the algorithm with the synthetic experimental data of multimodal fish behaviors,and the experimental results showed that the accuracy of U-Fusion Net for fish behavior recognition reached 93.71%,the F1 value reached 93.43%,and the recall rate reached 92.56%;compared with the existing model IFL deep model with better results,the recall rate,F1 value model,the recall,F1 value and accuracy rate were improved by 2.35%,3.45% and 3.48%,respectively.It is verified that the proposed method can effectively identify the swimming and feeding behaviors of fish under complex conditions.