Research And Implementation Of Fish Feeding Intensity Classification Based On Multimodal Fusion

Feeding is a critical aspect in industrial recirculating aquaculture,and fish with different degrees of hunger show different feeding behaviors during feeding.The mainstream feeding method uses automatic feeding machine to feed regularly and quantitatively every day.This method does not take the feeding needs of fish into account and cannot automatically adjust the feeding amount according to the feeding behavior of fish,which will lead to the waste of feed and reduce the efficiency of aquaculture.Therefore,it is of great practical significance to realize accurate and efficient feeding method.Domestic and foreign research shows that the intensity of fish feeding behavior can reflect the degree of hunger of fish.How to accurately identify feeding intensity and automatically adjust feeding quantity becomes the key to realize accurate feeding.To solve the above problems,this study takes golden trout cultured in factory circulating water system as the research object,and uses deep learning algorithm to integrate the information of water quality,vision and sound modalities.Then,proposes a high-precision fish feeding intensity identification method called Fish-MulT,and develops a fish feeding intensity detection system and intelligent feeding system based on this method.By integrating data characteristics of different modalities,a more comprehensive and accurate grasp of fish feeding intensity can be achieved,and the intelligent level of feeding aspect can be greatly improved.The main work contents of this paper are as follows:(1)In this study,GoPro motion camera was used to collect video and audio data during fish feeding,and electrochemical water quality detection probe was used to collect water quality data(dissolved oxygen,temperature and pH value)during fish feeding.1293 groups of arrays were selected to make fish feeding intensity data sets,and the data sets were divided into four categories:"strong","medium","weak" and "none" according to previous experience.(2)The Fish-MulT algorithm is proposed in this paper to solve the problem that a single mode cannot capture global features,which leads to low recognition accuracy.Based on the Multimodal Transformer(MulT)algorithm,it is improved to accurately identify fish feeding intensity by capturing a series of water quality,visual and sound features changes.First,feature vectors are extracted from the input water quality,sound and visual data;Secondly,a Multimodal Transfer Module(MMTM)was used to fuse the input feature vectors,and adaptive weights were added to the three modes after fusion;Finally,the Cross-modal Transformer of each modal branch in MulT algorithm is optimized from 2 to 1.Experimental results show that compared with MulT algorithm,the accuracy of the model proposed in this paper increases by 2%to 95.36%,the number of model parameters decreases by 38%,and the training time per epoch is shortened by 29%.Compared with the water quality method(Function fitting),sound(GFCC+ResNet)and visual modality(SlowFast)alone,the identification accuracy is increased by 68.56%,21.65%and 3.61%,respectively.(3)Developed a fish feeding intensity detection system and intelligent feeding system:developed a fish feeding intensity detection system using PyQt.The detection system first receives the water quality,visual and sound information,then extracts the data features using the corresponding feature extraction method,then selects the identification algorithm.Finally outputs the corresponding feeding intensity label through the algorithm.The feeeding intensity detection system also contains training mode,which can train different data sets and models and add the trained models into the detection system.The intelligent feeding system is divided into four modules:Data acquisition and identification module,Internet of Things cloud platform,Data Transfer Unit(DTU)and driving circuit.The detection and feeding system can overcome the shortcomings of single modality in instability and low accuracy with the multimodal fusion identification method.In the case of fuzzy or interfered data of a certain modality,it can rely on other modalities to complete the identification of feeding intensity,which can provide reliable technical support for precision feeding.