| In order to meet the demand of musical instrument information in the field of music information retrieval and in the real world,deep learning has been applied to the musical instrument recognition tasks.This thesis focuses on the recognition of predominant instrument recognition,containing the effect of convolutional neural network,multi-task learning network structure,composite network and loss function of network training in the instrument recognition tasks.This thesis mainly includes five parts.As the baseline of the instrument recognition task,the ConvNet network used for instrument identification is introduced and analyzed in detail,including the dataset,the network structure,the training and test configuration and the evaluation parameters used in experiments.Two different features are extracted and used as input data,and the experimental results based on the ConvNet are used as the baseline for subsequent studies.Aiming at the problem that the onset types influence the precision and recall of the instruments obviously,the thesis focuses on the improvement of the ConvNet.The auxiliary classification is introduced into the ConvNet,the multi-task learning is realized by performing the auxiliary classification and the principal classification at the same time.Three grouping strategies are proposed as labels of the auxiliary classification through the analysis of music signals,including the grouping strategy based on the analysis of misjudgment rates,the grouping strategy based on musical instrument families and the grouping strategy based on the types of vibration.The multi-task learning structure can learn a more general representation,and there is less risk of overfitting and falling into local minima.For the loss function used in the network training,the center loss is introduced into the binary cross-entropy to reduce the intra-class spacing.To verify the effectiveness of the proposed improvement methods,the thesis designs a series of experiments to verify the effect of the proposed multi-task network.The optimal hyper parameters are chosen by the experiments,including the loss ratio between the auxiliary classification and the principal classification,the ratio between the center loss and the binary crossentropy,and the size of mini-batch.The micro and macro F1 measurements reach 0.685 and 0.597 after introducing the auxiliary classification,the batch normalization layer and the center loss,which are 10.7% and 16.4% higher than the baseline results separately.In the meanwhile,in order to observe the internal processing mechanism of the proposed model,the t-SNE algorithm is used to reduce the dimension of the data and make a visualization.By comparing the visualization results,it is obvious that the network structure proposed in this thesis has improved the aggregation effect compared with the baseline ConvNet network structure.In addition,the composite network structure based on the ConvNet is investigated.The fault-tolerant ability of the model is improved by focusing on only two musical instruments at a time.The experiments find that as the number of musical instrument categories increases,the composite network begins to shows better performance.In the aspects of features,the harmonic and percussive related features are studied.The possible roles of the harmonic and percussive audios in instrument recognition are analyzed and the reasons for not getting better recognition effect are inferred by the auditory perception of the audios which have been processed and observing the audios' spectra. |
