Research And Implementation Of A CNN-based Polyphonic Piano Transcription Algorithm

Automatic Music Transcription(AMT)transforms the raw musical audio signal into symbolic notations,mainly including onsets,offsets and pitches of notes.It has widespread applications in music education,music search,musicological analysis of non-notated music and music enjoyment.However,there are still challenging problems for the polyphonic AMT.In this paper,we design a polyphonic piano transcription system based on Convolutional Neural Network(CNN).Firstly,time-frequency analysis of the original piano musical signal is performed.The raw audio signal will be read into dual-channel signals and then Constant-Q transform(CQT)will be performed on each of the channel,so the dualchannel spectrogram slice can be obtained as the input.Secondly,polyphonic onsets detection is carried out.We design the polyphonic onsets model to detect the onsets and pitches simultaneously.And in order to align the onsets of the parallel notes,the common onset model is proposed to detect the common onset for the parallel notes.Finally,polyphonic offsets will be detected.A frame-level pitches model is proposed to detect the offsets of the notes.The main works in this paper are following:(1)Investigate the related work at home and abroad and briefly describe the core technology of AMT.(2)Design and implement the polyphonic onsets detection module,and two CNN-based models are used to detect the onsets and pitches and the onsets alignment post-process operation is used to align the onsets of notes.(3)Design and implement the polyphonic offsets detection module,and a CNN-based model is used to detect the offsets and the model is optimized.MAPS dataset for AMT is used to train and evaluate.All the models are trained on the synchronized part of the dataset and evaluated on the real piano recordings(ENSTDkCl and ENSTDkAm).On the note-level results ignores offsets and requires that onsets be within ±50ms,the algorithm in this paper achieves a f1-measure of 85.15% on the ENSTDkCl subset and it is state-of-the-art.In addition,on the note-level results which requires that onsets be within ±50ms and offsets resulting in note durations within 20% of the ground truth,the algorithm in the paper achieves a f1-meausre of 55.28% on the ENSTDkCl and ENSTDkAm subsets and it is also state-of-the-art.