Study On Melody Extraction And Multi-pitch Estimation For Polyphonic Music

Music information retrieval includes music content analysis,genre classification,query by humming,music recommendation,and so on.It has been widely used in network music,mobile terminal,consumer electronics,game and entertainment in recent years.Melody extraction and multi-pitch estimation are two important topics in music information retrieval.Much work has been done,and important achievements have been achieved.However,their performance still cannot satisfy the practical requirements,and there are still many problems that need to be addressed further.This dissertation applies the techniques,such as time-frequency analysis,Euclidean algo-rithm,staged optimization,Bayesian filter,particle filter,signal dimension extension and so on,to study melody extraction and multi-pitch estimation.The contributions are summarized as follows.(1)The fundamental frequency is sometimes absent due to lower frequency accompani-ment,special singing strategy or some specific instruments.In this case,it is difficult to extract melody from the mixture by tracking the fundamental frequency.To address this problem,the Euclidean algorithm for calculating the greatest common divisor of two natural numbers is gen-eralized to float numbers,and the modified Euclidean algorithm(MEA)is proposed.Then,the pitch candidate can be obtained according to the overtones by this algorithm,and a melody ex-traction method based on the modified Euclidean algorithm is proposed.In this method,the short time Fourier transform and instantaneous frequency correction are adopted to obtain the s-inusoidal components.Then,the modified Euclidean algorithm is used to obtain the frame-wise multiple pitch candidates using the spectral peak pairs.Finally,the melodic pitch sequence is achieved according to the pitch contour duration and continuity characteristics.The proposed method can estimate the pitches without relying on fundamental frequencies.However,the short-time severe frequency shifting during one note duration sometimes occurs.To address this problem,the melody extraction combining modified Euclidean algorithm and dynamic program-ming is proposed.In this method,the modified Euclidean algorithm is adopted to estimate the frame-wise pitches,and the object function describing melodic pitches is solved recursively us-ing the dynamic programming to obtain a smoother melodic pitch contour,so that the short-time dramatic changes are reduced greatly.The experimental results show that this method efficient-ly estimates the melodic pitch in the fundamental frequency missing case,and overcomes the short-time severe frequency shifting.(2)As the melodic pitch,which progresses with time,has the sequential relevance prop-erty,melody extraction is modeled in the Bayesian framework.Particle filter is adopted to obtain the effective approximate solution,and the melody extraction based on particle filter and dynamic programming is proposed.In the particle filter stage,the pitch transition probability is described by logistic distribution,and the likelihood function is constructed considering the pitch salience,spectral smoothness and timbre similarity.The posterior probability density of melodic pitch sequence is estimated recursively,and the coarse melodic trajectory is obtained.In the dynamic programming stage,the coarse melodic contour is smoothed to confine the frame-level melodic pitch ranges,and the melodic pitch sequence is tracked by recursively solving the objective function describing melodic pitches.This method needs no prior information,and the two-stage strategy narrows the searching range of melodic pitches while reducing the dy-namic programming computational complexity.The experimental results demonstrate that the proposed method achieves higher accuracies.(3)As harmonic overlap is ubiquitous in music,the missing and error rates of multi-pitch estimation are high.To address this problem,a pseudo two-dimensional(2-D)spectrum is proposed,some related properties are derived,and the multi-pitch estimation method based on the pseudo two-dimensional spectrum is proposed.In this method,the one-dimensional music signal is first mapped into the two-dimensional frequency plane using the pesudo 2-D transform.Then,the two-dimensional pattern matching is done by calculating the cross correlation between the 2-D amplitude spectrum and the 2-D template,and the preliminary estimates are obtained.Finally,the outliers are removed,and the remained estimates are refined and complemented according to the pitch histogram using the estimates of neighboring frames.In this method,the notes with harmonic overlap due to harmony can be separated efficiently,the 2-D harmonic pattern can also be recognized when there are few harmonics,the computational cost is low.The experimental results show that the proposed multi-pitch estimation method achieves higher precision and recall rates comparing with the reference methods.