Research On Compound Electronic Musical Instrument Based On Computer Vision And Spectrum Synthesis

Since the world's first electronic musical instrument was invented at the end of the 19th century,the performance and quality of electronic musical instruments have been continuously improved,and it has been widely popularnzed.The keyboard is light and compact,and its tone varies,making it a favorite among music lovers.However,the keyboard is only a simulation of traditional instruments,and it is not a substitute for traditional instruments.How to further enhance the performance and expressiveness of the keyboard is a problem that needs to be solved in the future development of the keyboard.In particular,many of China's national musical instruments are stringed instruments,lacking their own sound source chips.How to achieve accurate MIDI recording and accurate replay in the case of continuous changes in the pitch frequency of stringed instruments is a difficult problem.In this paper,the electronic piano is explored from three aspects:the spectrum synthesis sound technology of the keyboard,the structure form of the keyboard and the MIDI notation method of the string instrument.The research work and results completed by the thesis are summarized as follows.The paper designed a composite electronic keyboard based on computer vision.In terms of hardware,the composite keyboard is made of a plexiglass box,and the piano surface is illuminated by an plexiglass plate with sidelight illumination,and the position of the finger is monitored by a wide-angle camera.The software of the composite keyboard is developed in the Visual Studio 2015 development environment with the OpenCV library function for image processing,which provides the centroid position of the fingertips in real time.In the keyboard instrument playing mode,the system calls the traditional MIDI function for sounding,notation and performance;in the stringed instrument playing mode,the system calls the enhanced MIDI function proposed in this paper for notation,and performs accurate sounding and performance through spectrum synthesis.Thereby achieving the unification of stringed instruments and keyboard instruments.The paper carried out physical modeling and MATLAB simulation on stringed instruments,and further performed Fourier spectrum analysis on several typical instruments,and confirmed the overtones whose pitch and vibration frequency are integer multiples of the fundamental frequency,and the sound patterns determined according to the spectral envelope.By performing spectrum synthesis,the original sound waves can be restored to a large extent,but the difference in sound quality between the two can still be heard.Further papers experimentally investigate the spectral contribution of the neighborhoods around the pitch and overtones,and sum up the double ten rules:for most instruments,just take the top ten harmonic spectrum including the pitch,and take the positive and negative around each overtone.The spectrum within ten Hz is spectrally synthesized according to the tone pattern determined by the spectral envelope,which can restore the original sound wave more accurately,and almost no difference between the two.For some instruments with a wide spectral peak,the spectrum of the spectrum around each harmonic can be appropriately expanded when performing spectral synthesis vocalization.This paper proposes an enhanced MIDI notation method for continuous vocal string instruments.This method first finds the standard pronunciation of the nearest neighbor that is slightly lower than the music frequency when the string is playing a certain tone(assuming that the frequency difference between the two is m Hz),and using this standard phonetic name,according to the traditional MIDI instructions for notation.Further,add another MIDI command,but the new command makes two changes.First,the phonetic number becomes m.Second,the timestamp adds a large time constant,so that the timestamp points to the whole A moment after the music is played.In this way,if you use a traditional MIDI player during playback,the new command'will not affect the performance of the current standard name after the whole song is played,but the performance is slightly worse;if you use the enhanced MIDI according to this method The player,after finding the long time stamp,automatically subtracts a large time constant,and extracts the sound name parameter m of the instruction,and adds m to the pitch frequency corresponding to the standard sound name to obtain an accurate pitch frequency.And then according to the aforementioned double ten rules for spectrum synthesis and vocalization,the accurate sound of the stringed instrument can be realized.