Research On The Neural Differential Representation Of Voice Pitch Contours In The High Unresolved And Low Resolved Harmonics Of Mandarin Tones In The Ascending Central Auditory Pathway

Pitch plays a vital role in music,discrimination of talker and understanding speech.Pitch is time-varying in tonal language,and it conveys the meaning of the word.Accurate pitch perception is really important for communication and social life.Pitch of music and speech is mostly contributed by the harmonics.If the harmonics could be separated out by the basilar membrane,and these harmonics are said to be‘resolved’,and if the harmonics could not be separated out from their neighbors on the basilar membrane,and these harmonics are said to be“unresolved”.The resolved and unresolved harmonics are both important for good pitch perception.However,people with hearing loss and aging have trouble with pitch perception,which is possibly caused by the broadening of the auditory filter and poor frequency selectivity,they could not depend on the resolved harmonics for pitch perception.Chinese cochlear implant users also have poor tone recognition ability.One of the possible reasons is that there are only 24 channel electrodes in the cochlea and the electrodes placed at the high-frequency region of the cochlear tonotopic structure,they could not perceive the low frequency.Due to the absence of the resolved harmonics,poor music perception and tone perception in tonal language for people with hearing loss,aging people and cochlear implant users.It is a challenge for improving the current technology to solve the problems.Their poor pitch perception implies that the resolved harmonics might be more important for pitch perception.Although there is evidence from psychoacoustic studies about the resolved harmonics are more important for time-varying pitch perception,and the noise more robust.However,the evidence from electrophysiology is also necessary.To improve the ability of cochlear implant users and hearing loss people,we need to understand how the auditory system code the resolved and unresolved harmonics.Therefore,to explore how the auditory system represents the time-varying pitch contours in the resolved and unresolved components of Mandarin tones are of great importance for clinic application.The evidence from the psychophysics is the behavior after the auditory system coding the speech.The sound arrives at the outer ear,and the basilar membrane would transfer the mechanical signal to the electrical activity of auditory nerve fibers,the electrical signals coded by the auditory nerve is sent via the cochlear nucleus and into the central auditory pathways to the cortex.The signals along this pathway are processed and analyzed,and then we can perceive the sound.Neurophysiological studies showed that auditory nerve coding the harmonics by the temporal coding,which is the auditory nerve phase-locked to the pattern of the basilar membrane.However,the response patterns for auditory nerve coding the resolved and unresolved harmonics are different.Studies in the auditory nerve show that neurons can phase-lock to individual harmonics（temporal fine structure）of complex tones that are aurally resolved（components with harmonic numbers from 1-5）,and to envelopes of groups of harmonics that are unresolved（harmonic number>10）.The periodicity pitch information coded by auditory never for resolved harmonics is more robust than unresolved harmonics.The temporal information coded by the auditory nerve is sent into the auditory brainstem,and it might be represented by periodicity or other forms of representation in the auditory brainstem.The auditory cortex receives the information from the auditory brainstem,integrate and process the information.Compared to stationary synthetic complex tones used in the current studies,the periodicity pitch of Mandarin speech is time-varying.However,it is not yet clear how periodicity pitch information related to resolved and unresolved harmonics in speech,especially in tonal languages such as Mandarin,is processed in the ascending central auditory pathway,from the auditory brainstem to the auditory cortex.The aim of this study is to explore how the auditory system represents the time-varying contours in the resolved and unresolved harmonics of Mandarin tones.The study included four parts:To explore whether noise has different effects on tone recognition performance of stimuli with resolved harmonics（‘resolved stimuli’）vs.stimuli with unresolved harmonics（‘unresolved stimuli’）,the tone recognition was tested by an adaptive procedure.Mandarin monosyllables with tonal voice-pitch contours were low-pass filtered(F_LP at 800 Hz)to generate resolved stimuli,and high-pass filtered(F_HP at 2000Hz)to generate unresolved stimuli.Speech-shaped noise was used to balance masking between resolved stimuli and unresolved stimuli.The adaptive procedure estimated the signal to noise ratio（SNR）corresponding to 79.4%correct recognition.The results showed that the SNR of resolved stimuli was significantly lower than unresolved harmonics,showing that resolved stimuli are more robust in noise than unresolved stimuli.To explore how the auditory brainstem responds to resolved and unresolved stimuli in quiet and noise,we recorded auditory evoked frequency-following responses（FFR）.One vowel with a rising tone was selected.The speech sound was filtered to create four stimuli:resolved and unresolved,both in quiet and in speech-shaped noise.FFRs were recorded to alternating-polarity stimuli and were added or subtracted to enhance the neural response to the envelope(FFR_ENV)or temporal fine structure(FFR_TFS),respectively.Neural representation of F0 strength,as reflected in the FFR_ENV,was evaluated by the peak autocorrelation value in the temporal domain and the peak phase-locking value（PLV）at F0 in the spectral domain.Both evaluation methods showed that in the FFR_ENV that noise significantly reduced F0 strength for unresolved stimuli,but not for resolved stimuli.Neural representation of temporal fine structure as reflected in the FFR_TFS was assessed by the PLV at the harmonic near to F1（4th harmonic of F0）.The PLV at harmonic near to F1（4th of F0）of FFR_TFS to resolved stimuli was significantly larger than to unresolved stimuli.Spearman’s correlation coefficient values showed that the FFR_ENV F0 strength of unresolved stimuli was significantly correlated with tone identification performance in noise.These results also showed that the FFR_ENV F0strength to speech sounds with resolved stimuli was not affected by noise.In contrast,F0-related responses to unresolved stimuli were significantly smaller in noise compared to quiet.Our results suggest that the encoding of resolved harmonics is more important than that of envelopes for tone identification performance in noise.The FFR reflects neural activity that synchronized over an entire population,with the main contributions coming from the inferior colliculus（IC）.The IC,located in the upper brainstem,is a key station in the auditory pathway,receiving ascending inputs from the cochlear nuclei,olivary complex,and lateral lemniscus as well as descending projections from auditory thalamus and cortex.Previous studies showed that FFR may reflect pitch-bearing information,but that the FFR itself is not a direct representation of pitch processing in the IC.Previous studies reported that the auditory nerve and cochlear nucleus can encode periodicity pitch through phase-locked neural firing patterns.However,little is known about how the IC utilizes this timing information to represent the time-variant periodicity pitch of natural speech.To explore how the IC represents the time-variant periodicity pitch of natural speech,we recorded IC neuron in guinea pig to Mandarin tones.In this study,the Mandarin syllable/ba/pronounced with four lexical tones（flat,rising,falling then rising and falling voice pitch contours）were used as stimuli.Local field potentials（LFPs）and single neuron activity were simultaneously recorded from 90 sites within contralateral IC of six urethane-anesthetized and decerebrate guinea pigs in response to the four stimuli.LFPs reflect dendritic currents and to a lesser extent the spiking activity of local neuronal ensembles.Analysis of temporal information in LFPs showed that 93%of the recorded LFPs exhibited robust encoding of periodicity pitch.F0-related response periodicities in LFPs,as derived from autocorrelograms,were significantly stronger for rising tones than for flat and falling tones.F0-related response magnitudes also significantly increased with characteristic frequency（CF）.Spike timing patterns in roughly half（47%）of the single neuron recordings were significantly synchronized to the fundamental frequency of the stimulus.The results suggest that the temporal spiking pattern of single IC neurons provide a robust representation of the time-varying periodicity pitch of speech.The difference between the number of LFPs and the number of single neurons that were found to encode the time-variant voice pitch supports the notion of a transition at the level of IC from direct temporal coding in the spike trains of individual neurons to some other form of neural representation.To explore whether features of the CPR correlate with the pitch salience of resolved and unresolved harmonics of speech when the temporal periodicity is identical,and whether CPRs could be a neural index for auditory cortical pitch processing,we recorded CPR to resolved stimuli and unresolved stimuli.In this study,CPRs were recorded to two speech sounds:a set including only resolved harmonics and a set including only unresolved harmonics.Speech-shaped noise preceding and following the speech was used to temporally discriminate the neural activity coding the onset of acoustic energy from the onset of the time-varying pitch.Analysis of CPR peak latency and peak amplitude（Na）showed that the peak latency to resolved stimuli was significantly shorter than unresolved stimuli（p=0.01）,and that peak amplitude to resolved stimuli was significantly higher than unresolved stimuli（p<0.001）.Further,the CPR peak PLV in response to resolved stimuli was significantly higher than unresolved stimuli（p<0.001）.Our findings suggest that the CPR changes with pitch salience and that CPR is a potentially useful indicator of auditory cortical pitch processing.In summary,the thesis explored neural responses to stimuli consisting of resolved and unresolved harmonics at the levels of the auditory brainstem,the midbrain and in the cortex.Psychophysical experiments demonstrated that the resolved harmonics are more important for tone recognition than unresolved ones,especially in noise.Neural correlates of these differences were observed at both midbrain and cortical levels.Our results might provide a new experiment method and solution for objective measurement of the time-variant pitch contour in resolved and unresolved harmonics.