| We are generally exposed to multisensory signals across the lifespan.Only when these signals are successfully resolved into a veridical perception,can we have effective communications with the world.What we hear is usually influenced by what we see.Previous research has indicated that visual information from a talker’s head,face,mouth,and hand gestures aids auditory speech perception in congruent audiovisual integration but changes speech perception in incongruent audiovisual integration.This crucial process of multisensory integration in speech perception has been proved to alter as the age increases,yet the aging mechanisms underlying audiovisual speech perception in complex contexts(e.g.,the noisy background and fast speech)are not well explored.Particularly,with the rapid increase of aging population in China,it is urgent to tackle these problems that Mandarin-speaking older adults meet in face-to-face speech communication.Numerous studies have targeted younger adults’ audiovisual speech perception at the segmental level of speech(i.e.,consonants and vowels).Nevertheless,the current study focused on investigating the aging mechanisms underlying audiovisual integration in Mandarin tone perception in older adults in complex and variable contexts in order to figure out the compensating mechanisms that older adults adapted to for the degeneration in their auditory and cognitive functions in speech perception.This study also provided novel evidence for the Theory of Embodied Cognition,the Compensation Hypothesis,as well as the Framework for Understanding Effortful Listening model.This dissertation mainly explored how Mandarin tone perception in older adults was influenced by the aging in complex and variable contexts.It addressed three specific research questions: 1)During the stable period of auditory and cognitive development,what could the overall situation of audiovisual Mandarin tone perception in Chinese younger adults be in complex contexts? 2)During the aging period of auditory function,whether and how would audiovisual Mandarin tone perception in Chinese older adults be affected by hearing attenuation in complex contexts? 3)During the aging period of cognitive function,whether and how would audiovisual Mandarin tone perception in Chinese older adults be influenced by cognitive degradation in complex contexts? In response to three research questions,this study focused on examining the impact of visual cues,altering speech rates,and the intrinsic attributes of tone categories on audiovisual Mandarin tone perception in older adults in the quiet and noisy contexts.A total of 149 participants(i.e.,53 younger adults and 96 older adults),all of whom speak Mandarin Chinese as their native language,were recruited to participate in this study.All participants’ hearing sensitivity,cognitive ability,working memory,and visual acuity were assessed before the formal experimental task was presented to make sure that all participants satisfied the requirements of experiments.All participants completed a Mandarin tone perception task with 720 tone stimuli in auditory-only(AO),auditory-facial(AF),and auditory-facial-plus-gestural(AFG)modalities,at three different speech rates(i.e.,fast,normal,and slow speech rates)under the conditions of the quiet and multi-talker babble noise.In AF and AFG modalities,both congruent and incongruent audiovisual cues were designed and presented.In each experiment,the accuracy of lexical tone perception across different conditions was analyzed by generalized linear mixed-effects model in R,with fixed effects(i.e.,modality,speech rate,tone,group,congruency,and context),random effects(i.e.,participants and stimuli),and covariates(i.e.,working memory,hearing level,and cognitive ability).The first experiment,a baseline experiment of this study,focused on examining the overall situation of Chinese younger adults’ audiovisual Mandarin tone perception in complex contexts and its influencing factors from the perspective of the Theory of Embodied Cognition.The general findings in Experiment 1 showed that the overall situation of audiovisual Mandarin tone perception in young adults tended to be stable,and their identification of Mandarin tones was influenced by multi-talker babble noise,visual cues,varying speech rates,and the inherent attributes of tone categories.The specific results demonstrated that in Mandarin tone perception,the magnitude of enhancement of AF and AFG cues across three speech rates was significantly higher than that of the AO cue in the adverse context of noise,yet supplementart metaphoric gestures didn’t show significant differences from the facial information.Furthermore,the performance of auditory tone perception at the fast speech rate was significantly better than that at the normal speech rate when the inputs were incongruent between auditory and visual channels in the quiet context.Additionally,the inherent properties of tone categories played a predominant role in tone perception in noise.Based on the design paradigm of Experiment 1,Experiment 2 investigated that how audiovisual Mandarin tone perception in Chinese older adults could be affected by hearing attenuation in complex contexts during the aging period of auditory function.The results of the second experiment found that when perceiving lexical tones,older adults with hearing impairment and hearing normal(i.e.,OHI and ONH adults)benefited from the audiovisual integration across fast,normal,and slow speech rates in the quiet and noisy contexts to a greater extent relative to younger adults with normal hearing(i.e.,YNH adults).However,the contribution of congruent audiovisual integration to YNH adults could be observed only under the condition of noise.OHI and ONH adults made use of facial cues and metaphoric hand gestural information,which was different from YNH adults who did not take advantage of metaphoric hand gestures.As to speech rate,OHI and ONH adults were weaker in perceiving and processing lexical tones at the fast and slow speeds than YNH adults.Particularly,fast and slow speech rates had the greater impact on OHI adults’ tone perception,compared with ONH adults.In other words,there were more degradation in OHI adults than in ONH adults when identifying audiovisual Mandarin tones under the conditions of fast and slow speech rates.However,fast speech rate had a positive impact on OHI and ONH adults’ lexical tone perception with incongruent audiovisual information in the quiet context,which could attribute to the attention transformation caused by incongruent audiovisual information.As for tone categories,the intrinsic properties of individual tones still played a predominant role in OHI and ONH adults’ audiovisual lexical tone perception in the quiet context,which was different from YNH adults’ perception.However,the influencing degree of the intrinsic properties of tone categories on older adults’ audiovisual Mandarin tone perception became weak in the adverse environment,which indicated that OHI and ONH adults’ ability of perceiving the inherent characteristics of individual tones had degraded.The findings of Experiment 2 supported the Compensation Hypothesis.Based on the design paradigm of Experiments 1 and 2,Experiment 3 explored that how audiovisual Mandarin tone perception in Chinese older adults could be influenced by cognitive degradation in complex contexts during the aging period of cognitive function.The results of Experiment 3 showed that older adults with mild cognitive impairment and normal cognition(i.e.,OMCI and ONC adults)benefited more from multisensory integration than younger adults with normal cognition(i.e.,YNC adults).Both OMCI and ONC adults relied on metaphoric hand gestural information in the quiet and noisy contexts to a large degree.Moreover,OMCI adults made use of less metaphoric hand gestural information than ONC adults in the quiet context,which revealed that OMCI adults’ ability of integrating facial cues and auditory acoustic information in the adverse environment had degenerated more than ONC adults’.With respect to speech rate,OMCI and ONC adults’ abilities of identifying audiovisual Mandarin tones in the quiet and noisy contexts could be affected by altering speech rates.The effects of congruent audiovisual information decreased as the speech rate speeded up and increased as the speech rate slowed down to a proper extent.However,OMCI and ONC adults’ abilities of identifying lexical tones was modulated and improved by fast speech rate under incongruent audiovisual modalities in the quiet context.Besides,OMCI adults’ performance of perceiving variations of speech rates had degenerated worse than ONC adults in both quiet and noisy contexts.With regard to tone categories,different from YNC adults’ perception,the intrinsic properties of individual tones played a dominant role in lexical tone perception in OMCI and ONC adults in the quiet environment.However,the influencing degree of individual tone categories on OMCI and ONC adults’ audiovisual lexical perception became degenerated in the adverse environment.The findings of Experiment 3 testified the FUEL model.In sum,this research topic is beneficial to clarify the relationship between the aging and audiovisual lexical tone perception.This study provided the compelling and novel evidence,indicating that integrated audiovisual information,as an effective compensating mechanism,plays a vital role not only in improving older adults’ lexical tone perception in noise,but also in modulating the effects of speech rate and tonal intrinsic properties on Mandarin tone perception in older adults in the quiet context,especially for those with hearing and cognitive impairment.This cross-sectional study,supporting the Theory of Embodied Cognition,the Compensation Hypothesis,and the FUEL model to a great extent,has proved that the aging of speech perception can be improved through multimodal integration and adjustment of speech rate,which may be facilitative for normal older adults’ daily communication as well as for speech and hearing rehabilitation training among older adults with hearing loss and mild cognitive impairment. |
PDF Full Text Request