A Multimodal Corpus-based Study On Metaphorical Gestures

Since the publication of Lakoff and Johnson’s (1980) landmark book Metaphorswe live by, metaphor is not primarily thought as a matter of language, but structuresboth thought and action, i.e., the nature of human’s thinking is metaphorical.Apparently, metaphorists considering themselves adherents of the Conceptual MetaphorTheory (CMT), if the conceptual nature claim of metaphor is correct, evidences shouldgo beyond other non-verbal modes, such as pictures, sounds, gestures and so on, thusled to the upsurge of multimodal metaphor pioneered by Charles Forceville (2006b).With the development of computer science, the technology of multimodal corpus isbecoming increasingly mature, in this thesis, we adopt the method of self-compiledcorpus analysis under the theory of conceptual metaphor as well as multimodalmetaphor, and select President Obama’s two inaugural addresses in2008and2013respectively, as well as his four national addresses raging from2010to2013to buildour own corpus, aiming to answer the following research questions in detail:(1) What are the ratios and characteristics of gestural types used in thespeech-gesture synchronization?(2) What are the types of metaphorical gestures according to metaphoricity of thetarget sentences they accompany? And what are the implications according tothe classifications?(3) What are the inner mechanisms and characteristics of multimodal meaningconstruction in the speeches? And what is the function of metaphorics inspeech-gesture synchronization?The thesis meets these research aims by an extensive study of relevant literatureand the implementation of quantitative study with the help of multimedia annotationsoftware ELAN4.6.2and qualitative research afterwards. Through detailed andscientific research, the research findings are presented as following:(1) The proportion of metaphorics (12%) is not the largest among the ratios of the total five types of gestures (beats,69%; metaphorics,12%; deitics,11%; indicates,7%;iconics,1%), but the other four gestural types are either stress target words (beats andindicates), or describe concrete entities (iconics), detics can present both concrete andabstract entities, but they do not contain cross-domain mapping process. Theimportance of metaphorics overweighs others in that they can present abstract entitiesand also they involve cross-domain mappings, so they worth further studies.(2) Metaphorics in the speeches can be divided as “metaphorics+metaphoricalutterances”(84%) and metaphorics+non-metaphorcal utterances (16%) according tothe metaphoricity of the target sentences they accompany. Metaphorics in the formertype can be further classified into “ontological metaphorics”(62%),“orientationalmetaphorics”(34%) and metonymics (4%), which proves that both the verbal metaphorand gestural metaphor have the identical mechanism in human’s cognitive system; inthe second type, metaphorics can be divided into “temporal metaphorics”(82%),“ontological metaphorics”(10%) and metonymics (8%), which confirms themetaphorical nature of human’s thought.(3) According to the qualitative analysis of the annotations, we find out that thereare three patterns of multimodal meaning construction:1)“Metonymy withinmetaphor” multimodal meaning making process (73%),2)“Metonymy-prominent”multimodal meaning making process (4%), and3)“A-ING IS B-ING” multimodalmeaning making process (23%). Through the classification and analysis of the threepatterns of multimodal meaning construction, we find out that: the target lexicalcategories in the first interactive pattern (nouns, verbs, verbal phrases, adjectives,prepositions, adverbials) are much more comprehensive than that of the other twopatterns, which shows it is the most frequently occur interactive pattern in multimodalmeaning construction; metonymies play an important role in multimodal meaningconstruction, they can either be used to form a similarity relation between the sourcedomain and target domain in the first interactive pattern, or to achieve its referentialfunction in the second pattern, and also they can be used to motivate metaphoricalmapping process level upon level in the third interactive pattern; such lexical categoriesas verbs, verbal phrases and nouns (76%) are most frequently occur words thataccompany metaphorics, which shows the importance of metaphorics in highlightingthe language focus. The functions of metaphorics can be viewed both from the side of thegesturer/addresser and the addressee/hearer. Through the perspective of the gesturer,metaphorics can raise consonance among the audiences and make their meanings fullyexpressed through the exbodiment of concept ideas; from the perspective of the latter,the hearer can get a better understanding of the multimodal meanings in speech-gesturesynchronization through the juxtaposition of both the visual and verbal modes.As to the above mentioned research findings, in the first place theoretically, theywill prove and further develop the present theory of conceptual metaphor as well asmultimodal metaphor, and also practically, for the overtness and consciousness ofmetaphorical expression, the speakers can use the concrete gestural modes to expressthe metaphorical keywords during the presentation of political speeches to display thelanguage focus. On the other hand, it will provide implications for language teachingsand learnings in China, that is more attention should be paid to the improvement ofstudents’ multimodal metaphorical abilities, which is helpful for the enhancement of“multiliteracy” as well as achieving greater effectiveness in communication.