Synergy Of Different Modes In Dynamic Multimodal Discourse

In the age of the multimedia, there is an increasing awareness that meaning israrely made with language alone. The media and modes for communication havechanged greatly. People conduct communication through many kinds of modessimultaneously, such as space, gestures, gaze, posture, motion, sound, tone of voice,music, three-dimension objects, oral, written, image, tables, diagrams, pictures,etc.(Jewitt,2009:14) Each of them becomes an independent mode of communication.However, in many cases, language and other semiotic recourses realize meaningsimultaneously. People can communicate with each other through many kinds ofsensations, such as vision, audition, tactus, and through many kinds of semioticresources, such as language, image, sound, motion and so on.(Zhang Delu,2009a:24)Multimodal discourse has become an indispensable part of our daily use.Many scholars at home and abroad pay close attention to MDA in the recent20years. The scholars study MDA in many other fields beyond linguistics and withindifferent schools of linguistics. However, among the many schools of linguistics, theMDA from the socio-semiotic approach becomes the main trend which is the mainview of systemic functional linguistics developed by Michael Hailliday (1978,1994).But, so far, the study of MDA is still at its initial stage, especially in China; mostof the researches of multimodal discourse analysis focus on two dimensional staticimages (image, semiotics, tables, etc) by using the framework of Visual Grammar setup by Kress&van Leeuwen (1996); but there are seldom studies on the analysis ofother types of multimodal discourses such as dynamic discourse and threedimensional objects (classroom teaching, film, telecast, architecture etc.).From1990s, more and more MDA scholars from the systemic functionallinguistics school become interested in the interplay relations between different modes and the integration of the lexicogrammatical system of different modes in multimodaldiscourses. But there are some limitations in the present researches on the synergy ofdifferent modes in multimodal discourses both at home and abroad because althoughthe studies on the synergy of different modes in multimodal discourses increasesteadily, they are scattered and incomplete. Most of the researches focus on thesynergy of different modes in the static multimodal discourses and there are seldomstudies on the synergy of different modes in the dynamic multimodal discourses. Sothe research on the synergy of different modes in the dynamic multimodal discoursesis of great importance and significance to the development and perfection of thetheories, frameworks, and methodologies of MDA.Chinese TV weather forecast discourse is one of the dynamic multimodaldiscourses. But, so far, there is no research on the Chinese TV weather forecastdiscourse either from the linguistic perspectives or from the multimodal discourseanalysis perspectives; no research on the synergy of different modes in Chinese TVweather forecast multimodal discourse and no research on the systematic andoverarching theoretical framework which can be applied to this kind of dynamicmultimodal discourses.So the present study, Synergy of Different Modes in Dynamic MultimodalDiscourse: Chinese TV Weather Forecast, studies the synergies of different modes inthe dynamic multimodal discourse by taking Chinese TV weather forecast multimodaldiscourses which haven’t been studied from the MDA research perspectives as anexample. It is of great significance and importance and is expected to contribute a lotto the development and perfection of the theories, frameworks and methodologies ofMDA and can make up the vacancy in this research field.In order to analyze the TV weather forecast multimodal discourses conveniently,we design the Synthetic Theoretical Framework for Dynamic MDA for the ChineseTV weather forecast dynamic multimodal discourses on the basis of the synthetictheoretical framework of MDA set up by Zhang Delu(2009a) which is based on thefoundation of the theory of SFL and the Integrative Multi-Semiotic Model (IMM) proposed by Lim（2004:220-246）; collect28pieces of CCTV1night weather forecastvideo texts as the research data; and investigate the method for the transcription of thedynamic multimodal discourses, which transcribe the dynamic multimodal discourseinto Phases, sub-Phases (Episode) and Events based on the three criteria,1) Meaningor discourse semantics2) Experiential meaning3) The foregrounded things andfeatures for the transcription of dynamic multimodal discourse proposed by ZhangDelu (2010c) and the emic method. For the sake of analysis, Event is defined as thesmallest meaningful unit of analysis for the dynamic multimodal discourse, whichmakes it convenient to discover the distributions of the typical semiotic patterns inthis genre and the distributions of the distributions of different synergies in the corpus.The present study analyzes the corpus of28CCTV1night weather forecast videotexts on the basis of the foundation of the Synthetic Theoretical Framework forDynamic MDA and the methodology which have been investigated with the aim toexplore the synergies of different modes in Chinese TV weather forecast multimodaldiscourses. This research has two stages; and the Stage2is a complement to Stage1,as it overcomes the inherent defects of MCA in Stage1by presenting a finer anddown-to-earth analysis.Stage1aims to explore the distributions of different typical semiotic patternsused in this genre and the distributions of different synergies of different modes inCCTV1night weather forecast with quantitative and qualitative analysis methods.Stage2aims to investigate how and why different modes interact and cooperatewith each other to construct the discourse meaning in CCTV1night weather forecaston the basis of the findings of the first stage under the synthetic theoretical frameworkfor dynamic MDA.Based on the two stage analysis, the major findings include:First, in TV weather forecast, the broadcasters should choose differentcommunicative mode patterns to provide different information of weather conditionswith different communicative purposes. The communicative mode patterns can bedivided into three types: Presenting Beginning and End Type, Giving InformationType and Demanding Attention Type (D. A. Type). The Presenting Beginning and End Type includes two sub-types: Presenting Beginning Type (P. B. Type) and PresentingEnd Type (P. E. Type). Giving Information Type also includes three sub-types: LeadIn Type (L. I. Type), Giving Information turn to Pause Type (G. I. turn to P. Type) andGiving Information turn to Finish Type (G. I. turn to F. Type).Second, through the investigation of the synergies between the auditory mode asthe main mode and the visual modes which are realized by the bodily media and thenon-bodily media, it is found that the synergies between the auditory mode and thevisual mode of bodily media include Complementary relation of non-Reinforcementand Complementary relation of Reinforcement and the synergies between the auditorymode and the visual modes of non-bodily media include non-ComplementaryRelation of Context Interaction, Complementary Relation of Reinforcement andComplementary Relation of non-Reinforcement.Third, Through the analysis of the results of the distributions of the synergiesbetween the auditory mode and the visual mode of non-bodily media, it is found that,the Complementary relations get the most proportions both in the synergies betweenthe auditory mode and the visual modes of bodily media and in the synergies betweenthe auditory mode and the visual modes of non-bodily media, of which the proportionof78.63%is related to the synergies between the auditory mode and the visual modesof bodily media, and the proportion of77.04%is related to the synergies betweenauditory mode and the visual modes of non-bodily media. The non-Complementaryrelation of Context Interaction is just related to the synergies between auditory modeand visual mode of non-bodily media and gets the second proportion of22.96%. Inother cases, the disfunctional visual mode is related to the synergies between theauditory mode and the visual modes of non-bodily media and gets the least proportionof21.37%.Fourth, in the communication process, auditory mode as the main mode isrealized by the linguistic media: the broadcaster’s spoken language is in foregroundplace from the beginning to the end. When the synergies between the visual modesand the auditory mode are Complementary relations, the visual modes are in theforeground place. When the synergies between the visual modes and the auditory mode are Complementary relations of Reinforcement, either the visual mode of bodilymedia or the visual modes of non-bodily media are in the foreground place. When thesynergies between the visual modes and the auditory modes are Complementaryrelations of non-Reinforcement, both the visual modes of bodily media or the visualmodes and non-bodily media are in the foregrounding place. When the synergiesbetween the visual modes and the auditory modes are non-Complementary relationsof Context Interactions, the visual modes are in background place.To sum up, the present study puts forward the Synthetic Theoretical Frameworkfor Dynamic MDA and the transcription method for the analysis of dynamicmultimodal discourses; investigates the characteristics of the distributions of thesynergies of different modes in CCTV1night weather forecast multimodal discourses;and discusses how and why the different modes interact and cooperate with each otherto construct the discourse meaning in CCTV1night weather forecast. It enriches andimproves the studies on MDA both practically and theoretically and provides a newperspective for the analysis of the dynamic multimodal discourses.