A Multimodal Analysis Of Image-Text Relationship In Public Transportation Signs:Intersemiotic Translating Perspective

This is a multimodal study of public transportation signs from the perspective of intersemiotic translation.With the rapid development of information technology,multimodal text has taken the place of verbal text in people's daily communication.Studies of discourse analysis before usually give priority to textual structures,grammatical rules,sentences and clauses,while other forms of expression like image,sound,color and space have been overlooked.In the 1990 s,international linguistic community witnessed a boom in multimodal discourse analysis.But related work in China started late.As the first scholar to introduce the multimodal theory at home,Li Zhanzi elaborated on the theory of visual grammar created by Kress and van Leeuwen.Since then,multimodal research has made great progress in China.Many scholars,such as Hu Zhuanglin,Zhu Yongsheng,Zhang Delu,etc.,explored its theoretical basis and research methodology.On the other hand,researches on public signs mainly focus on the correction of translation errors,while images,as an important part of those signs,are seldom paid attention to.Therefore,the theory of intersemiotic translation is used to conduct a multimodal analysis on public transportation signs.This thesis mainly aims to analyze the structure of images and text in public transportation signs from the perspective of three metafunctions.Moreover,the characteristics of intersemiotic translation is analyzed for images and text in public transportation signs correspond one-to-one.This thesis mainly adopts a qualitative method.126 samples are collected from public transportation like road,subway station,coach terminal,etc.Some samples collected are blurry and distorted because of dim light or bad angles so substitutions of them are found on the Internet for better clarity.Under the guidance of semiotics and systemic functional linguistics,these samples are analyzed and the ways through which images and text realize metafunctions in public transportation signs are summarized.Apart from that,the term “intersemiotic translation” used by Jakobson is re-defined.During the research process,problems are found in the design of public transportation signs,along with some deep-rooted gender issues.Jakobson believes that there are three forms of translation: intralingual translation,interlingual translation,and intersemiotic translation.Intersemiotic translation refers to translation of verbal signs through non-verbal systems,or vice versa.Under the basis of Jakobson's theory and many other scholars' modification,it is believed that any process or behavior involving the transformation of signs should be deemed as intersemiotic translation.There are three forms of it: 1)translation within the same semiotic mode;2)translation from one semiotic mode into another;3)translation of a multimodal text into a single-modal(multimodal)text,or vice versa.Translation within images and text in public transportation signs belongs to the second category.Next,the understanding of image-text relationship is developed on the basis of separate studies on these two modes.This thesis uses systemic functional linguistics to analyze verbal text in public transportation signs,and uses visual grammar to analyze the images in them.Ideationally,there are mainly material,relational and existential processes in the text.Text in public transportation signs appears to be as concise and efficient as possible.Correspondingly,images in those signs are mostly in the form of vector graphs with minimum details.Narrative structures and conceptual structures are both used in those images.There are mainly action processes and geometrical symbolism in narrative structures,while reactional,speech and mental processes are seldom used.Conceptual structures usually employ classificatory processes,while analytical processes mainly appear in maps.Interpersonally,text in public transportation signs uses words like“?”(“please”),“??”(“suggestion”),“??”(“caution”)and“??”(“forbid;no”)to express different moods.While in images they use different colors and combinations of geometric shapes to do it.And textually,verbal text mainly uses unmarked themes,and only in a few special cases does it use marked themes.Similarly,information structures in most of the verbal text are unmarked.On the other hand,because different information value is attached to different positions,images in public transportation signs often place primary information in the middle and secondary information at the edge.They also use tools like color contrast and framing device to separate different elements.In the end,it is found that intersemiotic translation within public transportation signs is completely different from translation in a traditional sense.It does not seek to achieve equivalence,nor is it restricted by certain “grammatical rules”.A dynamic process can be translated into a static structure,and concrete ideas can also be translated into abstract forms.But generally,the image and the text are always related in one way or another.It is also noticed that a small proportion of public transportation signs related with concepts like “take care” and “children” always use female figures in the image while there's no gender difference in the verbal text,from which it can be seen that intersemiotic translation often carries the translator's subjective point of view.