Inter-semiotic Cohesion Research Of Multimodal Resources On English Textbooks

With the development of technology and science,the design of the textbooks has changed from single-mode,such as written mode,to multi-modes,such as pictures,cartoons,charts,videos,multi-medias or combined them together.According to the curriculum reform and the needs of English learners,the discourse analysis only for the written mode in the textbooks can’t meet the demand of social development and learners,and multimodal discourse analysis will be an irresistible trend in the future.Therefore,this thesis aims to analyze and discuss the multimodal discourses based on the cohesion theory and inter-semiotic complementarity theory to develop students’ the multi-literacy.Multimodal discourse analysis is based on social semiotics.Later,visual grammar proposed by Kress and van Leeuwen was widely used in multimodal discourse analysis.Royce combined the concept of cohesion proposed by Halliday and Hasan and visual grammar when he analyzed and discussed written and visual modes,and proposed the inter-semiotic complementarity theory.This paper aims to answer the following questions:1.What are the features of textual cohesion and inter-semiotic relation between written and visual modes in the multimodal discourses?2.What are the features of transitivity system in the multimodal discourses?3.How do English teachers improve students’ multiliteracy based on inter-semiotic complementarity theory in the multimodal discourses?The research objects are from texts and images in the seventh grade’s textbook(B)of the SEP version.First,this paper aims to analyze the textual cohesion based on cohesion theory proposed by Halliday and Hasan.Second,the framework of visual message elements(VEM(s))aims to analyze visual modes by example analysis.Finally,the inter-semiotic complementarity theory proposed by Royce aims to analyze the cohesion between written and visual modes in multimodal discourses.The content analysis and quantitative research method are mainly used in this thesis.The actual data which are replaced with transcription and representation that could be easily calculated in the Office Excel Table is obtained by the coding method,and they are summarized and presented as tables in the thesis for further analysis and discussion.Some major findings of this study are presented as follow:1.Reference and repetition are widely used in the textbook,which are helpful to improve students’ reading and writing skills.By using the textbook reasonably,teachers can improve the teaching efficiency.Also,inter-semiotic repetition is mostly used in the textbook and that the written text along with visual modes can realize the whole meaning reconstructio n of the intended discourse.Therefore,visuals can be used in class to make the textual discourse easier to understand.2.Among all the processes,material processes for written mode and visual mode take up the most substantial proportion,which are coherent with each other.Teachers can improve students’ ability to think,understand and solve problems by using series of visuals.3.English teachers can explore the visual,linguistic,and different learning resources in the textbooks with their students,such as the PPT,videos,images,sounds,games,activities to improve their multimodal literacy.Also,English teachers must make use of different semiotic resources in specific contexts effectively,not limited to the images and written forms.At the end of the study,according to the findings and the teaching requirements of curriculum standard for multimodal discourses,the suggestions of multiliteracies’ cultivation,are presented and discussed in the thesis,such as reading and writing,listening and speaking.