An Audience Reception Study On The Multimodal Relations In Subtitled Films

The increasingly diverse forms of communication empowered by technological advancement over the past decades have brought ever-growing complexity in the activity of translation.A translator,especially when translating film subtitles,has to consider not only the linguistic symbols but other forms of communication such as images(visual mode)and sounds(aural mode),which could play an equally essential role in representing the meaning of the film.This study aims to shed new light on Dicerto’s(2018)framework of description and analysis for dynamic multimodal texts,and to examine the emerging role of multimodality in subtitle translation.By building a multimodal corpus to analyze and compare the verbal-visual relations of the English and Chinese subtitles from the film Coco and conducting an audience reception experiment on eighty Chinese teenagers,this study provides empirical evidence for the argument that the ultimate goal of a subtitle translator is to reconstruct a new relevant relation between the target subtitles and the non-verbal contents in the film.Results from the small-scale corpus show that the complementary verbal-visual relation,in contrast to the equivalent relation,is most frequently found in the film Coco,indicating that the meaning of most of the subtitles in the film are revealed by combining visual information on the screen.The subtitles and the images in a film are to a large extent complementary to each other.In the reception experiment,six hypotheses are proposed regarding the audience’s comprehension,cognitive load,and perception of verbal-visual congruence when viewing the film with target subtitles.Findings show that the addition of apposition relations in the target subtitles(i.e.more descriptive information provided)improves viewers’ comprehension while their self-reported cognitive load and perception of verbal-visual congruence are not significantly affected.Moreover,when more exophora relations are omitted in the target subtitles(i.e.fewer referential information provided),viewers do not perform significantly worse in comprehension,nor do they significantly rate higher scores regarding self-perceived cognitive load or rate lower in regard to verbal-visual congruence.As subtitle translation is intended for a different audience in a different context,it is recommendable for the translator to change the original multimodal relations or inferential meanings detected in the source language,so as to enhance the optimal relevance to the target audience.