Relating visual images to Chinese subtitle translation in English language films: A multimodal study

Abstract

Audiovisual translation not only involves meaning transmission at the linguistic level, but closely relates to the interplay among a range of non-verbal modes. Recently, semiotic interrelations have been recognized as significant elements that have influenced the subtitling process. Focusing on the role of semiotic interplay in subtitling, this study draws upon Systemic Functional Linguistics (SFL)-informed multimodality and semiotic translation to examine Chinese subtitle translation in ten English language films with the aim of exploring the semiotic interplay between the visual and the verbal modes. This study examines three metafunctional meanings in subtitle translation with three research foci, i.e. the main types of cross-modal interrelations, the primary function of the semiotic interplay and the key linguistic components influencing the subtitles. This study arrives at the findings in three main areas. First, subtitling should relate to filmic units beyond stills or shots and consider all the filmic units such as scenes, sequences, stages and even the whole film. In this way, the semiotic interplay in subtitle translation ensures the holistic entity of subtitled films in the construction of the compositional meanings. Second, subtitling should facilitate film viewers’ comprehension by building up a close and direct relation between viewers and subtitled films through synchronization between the visual and the verbal information. This point is primarily demonstrated in the interactive meanings in subtitling. Third, one of the key functions of semiotic interplay in subtitle translation is to maintain the narrative flow. This is demonstrated in the representational, compositional and interactive meanings of the subtitle translation. In sum, this study goes beyond traditional textual analysis in translation studies and addresses subtitle translation from the perspective of multimodality. It breaks the linguistic confinement of subtitling research by emphasizing the role of semiotic interplay. In the field of multimodality, this study bridges subtitling and multimodality through investigating the interweaving relationships among different semiotic modes and the corresponding impacts on subtitle translation.