Audiovisual Translation as multimodal mediation
Jan-Louis Kruger (Macquarie University, Australia) gave this keynote talk at the LEAD ME Summer Training School Warsaw 2021 online.
The world of audiovisual media has changed on a scale last seen with the shift away from print to digital photography. VOD has moved from an expensive concept limited by technology and bandwidth, to the norm in most of not only the developed world, but also as an accelerated equaliser in developing countries. This has increased the reach and potential of audiovisual translation to facilitate intercultural communication, although automated means of creating subtitles poses some threats.
While the skills required to create AVT have come within reach of a large groups of practitioners due to advances in editing software and technology, with many processes from transcription to cuing being automated, research on the reception and processing of multimodal texts has also developed rapidly. This has given us new insights into the way viewers, for example, process the text of subtitles while also attending to auditory input as well as the rich visual code of film. This multimodality of film, although being acknowledged as one of the unique qualities of translation in this context, is also often overlooked in technological advances. When the emphasis is on the cheapest and simplest way of transferring spoken dialogue to written text, or visual scenes to auditory descriptions, the complex interplay between language and other signs is often overlooked.
Eye tracking provides a powerful tool for investigating the cognitive processing of viewers when watching subtitled film with research in this area drawing on cognitive science, psycholinguistics and psychology. I will present a brief description of eye tracking in AVT as well as the findings of some recent studies on subtitle reading at different subtitle presentation rates as well as in the presence of secondary visual tasks.
Website: https://lead-me-cost.eu/
Twitter: @COST_CA19142
Linkedin: / leadme-ca19142
This publication is based upon the work from COST Action LEAD ME CA19142, supported by COST (European Cooperation in Science and Technology) and is funded by the Horizon 2020 Framework Programme of the European Union.