Abstract
Simultaneous Interpreting (SI) is a real-time oral translation of a speaker's oral discourse by an interpreter who usually works from a sound-proof cabin in a conference room. SI requires complex information processing, since the interpreter has to carry out several tasks in parallel (i.e., listening, comprehending and producing speech). Interpreters are often provided with a written version of the oral speech to be delivered by the speaker beforehand, an additional information source that can be used for task preparation, but also during the delivery of the oral speech. The latter practice is referred to as "SI with text" or "SIMTEXT". This involves multimodal cognitive processing since the interpreter now receives information through different channels, simultaneous auditory and visual. Although intuitively, the availability of the text appears to be useful support for achieving the interpreting task, previous experiments with SIMTEXT have indicated that multimodal input may in fact complicate cognitive processing and, as such, undermine the potential facilitating effect. However, the evidence supporting this hypothesis is inconclusive, since only few studies have addressed the matter, and their methodological design often lacks ecological validity due to artificial experimental conditions, the use of less accurate tools, subjective measurements or the exclusive focus on only one input variable.
We therefore propose to explore the effect of multimodal input on cognitive processing in SIMTEXT by designing experiments with professional simultaneous interpreters who will carry out one SI and one SIMTEXT task in near to real-life conditions, that is, in an interpreting booth, wearing eye tracking glasses. By manipulating a combination of input variables related to content, form and delivery of the oral speeches in both experimental conditions (SI and SIMTEXT), we will investigate the facilitating effect of text in SI. The study has a mixed-method design. One the one hand, quantitative data are collected by means of mobile eye tracking glasses to measure the effect of the use of text on cognitive processing and on the allocation of visual attention. On the other hand, qualitative data will be collected by means of retrospective online questionnaires aimed to elicit the participants' perceived cognitive load. Apart from studying the effect of the use of text on information processing, we will also look into its effect on the interpreting product in the form of recordings from the interpreting tasks, which will be submitted to a qualitative analysis.
By using a combination of input variables and complementary methods, we intend to investigate the cognitive processing of multimodal information sources in SI from a much broader perspective than has so far been the case in studies on this topic. As such, we intend to generate new insights into the study of multimodality and cognition, that will also be invaluable for SI practice and training.
Researcher(s)
Research team(s)
Project type(s)