1

Theoretical Background

Today, language learning has turned out to be more available by implementing

multimedia with spoken information and full visual context, such as subtitles. For

instance, subtitled videos representing words and pictures in oral and visual form are

more probable to activate both coding systems in the processing than words or pictures

alone. Vocabulary learning has commonly been one the most neglected areas of

language learning, since many educators prefer to dedicate more time and effort to

other more complicated‟ issues, leaving vocabulary learning for the student to cope

with by himself/herself. Luckily, nowadays, vocabulary learning is claimed to be more

complex and less straight forward than what it was often considered to be. Likewise,

vocabulary learning through video reproduction has traditionally been scarce, since

teachers have typically regarded audio-visual materials as resources used to improve

“more intricate” learning skills

DUAL CODING THEORY

The dual-coding theory proposed by Paivio (1971) suggests that when pictures are

added to the meaning, the number of signals connected with the message increases.

Viewers then will be more probable to keep the message in mind. Therefore, the results

of the past research appear to sustain the aspect that the use of subtitles causes multi-

sensory processing, interacting with audio, video and print mechanisms. These

information input foundations make the process of language learning enhanced,

improve the comprehension of the content, and increase vocabulary by looking at the

subtitled words in meaningful and stimulating circumstances.

In 1999, Kosslstra and Beentjes conducted a study on the effectiveness of the use of

L1 caption L2 spoken videos on the fourth and sixth graders' vocabulary acquisition and

word recognition in Dutch. There were 126 students from the fourth grade and 120

students from the sixth grade, however assigned into three groups. The results showed

that students in the experimental group viewing L1 caption L2 spoken video

outperformed on the vocabulary learning.

A study conducted by Neuman (1990), observing 129 seventh and eighth graders in

bilingual programs, examined the effectiveness of captioning in foreign language

instruction by designing four different modes. Nine segments of an educational science

series lasting 5-8 minutes were subtitled in the subject's target language. The results of

the experiment showed that participants who watched subtitled programs learned more

new words from the second language than those in any of the three other conditions.

Bean and Wilson (1989) reported that their adult non-native-speaking students

showed extremely positive attitudes toward captioning and increased in vocabulary.

Students who viewed L2 subtitled materials showed significant improvement in reading

comprehension, listening comprehension, vocabulary acquisition, and word recognition.

One of the most robust experiments along this line was conducted by Bird and Williams

(2002), who investigated the claim that captioning is a beneficial language learning tool

by looking at how a bimodal presentation (aural and visual) of novel words would affect

the learning of the words. In their study, vocabulary was presented to advanced
 3

learners of English fewer than three conditions: (a) text with sound, (b) text without

sound, and (c) sound without text. Bird and Williams considered the effects of these

conditions on word learning, as measured by both improvements in spoken word

recognition efficiency (how long it took the learners to remember the words) and

recognition memory. Results showed that vocabulary presented with text and sound

resulted in better recognition memory for spoken words when compared to the other

two presentation modalities.

Markham (1993) found that captions were more helpful to advanced learners when

the video materials were more abstract or complex. He concluded that for intermediate

to advanced learners, captioning should be used only when the video material is difficult

for the learners. Guillory (1998) found that captions are beneficial for beginning-level

learners. However, she found that beginning-level students benefit more when only key

words are presented as captions, rather than having entire sentences presented on

screen as captions.

Markham (2001) investigated whether the presence of captions affects learning. He

also explored whether familiarity with the content of the video differentially affects the

usefulness of captions. He presented Muslim, Buddhist, and non-Muslim/Buddhist ESL

students with videos in English about their respective religions. Within each of the three

groups, half of the students were shown the videos with captions, and half without.

Results revealed that both background knowledge and captions contributed

substantially to the learners’ comprehension of the videos.

• https://plato.stanford.edu/entries/mental-imagery/theories-memory.html
 4

• https://en.wikipedia.org/wiki/Dual-coding_theory

• https://www.sciencedirect.com/topics/neuroscience/dual-coding-theory

• https://www.instructionaldesign.org/theories/dual-coding/

COGNITIVE LOAD THEORY

Based on the cognitive theory of multimedia learning, the initial learning process

Involves five steps: (a) selection of relevant words to process into working memory; (b)

selection of relevant images to process into working memory; (c) organization of

selected words; (d) organization of selected images; and (e) integration of visual and

auditory information with prior knowledge (Horz & Schnotz, 2010; Mayer, 2009).

Learners may activate the knowledge to be retrieved in long-term memory and then

process the information in their working memory. This process establishes a link

between the incoming message and prior. If the message is further explained with

illustrations, comprehension and retention of the message may be enhanced.

Now, let’s look at how authentic video accompanied by subtitles can help vocabulary

learning in the foreign language class. A subtitled video clip provides a triple connection

between image, sound and text, sound and text being linked by translation (Danan,

1992) when standard subtitles are used. This fact is relevant enough when we think

about the importance of visual associations in memory for vocabulary learning and how
 5

this type of connection generally encourages greater retention in lexical terms. This

mnemonic power of images is here enhanced by the presence of sound and text

together; when this text appear in the form of translation, the associations in memory

are enhanced even further. Hence, the potential usefulness of authentic subtitled video

clips in vocabulary learning should not be negated. Canning-Wilson (2000) reaffirms this

statement when she suggests that „images contextualized in video or on its own can

help to reinforce language learning, provided the learner can see immediate meaning in

terms of vocabulary recognition‟.

• https://www.mindtools.com/pages/article/cognitive-load-theory.htm

• https://www.instructionaldesign.org/theories/cognitive-load/

• https://www.psychologistworld.com/memory/cognitive-load-theory

• https://en.wikipedia.org/wiki/Cognitive_load

Schema Theory

A schema (plural schemata) is termed as a ‘mental structure (which) derives from all

the particular experiences we have had’ (Nuttall 1982:7). Schema theory was first

proposed by the gestalt psychologist Bartlett (1932), who observed that when people

were asked to repeat a story from memory, they often filled in details which were not

included in the original but related to what they already knew based on their cultural

background. Schema then is culturally bound, as shall be shown in this essay. The

concept of schema can be found from anthropology to philosophy and has been

considered for both instruction and assessment in language learning, examples of which
 6

will be referred to and considered later. Kant (1781) claimed that new information,

concepts and ideas can only have meaning when they can be related to something

already known by the individual. This exemplifies how the idea of background

knowledge in language-related performance has been around for centuries, if not

actually coined as a term at the time. McNamara, Miller and Bransford (1991) in

Clapham (1996), who referred to the theory as ‘mental models‘ which consist of ‘mental

tokens’, agree that schemata are used in comprehension but that it is not clear what we

do with them. There are two main types of schema that have since been termed by

theorists (Cook 1997). The first is formal schema and is based on the background

knowledge of the structure of any given text. The second is content schema and relates

to the background knowledge of the content area. We can see then that all schemata

relates to background knowledge of some kind. Carrell (1998) states that these

processes are not understood well.

Within the context of a multi-skills approach, it more often than not leads to

processing via receptive skills (reading and listening) and then progresses to productive

skills work (speaking and writing). Research on schema theory has had a great impact

on understanding both reading and listening skills, with the amount of research done

seemingly focussing on the former. Most methodologies investigating the role of

schemata or background/prior knowledge are variations on Carrell’s (1987).

Traditionally in the study of second language comprehension, the emphasis has been

focussed almost exclusively on the language itself rather than the individual attempting

to interpret it (Cook 1997). It seems that more recently the language learner

themselves, with the knowledge and ideas that they bring from the outside world, is
 7

perhaps as important a factor as the language itself. Carrell and Eisterhold (1983:80)

point out that one of the most obvious reasons why a specific content schema may fail

to exist for a reader is that the schema is culturally specific and is not part of a reader’s

cultural background. Aebersold and Field rather philosophically describe it when they

say, ‘if the topic…is outside of their experience or base of knowledge, they are adrift on

an unknown sea’ (1997:41). It is then surely the teacher’s responsibility to help learners

and make sure that they are guided in the right direction by providing that which may

be missing, or at the very least, helping to stimulate what is already there. In an ideal

scenario, it would include a combination of both factors.

Focusing as we are mainly on reading skills, a useful distinction can be made here

between top-down and bottom-up processing. The former relates to making predictions

based on background knowledge (knowledge-based) and the latter to ‘building textual

meaning from the individual linguistic units’ (text-based) (Carrell 1982:101). This has

also been referred to by Anderson & Lynch (1988) as schematic knowledge, which is

background knowledge that includes factual/sociological factors and procedural

knowledge or how the language is used in discourse; versus systemic knowledge, which

is the knowledge of the language system and includes semantic, syntactic and

phonological features. All of which, when related by context, combine to provide

comprehension. So then, in relation to schema theory, it is with the former of both the

above that we are most concerned. However, evidence points to reading skills involving

both equally, working hand-in-hand in order to achieve the best results possible. This

view is supported by Spiro (in Carrell 1998), who believes that skilled readers constantly

adapt their mode of processing, changing to meet the demands of a particular

text/reading situation; less skilled readers tending to rely overly on processes in one

direction and thus experiencing negative effects on overall comprehension. Overreliance

on top-down processing has been referred to as ‘schema interference’, or a lack of

understanding (Carrell 1998). When faced with unfamiliar topics, some students may

‘overcompensate for absent schemata by reading in a slow, text-bound manner; other

students may overcompensate by wild guessing’ (Carrell 1988:101). Swaffar, Arens and

Byrnes (1991) in Clapham (1996) argue that most reading comprehension items only

test bottom-up skills and fail to actually measure the way a reader understands,

advertising the use of ‘recall protocols’ in tests of reading comprehension.

Captioning and subtitling: Undervalued language learning strategies

Martine Danan

Meta: Journal des traducteurs/Meta: Translators' Journal 49 (1), 67-77, 2004

Audiovisual material enhanced with captions or interlingual subtitles is a particularly

powerful pedagogical tool which can help improve the listening comprehension skills of

second-language learners. Captioning facilitates language learning by helping students

visualize what they hear, especially if the input is not too far beyond their linguistic

ability. Subtitling can also increase language comprehension and leads to additional

cognitive benefits, such as greater depth of processing. However, learners often need to

be trained to develop active viewing strategies for an efficient use of captioned and

subtitled material. Multimedia can offer an even wider range of strategies to learners,

who can control access to either captions or subtitles.

Noa Talaván

New insights into audiovisual translation and media accessibility, 285-299, 2010

the role of both reading and producing subtitles as functional activities and didactic

tools in foreign language education.

the didactic application of an activity based on the use of subtitles as learning support

and of subtitling as the active production of subtitles by students in front of the

computer, entails a series of benefits that are worth noting: it assists students in the

development of oral comprehension skills, provides them with different types of support

(visual, textual, and technological) for language development, encourages learners to

face authentic input, and produces tangible output (the subtitles produced by students)

that can be shared with their peers (or even on the web).

 http://web.mit.edu/pankin/www/Schema_Theory_and_Concept_Formation

 https://en.wikipedia.org/wiki/Schema

 https://education.stateuniversity.com/pages/2175/Learning-Theory-SCHEMA-

THEORY.html

 https://elearningindustry.com/schema-theory
10