This article was downloaded by: University of Pennsylvania

On: 30 Aug 2018

Access details: subscription number 10622
Publisher:Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place, London SW1P 1WG, UK

International Handbook of the Learning Sciences

Frank Fischer, Cindy E. Hmelo-Silver, Susan R. Goldman, Peter Reimann

Multiple Representations and Multimedia Learning

Publication details
https://www.routledgehandbooks.com/doi/10.4324/9781315617572-10
Shaaron Ainsworth
Published online on: 20 Apr 2018

How to cite :- Shaaron Ainsworth. 20 Apr 2018 ,Multiple Representations and Multimedia Learning
from: International Handbook of the Learning Sciences Routledge.
Accessed on: 30 Aug 2018
https://www.routledgehandbooks.com/doi/10.4324/9781315617572-10

PLEASE SCROLL DOWN FOR DOCUMENT

Full terms and conditions of use: https://www.routledgehandbooks.com/legal-notices/terms.

This Document PDF may be used for research, teaching and private study purposes. Any substantial or systematic reproductions,
re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will be complete or
accurate or up to date. The publisher shall not be liable for an loss, actions, claims, proceedings, demand or costs or damages
whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10

10
Multiple Representations and
Multimedia Learning
Shaaron Ainsworth

Consider three diverse learning situations: in the first, a high school student is asked to assess the evidence
that the universe is expanding; in the second, a training dentist learns to clean and fill a root canal; and
in the third, a family visit to a natural history museum prompts conversation about whether the diversity
of animal life is related to how continents are formed. It is easy to see how these situations differ: they
involve individual and social learning, occur in formal schooling, professional education and informal
contexts, with learners of any age and whose duration ranges from minutes to months. But there is
something they all have in common, and that is learning will be mediated by external representations
such as pictures, animations, graphs, augmented reality, haptics, as well as text and speech.
Human learning is increasingly (multi-)representational, as we constantly invent new forms of
representations whose appearance and interactive possibilities are partly due to technological devel-
opment. Accordingly, the purpose of this chapter is to review research from diverse branches of
learning sciences to trace some of the history of the field, before summarizing what we currently
know about the opportunities offered by multi-representational learning, as well as addressing the
challenges that it brings. Finally, it will end by predicting future trends and suggesting a focus for new
research on multi-representational learning.

Background
The learning sciences approach to multi-representational learning draws together three main themes,
each with a distinguishing preoccupation and specific methodological approach. It is the combina-
tion of these three that give the learning sciences approach its distinctive flavor.
The first approach is based upon cognitive accounts of instructional psychology and is particularly
associated with Multimedia Learning Theory (Mayer, 2014), Cognitive Load Theory (Sweller, Van
Merrienboer, & Paas, 1998), and the Integrated Theory of Text and Picture Learning (Schnotz,
2005). These theories share some assumptions. They argue that understanding is enhanced when
learners’ working memories are not overloaded, and therefore environments should be designed
to use representations in ways that minimize their impact on working memory. They assume that
there are limited-capacity modality specific processing systems: one focused on verbal, auditory, or
descriptive representations (depending upon the theory), and one for visual, pictorial or depictive
representations. Thus, learning is more effective when learners actively process representations, by
selecting and organizing relevant information from material and integrating it into coherent long-
term memory structure(s).

96
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Multiple Representations

Arguably, these theories provided the main impetus behind the argument that multiple representa-
tions can be advantageous for learning: for example, Mayer’s multimedia learning principle famously
concludes that people learn better from pictures and text than text alone (e.g. Mayer, 2014). A large
number of researchers have adopted the approach and together have produced a body of research that
can be distilled into guidelines for the design of learning material. Examples include avoid a split atten-
tion effect (Ayres & Sweller, 2014), so that materials a student needs to mentally integrate are collocated
spatially and temporally, or use spoken not written text with an animation, so that a learner can direct their
visual attention to the pictorial elements (Sweller, 2005). Furthermore, as the theories have developed
these claims have become more nuanced, with research focused on identifying the boundary conditions
for the principles—the most common condition being that materials designed for learners low in prior
knowledge may be less suitable for those with high prior knowledge (Kalyuga, 2007), and vice versa.
Methodologically, the vast majority of this research is experimental. The canonical study recruits
from a broad university population to achieve large numbers of participants who are assigned to
study variants of material that differ only in representation; for example, an animation which explains
how lightning is formed accompanied by either written, spoken, or both forms of text. Assessments
are given immediately after a short period of study and test the students’ retention of the mate-
rial and whether they can transfer their understanding to new issues. Given a common theoretical
framework, as well as sharing materials and tests, it has proven possible to produce meta-analyses of
such studies allowing estimations of the effects of using spoken text with pictures (e.g., Ginns, 2005).
However, there are still relatively few studies that test these principles in more realistic learning
situations, with people studying authentic materials for longer periods of time where understand-
ing is tested after some delay; see Eilam and Poyas (2008) for such an example. Consequently, it
is not clear the extent to which the guidelines formulated and tested under lab conditions apply in
classrooms, professional training, or museums. Nor is it certain that the underlying theoretical expla-
nation, based upon working memory, is sufficient (e.g., Rummer, Schweppe, Furstenberg, Scheiter,
& Zindler, 2011). Finally, these cognitive studies typically rest on a taxonomic approach to the rep-
resentation’s form (e.g., picture, text), whereas frameworks such as DeFT (Ainsworth, 2006) argue
that a functional analysis that first considers the specific educational purpose is vital.
The second area of research that informs the learning sciences approach to multiple representa-
tions develops from a very different perspective, as it starts from understanding expert performance
in a domain (see Reimann & Markauskaite, this volume). From radiologists detecting tumors on
X-rays (Lesgold et al., 1988) to archaeologists studying soil using a Munsell colour chart (Goodwin,
1994), expertise is seen as fundamentally representational. Historians of science (Gooding, 2004)
describe phases of representational construction and invention as scientists make new discoveries.
Moreover, working with representations is not the individualized practice common to instructional
psychology; instead, representations are at the center of a community of practice where they mediate
communication between members, drive explanation, or are martialled in argument (Kozma, Chin,
Russell, & Marx, 2000; Latour, 1999). Thus, developing expertise in a domain is judged by increas-
ing proficiency in using representational tools at the heart of cultural practices.
The theoretical frameworks that underpin this approach to multiple representations are more
diverse. Clearly, grounded in a sociocultural approach emphasizing how communities develop rep-
resentations and how membership involves acquiring cultural tools and practices (Säljö, 1999). This
approach draws more explicitly on semiotic theorists (e.g., Roth & Bowen, 2001), particularly social
semiotics (Kress, 2009). Cognitive theories are also important, although, rather than the information
processing models above, representational learning is understood in relation to situated (Tversky,
2005) or distributed (Zhang, 1997) approaches to cognition. Additionally, there are increasingly
welcome attempts to combine these sociocultural and cognitive approaches in integrative perspec-
tives (Airey & Linder, 2009; Prain & Tytler, 2012).
Typical studies explore how representations are used in practice. Kozma et al. (2000) spent 64
hours studying chemists in laboratories and found structural diagrams and equations drawn onto

97
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Shaaron Ainsworth

flasks, glass hoods, and white boards; in reference books and articles; and numeric and graphic output
from instruments. They found that chemists’ understanding was inherently multi-representational, as
they selected specific representations for particular functions. They recount how scientists coordinate
these different representations to justify a particular interpretation of a chemical reaction, where ini-
tial disagreement becomes joint understanding as interlocutors draw diagrams, consult instruments,
and look up findings. Similarly, Hutchins (1995) describes how practices, such as navigating ships
or landing planes, involved coordinating a distinct set of representations. These could include con-
stantly changing visual displays of speed or weight with static representations such as numbers looked
up on paper records. Individuals in the team take responsibility for coordinating different actions
and must remember different parts of the system. Together, the team uses a combination of external
representations, spoken talk, and their memories to safely land airplanes.
These accounts offer a rich picture of how constructing, selecting, and coordinating different
representations is fundamental to professional practice. As such they are effective at describing the
results of long-term representational learning. However, translating this knowledge into classroom
practices is not easy. Whilst we might argue that representational practices are necessary knowledge,
students themselves find such learning highly complicated. Many studies have shown that when
learning representational systems, students fail to integrate representations (see Ainsworth, 2006 for
examples) and that attempts to teach them are often surprisingly unsuccessful.
The final theme in this account is one that is often implicit: the role of technological development
in shaping our approach to multi-representational learning. As digital technologies have become
commonplace in classrooms, museums, workplaces, and homes, so the representations that support
teaching and learning have rapidly changed, although not automatically for the better. This is seen
in learning material as familiar as the textbook, which has seen a growth in the variety and number
of representations on a typical page, and an increased use of high-fidelity representations such as
photographs (Lee, 2010), as well as infographics (Polman & Gebre, 2015). And, of course, “text-
books” are increasingly digital and so now routinely include sound, video, and animation.
In addition, representational technologies that used to be only found in the hands of professionals are
increasingly present in classrooms offering opportunities to engage in authentic disciplinary practices.
Scientific visualization allows school students to experience the scale of physical phenomena—from
atomic interactions to planetary imaging (Gordon & Pea, 1995). Geographic information systems
combine satellite images, maps, field data, and aerial photographs in ways that can help students under-
stand the complexity of their local communities (Kerski, 2003).
Finally, some representational technologies offer students participation in digitally simulated
experiences that are inaccessible within the physical world. For example, simulations are now
commonplace in science and engineering classrooms, as teachers hope to save time, offer safe
spaces to acquire skills, allow manipulation of variables that would otherwise be unmanipulable
(e.g., alter gravity), and offer students opportunities for increased control of scientific inquiry
(Rutten, van Joolingen, & van der Veen, 2012). They also easily permit multiple representations
of phenomena (van der Meij & de Jong, 2006). Whilst there is some debate as to whether simu-
lations are “better” or “worse” than physical laboratories, most researchers accept that the two
situations have distinct affordances and learning is maximized by sensible combinations of both (de
Jong, Linn, & Zacharia, 2013).
Methodologically, much research in this arena consisted of developing systems and pronouncing
their success (Dillenbourg, 2008), but learning sciences approaches tend to be more nuanced. They
typically involve design-based research studies, iterating through cycles of development and use, to
refine the systems themselves or the way they are used in practice (e.g., Barab, Thomas, Dodge,
Carteaux, & Tuzun, 2005; Puntambekar, this volume).
Looking backwards, we can see that learning sciences approaches to multi-representational learn-
ing have been theoretically and methodologically diverse. I will argue in the next section that this has
resulted in a mature field of inquiry with useful insight, as well as more open questions.

98
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Multiple Representations

Where Is Research on Multiple Representations Now?

The first and most important lesson we have learned about multiple representations is to be skeptical
about implicit or explicit claims that more is always better: for example, two representations are not
always better than one (Kalyuga & Sweller, 2014), three dimensions not better than two (Keller,
Gerjets, Scheiter, & Garsoffky, 2006), interactive and dynamic representations are not always better
than static (Bétrancourt, 2005). Consequently, one important message is to embrace the affordances
that new technologies bring to representational learning, without assuming that this will magically
resolve learners’ difficulties.
So what are effective multi-representational systems? I want to argue against general principles
and suggest the following formulism: well-designed combinations of representations manipulate
information to make their key (task-relevant) aspects more accessible to learners for beneficial cogni-
tive, social, and affective processes. This claim makes salient aspects of representational learning that
I consider to be the most important. First, it identifies the importance of task analysis—representations
are not generally good or bad; they are more or less suitable for a particular task for a specific learner.
So, when looking at a multi-representational simulation of population density (see Figure 10.1),
a graph or (even better) a phase-plot allows for perceptual inferences (is this ecosystem oscillating
or moving towards stable equilibrium?), a table supports precise read-off (the number of prey and
predators at this point in time), whereas an equation allows precise calculation of expected future
states (Larkin & Simon, 1987). Second, this definition also draws attention to the way that this
analysis needs to embrace the representational system as a combination and not each representation in
isolation. Consequently, by combining the table, graph, and equation in a system, these representa-
tions complement one another and allow an experienced student to select the representations most
appropriate for their specific needs at that time. Alternatively, someone learning to read phase-plots
may benefit from the support of the more familiar table or time-series graph to help them interpret

Figure 10.1 An Example of Many Representations From a Predator–Prey Simulation Including

Table, Equation, Time-Series Graph, and Phase-Plot

99
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Shaaron Ainsworth

this less familiar form (see Ainsworth, 2006). Hopefully, this example also makes clear that repre-
sentations should be seen as something that stands to somebody for something in some respect or
capacity (Peirce, 1906) and so, when considering their suitability, it is likely that both learner and
task factors will be important (Acevedo Nistal, Van Dooren, & Verschaffel, 2013; Kalyuga, 2007).
The third main point of this definition is the equal attention given to the triumvirate of cognitive,
social, and affective processes. Classical research on multi-representational and multimedia learning
focused on individualized cognitive accounts of learning. This is clearly an important consideration
for multiple representations. However, the learning sciences traditions of sociocultural and situated
accounts of learning and professional practice make clear that representations are developed and used
by communities of practice to mediate communication between members. Representations serve
multiple social functions as students learn: facilitating communication between peers, and becom-
ing joint resources for subsequent problem solving (White & Pea, 2011); supporting teachers and
students to work together in the classroom as well as becoming the focus of what is to be learned
(e.g. Prain & Tytler, 2012), and, of course, are vivid in the account of workplace learning and action
(Kozma et al., 2000; Latour, 1999). Finally, this draws attention to the affective aspects of learn-
ing. Unfortunately, wider claims about affect or motivation have too often just been simplistic and
overgeneralized: “video, multimedia, augmented reality, etc. helps children learn as they enjoy it
so much.” Happily, learning sciences approaches can draw on design studies and conceptual frame-
works from game-based learning, which have provided better theorized impetus to study and design
for affect and motivation in representational learning (Habgood & Ainsworth, 2011; Virk, Clark, &
Sengupta, 2015). Traditional cognitive approaches have also been broadened to focus on motiva-
tional components (Moreno & Mayer, 2007).
The final point that I wish to draw attention to is the importance of processes. Learning with mul-
tiple representations is not a magical activity whereby simply presenting an animation with a picture
results in new understanding. There is abundant evidence that learners need to master many com-
plicated aspects of these representations in isolation and in combination for multi-representational
learning to be successful. Learners need to understand how representations encode and present infor-
mation, they need to know how to select or construct representations and, particularly for multiple
representations, how they relate to one another (e.g., Ainsworth, 2006). This understanding can take
a long time to develop as learners become more familiar with the representations and the roles they
play in their communities (Kozma & Russell, 2005). Novices may be misled by features of represen-
tational systems that are vivid but not important (Lowe, 2004), or lack metacognitive insight into the
need to process representations actively (Salomon, 1984). However, it is not all doom and gloom!
There is evidence that that visual representations can encourage the use of effective metacognitive
strategies (Ainsworth & Loizou, 2003) and that even younger students have insight into how to
design considered representations (diSessa, 2004).

The Future
Certain things are clearly predictable, at least in general, about the future of multi-representational
learning. We will continue to learn surrounded by representations as we read textbooks, run simula-
tions, visit museums and play games. It is also highly likely these representations will take forms that
we cannot currently imagine. Furthermore, some scenarios seem likely given the current direction
of travel. First, we may expect increasing attention to representations that more actively involve the
body. These draw upon the development of natural user interfaces (NUIs) whereby the interaction is
felt to be “natural” and draws upon increasingly invisible forms of control such as body movements,
gesture, or speech; for example, when children visiting an art gallery can interact and animate paint-
ings by moving on the floor (Price, Sakr, & Jewitt, 2016). This is closely related to approaches such as
participatory simulations, where students become viruses in an ecosystem by wearing programmable
tags (Colella, 2000) or experience and investigate earthquakes in their classrooms (Moher, 2008).

100
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Multiple Representations

Another approach is to provide representations of somatosensory (touch) information through hap-

tics allowing children to feel viruses (Minogue & Jones, 2006) or training dentists to prepare a
cavity (Suebnukarn, Haddawy, Rhienmora, & Gajananan, 2010). Opportunely, at the same time
that technology is permitting more body-based representation and interaction, developments in our
understanding of the importance of these body-based representations is increasing. This resonates
with those who argue for a more embodied approach to cognition and where the importance of
gesture for supporting learning is increasingly demonstrated (Kontra, Goldin-Meadow, & Beilock,
2012; Alibali & Nathan, this volume).
A second emerging theme is the importance of learners constructing and even inventing their
own representations for learning. This is seen in those who argue for this as a fundamental aspect of
representational (Kozma & Russell, 2005) or meta-representational competence (Disessa, 2004). It is
also resonant with the resurgence of interest in construction and making (e.g., Halverson & Peppler,
this volume). Again, technology progress is now bringing these practices more easily to university
classrooms, where tools such as CogSketch (Forbus, Usher, Lovett, Lockwood, & Wetzel, 2011) or
beSocratic (Bryfczynski et al., 2015) support learning and assessment in subjects like geology and chem-
istry. Simpler interfaces can even allow children to engage in model based reasoning through drawing
(van Joolingen, Aukes, Gijlers, & Bollen, 2015). However, like others, I would not want to forget the
importance of good old-fashioned pen and paper drawing (Ainsworth, Prain, & Tytler, 2011).
Another important arena for research is how best we support learners to work with multiple
representations. Unfortunately, far too much research can still mistake the process of learning a
multi-representational system with learning with a multi-representational system. Learners need
more time to master the environment and probably more explicit teaching in representational prac-
tices if we are to describe successful multi-representational learning. In addition, we need to actively
research the best ways to support learning with multiple representations. This might include teach-
ing learners effective ways to engage with standalone learning environments, such as when Stalbovs,
Scheiter, and Gerjets (2015) teach learners If-Then plans to integrate text and pictures in multime-
dia learning or when students are taught specific animation understanding strategies (Kombartzky,
Ploetzner, Schlag, & Metz, 2010).
However, compared to the sizable amount of research on learning with representations, there
has been relatively little to say about how teachers teach with and about multiple representations
(a strong exception to that claim being the work of socio-semioticians, e.g., Kress et al., 2005).
Fortunately, this gap is increasingly being filled by researchers exploring what teachers understand
about representations (e.g., Eilam, Poyas, & Hashimshoni, 2014), how they can support their stu-
dents so that their classrooms become sites of representational activities (Prain & Tytler, 2012), and
how teachers provide instruction in representational conventions as part of learning (Cromley et al.,
2013). Nevertheless, there is still much to explore about the teachers’ roles in multi-representational
classrooms.
A final theme to explore is the importance of considering assessment. At present formal schooling
relies on written and mathematical forms (Yore & Hand, 2010). This is of concern when learning
is multi-representational, as it not only may be an unsuitable way to assess students’ understand-
ing, but also sends a worryingly message of what knowing in a professional domain actually means
(Lemke, 2004). In research, we do see improvement with researchers designing assessment aligned
to representational goals (Lowe, Schnotz, & Rasch, 2010) as well as more usage of online process-
ing measures such as eye movements (van Gog & Scheiter, 2010), data mining and verbal protocols
(Rau, Michaelis, & Fay, 2015) to relate processing of learning to outcomes. Moreover, representa-
tional technologies for assessment are being developed. For example, beSocratic or CogSketch assess
students’ understanding by asking them to construct visual and multiple forms of representations, and
are becoming sufficiently mature to move from proof of concepts studies to large-scale deployment.
Games and simulations are also increasing being used as innovative forms of assessment (Clarke-
Midura & Dede, 2010).

101
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Shaaron Ainsworth

Conclusion
The future is multi-representational! There seems little doubt that learning will be mediated by a
variety of representational forms whose interactive possibilities are increasingly diverse. Learning sci-
ences can contribute to this evolution by offering thoughtful insight into how representations can be
designed, taking into account insight into cognitive, affective and social processes of learning, discov-
ering ways that learners can be supported to use these representations effectively in ways appropriate
to the contexts and, where required, ensuing assessment is sensitive to the multi-representational
learning that has taken place.

Further Readings
Hegarty, M. (2011). The cognitive science of visual-spatial displays: Implications for design. Topics in Cognitive
Science, 3(3), 446–474. doi:10.1111/j.1756-8765.2011.01150
This paper synthesizes cognitive science approaches to design of visual-spatial displays and an overview of the
perceptual and cognitive processes involved in reading such displays
Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In
J. K. Gilbert (Ed.), Visualization in science and education (pp. 121–146). Dordrecht: Kluwer Academic
Publishers.
This chapter summarizes evidence to explore the types of representational practices that expert chemists are
proficient in and novices need to acquire. It illustrates how foundational theory and disciplinary knowledge
integrate in learning sciences approaches to multi- representational learning.
Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational
Psychologist, 38(1), 43–52. doi:10.1207/S15326985EP3801_6
This paper is an excellent example of a paper that illustrates the classic cognitive approach to multimedia
learning.
Rau, M. A., Michaelis, J. E., & Fay, N. (2015). Connection making between multiple graphical representations:
A multi-methods approach for domain-specific grounding of an intelligent tutoring system for chemistry.
Computers & Education, 82, 460–485. doi:10.1016/j.compedu.2014.12.009
This paper provides an account of how connection making between multiple representations is important as
well as demonstrating how eye tracking and log data can be used to relate the processes of multi-representational
learning-to-learning outcomes.
Stieff, M. (2017). Drawing for promoting learning and engagement with dynamic visualizations. In R. Lowe &
R. Ploetzner (Eds.), Learning from dynamic visualization (pp. 333–356). Cham, Switzerland: Springer.
This reference illustrates a number of themes in this chapter. It is theoretically integrative, addresses the con-
struction of representations, and considers both cognitive and affective processes involved in drawing to support
learning from visualization.

NAPLeS Resources
Ainsworth, S., Multiple representations and multimedia learning [Video file]. Interview. In NAPLeS video series.
Retrieved October 19, 2017, from http://isls-naples.psy.lmu.de/video-resources/interviews-ls/ainsworth/
index.html

References
Acevedo Nistal, A., Van Dooren, W., & Verschaffel, L. (2013). Students’ reported justifications for their repre-
sentational choices in linear function problems: An interview study. Educational Studies, 39(1), 104–117. doi:
10.1080/03055698.2012.674636
Ainsworth, S. (2006). Deft: A conceptual framework for considering learning with multiple representations.
Learning and Instruction, 16(3), 183–198. doi:10.1016/j.learninstruc.2006.03.001
Ainsworth, S., & Loizou, A. T. (2003). The effects of self-explaining when learning with text or diagrams.
Cognitive science, 27(4), 669–681. doi:10.1016/s0364-0213(03)00033-8
Ainsworth, S., Prain, V., & Tytler, R. (2011). Drawing to learn in science. Science, 333(6046), 1096–1097.
doi:10.1126/science.1204153

102
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Multiple Representations

Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving
fluency in a critical constellation of modes. Journal of Research in Science Teaching, 46(1), 27–49.
Alibali, M. W., & Nathan, M. (2018). Embodied cognition in learning and teaching: Action, observation, and
imagination. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International handbook
of the learning sciences (pp. 75–85). New York: Routledge.
Ayres, P., & Sweller, J. (2014). The split-attention principle in multimedia learning. In R. E. Mayer (Ed.), The
Cambridge handbook of multimedia learning (Vol. 2, pp. 206–226). New York: Cambridge University Press.
Barab, S., Thomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making learning fun: Quest Atlantis, a
game without guns. Educational Technology Research and Development, 53(1), 86–107. doi:10.1007/bf02504859
Bétrancourt, M. (2005). The animation and interactivity principles. In R. E. Mayer (Ed.), The Cambridge hand-
book of multimedia learning (pp. 287–296). New York: Cambridge University Press.
Bryfczynski, S., Pargas, R. P., Cooper, M. M., Klymkowsky, M., Hester, J., & Grove, N. P. (2015). Classroom
uses for besocratic. In T. Hammond, S. Valentine, A. Adler, & M. Payton (Eds.), The impact of pen and touch
technology on education (pp. 127–136). Cham: Springer International Publishing.
Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in
Education, 42(3), 309–328. doi:10.1080/15391523.2010.10782553
Colella, V. (2000). Participatory simulations: Building collaborative understanding through immersive dynamic
modeling. Journal of the Learning Sciences, 9(4), 471–500. doi:10.1207/s15327809jls0904_4
Cromley, J. G., Perez, T. C., Fitzhugh, S. L., Newcombe, N. S., Wills, T. W., & Tanaka, J. C. (2013).
Improving students’ diagram comprehension with classroom instruction. Journal of Experimental Education,
81(4), 511–537. doi:10.1080/00220973.2012.745465
de Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and virtual laboratories in science and engineering
education. Science, 340(6130), 305–308. doi:10.1126/science.1230579
Dillenbourg, P. (2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140.
doi:10.1080/01587910802154939
diSessa, A. A. (2004). Metarepresentation: Native competence and targets for instruction. Cognition and
Instruction, 22(3), 293–331. doi:10.1207/s1532690xci2203_2
Eilam, B., & Poyas, Y. (2008). Learning with multiple representations: Extending multimedia learning beyond
the lab. Learning and Instruction, 18(4), 368–378. doi:10.1016/j.learninstruc.2007.07.003
Eilam, B., Poyas, Y., & Hashimshoni, R. (2014). Representing visually: What teachers know and what they
prefer. In B. Eilam & J. K. Gilbert (Eds.), Science teachers’ use of visual representations (pp. 53–83). Dordrecht:
Springer.
Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). Cogsketch: Sketch under-
standing for cognitive science research and for education. Topics in Cognitive Science, 3, 648–666.
doi:10.1111/j.1756-8765.2011.01149.x
Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15(4), 313–331. doi:10.1016/j.
learninstruc.2005.07.001
Gooding, D. C. (2004). Cognition, construction and culture: Visual theories in the sciences. Journal of Cognition
and Culture, 3(4), 551–593. doi:10.1163/1568537042484896
Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606–633. doi:10.1525/aa.1994.
96.3.02a00100
Gordon, D. N., & Pea, R. D. (1995). Prospects for scientific visualisation as an educational technology. Journal
of the learning sciences, 4(3), 249–279. doi:10.1207/s15327809jls0403_1
Habgood, M. P. J., & Ainsworth, S. E. (2011). Motivating children to learn effectively: Exploring the value of
intrinsic integration in educational games. Journal of the Learning Sciences, 20(2), 169–206. doi:10.1080/1050
8406.2010.508029
Halverson, E., & Peppler, K. (2018). The Maker Movement and learning. In F. Fischer, C. E. Hmelo-Silver,
S. R. Goldman, & P. Reimann (Eds.), International handbook of the learning sciences (pp. 285–294). New York:
Routledge.
Hutchins, E. (1995). How a cockpit remembers its speeds. Cognitive Science, 19(3), 265–288. doi:10.1207/
s15516709cog1903_1
Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational
Psychology Review, 19(4), 509–539. doi:10.1007/s10648-007-9054-3
Kalyuga, S., & Sweller, J. (2014). The redundancy principle in multimedia learning. In R. E. Mayer (Ed.), The
Cambridge handbook of multimedia learning (p. 247). New York: Cambridge University Press.
Keller, T., Gerjets, P., Scheiter, K., & Garsoffky, B. (2006). Information visualizations for knowledge acquisition:
The impact of dimensionality and color coding. Computers in Human Behavior, 22(1), 43–65. doi:10.1016/j.
chb.2005.01.006
Kerski, J. J. (2003). The implementation and effectiveness of geographic information systems technology and
methods in secondary education. Journal of Geography, 102(3), 128–137. doi:10.1080/00221340308978534

103
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Shaaron Ainsworth

Kombartzky, U., Ploetzner, R., Schlag, S., & Metz, B. (2010). Developing and evaluating a strategy for learning
from animations. Learning and Instruction, 20(5), 424–433. doi:10.1016/j.learninstruc.2009.05.002
Kontra, C., Goldin-Meadow, S., & Beilock, S. L. (2012). Embodied learning across the life span. Topics in
Cognitive Science, 4(4), 731–739. doi:10.1111/j.1756-8765.2012.01221.x
Kozma, R., Chin, E., Russell, J., & Marx, N. (2000). The roles of representations and tools in the chemis-
try laboratory and their implications for chemistry learning. Journal of the Learning Sciences, 9(2), 105–143.
doi:10.1207/s15327809jls0902_1
Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In
J. K. Gilbert (Ed.), Visualization in science and education (pp. 121–146). Dordrecht: Kluwer Academic
Publishers.
Kress, G. (2009). Multimodality: A social semiotic approach to contemporary communication. London: Routledge.
Kress, G., Jewitt, C., Bourne, J., Franks, A., Hardcastle, J., Jones, K., & Reid, E. (2005). English in urban class-
rooms: Multimodal perspectives on teaching and learning. New York: Routledge.
Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth 10000 words. Cognitive Science,
11(1), 65–99. doi:10.1016/S0364-0213(87)80026-5
Latour, B. (1999). Pandora’s hope: Essays on the reality of science studies. Cambridge, MA: Harvard University Press.
Lee, V. R. (2010). Adaptations and continuities in the use and design of visual representations in
U.S. middle school science textbooks. International Journal of Science Education, 32(8), 1099–1126.
doi:10.1080/09500690903253916
Lemke, J. L. (2004). The literacies of science. In E. W. Saul (Ed.), Crossing borders in literacy and science instruction:
Perspectives on theory and practice (pp. 33–47). Newark, DE: International Reading Association.
Lesgold, A., Rubinson, H., Feltovich, P., Glaser, R., Klopfer, D., & Wang, Y. (1988). Expertise in a com-
plex skill: Diagnosing x-ray pictures. In M. T. Chi, R. E. Glaser, & M. J. Farr (Eds.), The nature of expertise
(pp. 311–342). Hillsdale, NJ: Lawrence Erlbaum Associates.
Lowe, R. (2004). Interrogation of a dynamic visualization during learning. Learning and Instruction, 14(3),
257–274. doi:10.1016/J.Learninstruc.2004.06.003
Lowe, R., Schnotz, w., & Rasch, T. (2010). Aligning affordances of graphics with learning task requirements.
Applied Cognitive Psychology, 25(3), 452–459. doi:10.1002/acp.1712
Mayer, R. E. (2014). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook
of multimedia learning (pp. 43–71). New York: Cambridge University Press.
Minogue, J., & Jones, M. G. (2006). Haptics in education: Exploring an untapped sensory modality. Review of
Educational Research, 76(3), 317–348. doi:10.3102/00346543076003317
Moher, T. (2008). Learning and participation in a persistent whole-classroom seismology simulation. Proceedings
of the 8th International Conference for the Learning Science, Utrecht, Netherlands.
Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review,
19(3), 309–326. doi:10.1007/s10648-007-9047-2
Peirce, C. S. (1906). Prolegomena to an apology for pragmaticism. The Monist, 16, 492–546.
Polman, J. L., & Gebre, E. H. (2015). Towards critical appraisal of infographics as scientific inscriptions. Journal
of Research in Science Teaching, 52(6), 868–893. doi:10.1002/tea.21225
Prain, V., & Tytler, R. (2012). Learning through constructing representations in science: A framework of
representational construction affordances. International Journal of Science Education, 34(17), 2751–2773. doi:
10.1080/09500693.2011.626462
Price, S., Sakr, M., & Jewitt, C. (2016). Exploring whole-body interaction and design for museums. Interacting
with Computers, 28(5), 569–583. doi:10.1093/iwc/iwv032
Puntambekar, S. (2018). Design-based research (DBR). In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, &
P. Reimann (Eds.), International handbook of the learning sciences (pp. 383–392). New York: Routledge.
Rau, M. A., Michaelis, J. E., & Fay, N. (2015). Connection making between multiple graphical representations:
A multi-methods approach for domain-specific grounding of an intelligent tutoring system for chemistry.
Computers & Education, 82, 460–485. doi:10.1016/j.compedu.2014.12.009
Reimann, P., & Markauskaite, L. (2018). Expertise. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, &
P. Reimann (Eds.), International handbook of the learning sciences (pp. 54–63). New York: Routledge.
Roth, W. M., & Bowen, G. M. (2001). Professionals read graphs: A semiotic analysis. Journal for Research in
Mathematics Education, 32(2), 159–194. doi:10.2307/749672
Rummer, R., Schweppe, J., Furstenberg, A., Scheiter, K., & Zindler, A. (2011). The perceptual basis of
the modality effect in multimedia learning. Journal of Experimental Psychology—Applied, 17(2), 159–173.
doi:10.1037/a0023588
Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations
in science education. Computers & Education, 58(1), 136–153. doi:10.1016/j.compedu.2011.07.017

104
Downloaded By: University of Pennsylvania At: 20:23 30 Aug 2018; For: 9781315617572, chapter10, 10.4324/9781315617572-10
Multiple Representations

Säljö, R. (1999). Learning as the use of tools. In K. Littleton & P. Light (Eds.), Learning with computers: Analysing
productive interaction (pp. 144–161). London: Routledge.
Salomon, G. (1984). Television is easy and print is tough—The differential investment of mental effort in
learning as a function of perceptions and attributions. Journal of Educational Psychology, 76(4), 647–658.
doi:10.1037/0022-0663.76.4.647.
Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. Mayer (Ed.), The Cambridge
handbook of multimedia learning (pp. 49–69). Cambridge: Cambridge University Press.
Stalbovs, K., Scheiter, K., & Gerjets, P. (2015). Implementation intentions during multimedia learning:
Using if-then plans to facilitate cognitive processing. Learning and Instruction, 35, 1–15. doi:10.1016/j.
learninstruc.2014.09.002
Suebnukarn, S., Haddawy, P., Rhienmora, P., & Gajananan, K. (2010). Haptic virtual reality for skill acquisi-
tion in endodontics. Journal of Endodontics, 36(1), 53–55. doi:http://dx.doi.org/10.1016/j.joen.2009.09.020
Sweller, J. (2005). The redundancy principle in multimedia learning. In R. Mayer (Ed.), The Cambridge handbook
of multimedia learning: (pp. 159–168). Cambridge: Cambridge University Press.
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional
design. Educational Psychology Review, 10(3), 251–296. doi:10.1023/a:1022193728205
Tversky, B. (2005). Spatial cognition: Embodied and situated. In P. Robbins & M. Aydede (Eds.), The Cambridge
handbook of situated cognition (pp. 201–216). Cambridge: Cambridge University Press.
van der Meij, J., & de Jong, T. (2006). Supporting students’ learning with multiple representations in a
dynamic simulation-based learning environment. Learning and Instruction, 16(3), 199–212. doi:10.1016/j.
learninstruc.2006.03.007
van Gog, T., & Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning
and Instruction, 20(2), 95–99. doi:10.1016/j.learninstruc.2009.02.009
van Joolingen, W., Aukes, A. V., Gijlers, H., & Bollen, L. (2015). Understanding elementary astronomy by
making drawing-based models. Journal of Science Education and Technology, 24(2–3), 256–264. doi:10.1007/
s10956-014-9540-6
Virk, S., Clark, D., & Sengupta, P. (2015). Digital games as multirepresentational environments for science
learning: Implications for theory, research, and design. Educational Psychologist, 50(4), 284–312. doi:10.1080
/00461520.2015.1128331
White, T., & Pea, R. (2011). Distributed by design: On the promises and pitfalls of collaborative learning with
multiple representations. Journal of the Learning Sciences, 20(3), 489–547. doi:10.1080/10508406.2010.542700
Yore, L., & Hand, B. (2010). Epilogue: Plotting a research agenda for multiple representations, multiple modal-
ity, and multimodal representational competency. Research in Science Education, 40(1), 93–101. doi:10.1007/
s11165-009-9160-y
Zhang, J. J. (1997). The nature of external representations in problem solving. Cognitive Science, 21(2), 179-217.
doi: 10.1207/s15516709cog2102_3.

105