342 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 2, NO.

4, OCTOBER-DECEMBER 2009

Remote Laboratories Extending Access

to Science and Engineering Curricular
Martyn Cooper and José M.M. Ferreira

Abstract—This paper draws on research, development, and deployment of remote laboratories undertaken by the authors since 2000.
They jointly worked on the PEARL project (http://iet.open.ac.uk/pearl/) from 2000 to 2003 and have worked on further projects within
their own institutions (the Open University, United Kingdom, and the University of Porto, Portugal, respectively) since then. The paper
begins with a statement of the rationale for remote experiments, then offers a review of past work of the authors and highlights the key
lessons for remote labs drawn from this. These lessons include 1) the importance of removing accessibility barriers, 2) the importance
of a pedagogic strategy, 3) evaluation of pedagogic effectiveness, 4) the ease of automation or remote control, and 5) learning
objectives and design decisions. The paper then discusses key topics including assessment issues, instructional design, pedagogical
strategies, relations to industry, and cost benefits. A conclusion summarizes key points from the paper within a review of the current
status of remote labs in education.

Index Terms—Information interfaces and representation (HCI), user-centered design, virtual labs.

1 INTRODUCTION

R EMOTE experiments for teaching and learning in the

science and engineering subject areas have been around
for over 20 years now. Their widespread penetration across
[3], [4]. It appears to offer a simple solution to problems of
distance, collaboration, expensive equipment, and limited
availability. In this section, we consider whether these
the curricular at higher education level, as their proponents provide sufficient reasons; for the control issues, the software
have predicted, has yet to be achieved. However, the and learning design challenges mean that providing remote
authors maintain there is still a significant potential here.
access is not a prospect to be entered into lightly; rather
This paper highlights key lessons learned from the authors’
analysis should identify where the benefits really can be.
work in this field over the last 10 years. Key to this is the
move of the focus from solving the technical issues of how Remote access is not enough; there should be better learning
to make remote control of teaching experiments possible to provided to otherwise disadvantaged learners [5], [6], [7].
addressing the pedagogical challenges of doing so in a way Practical work is universally recognized as being a key
that is effective in teaching and learning. part of science and engineering education. However, there
The paper restates the rationale for remote experiments. are challenges to making practical work available to
Fundamental to this rationale are issues of access. This students in today’s higher education environment. There
means access for all students to lab facilities in a way that is are diverse reasons for considering the provision of
effective in their learning, including the case of students practical work remotely in a given context but these all
with disabilities. The paper gives an overview of the revolve around issues of students’ access to the equipment
various projects that the authors have led and been and facilities they need to undertake teaching experiments.
involved in, the lessons and the issues that surfaced from
them. Promising research and development directions that 2.1 Access to the Laboratory Equipment
will shape the future generations of remote laboratories in There are three particular circumstances when the provision
the authors view are outlined. of experimental work remotely can enable experimental
work to be more readily offered to students [8], [9], [10]:
2 RATIONALE FOR REMOTE EXPERIMENTS 1. when the students are studying at a distance from
Providing remote access to practical experiments may seem the institution;
like a straightforward idea within distance education [1], [2], 2. when the equipment required for the desired experi-
mental work is considered prohibitively expensive;
3. when it is difficult to cope with large student
. M. Cooper is with the Accessible Educational Media (AEM) Group,
Institute of Educational Technology (IET), Open University (OU), Walton numbers given the lab space available.
Hall, Milton Keynes MK7 6AA, UK. E-mail: [email protected]. In the first case, distance learning, traditionally experi-
. J.M.M. Ferreira is with the Department of Electrical and Computer
mental work has been offered by simple “home experimen-
Engineering, Faculty of Engineering, University of Porto, 4200-465 Porto,
Portugal, and the Buskerud College of Engineering, 3603 Kongsberg, ter” kits and intensive residential schools as part of a
Norway. E-mail: [email protected]. distance learning course. The home experiment kits ob-
Manuscript received 31 Mar. 2009; revised 22 June 2009; accepted 6 Oct. viously have limitations on the range of experimental work
2009; published online 14 Oct. 2009. that can be undertaken. The residential schools can offer
For information on obtaining reprints of this article, please send e-mail to:
[email protected], and reference IEEECS Log Number TLTSI-2009-03-0067. access to high quality laboratory facilities but here most of
Digital Object Identifier no. 10.1109/TLT.2009.43. the practical work associated with a particular course has to
1939-1382/09/$25.00 ß 2009 IEEE Published by the IEEE CS & ES
COOPER AND FERREIRA: REMOTE LABORATORIES EXTENDING ACCESS TO SCIENCE AND ENGINEERING CURRICULAR 343

be grouped into an intensive week say. Remote experiments and pedagogical issues. The next sections summarize the
in a distance learning context offer the possibility of access to objectives and scope of these projects.
exciting experimental facilities and the undertaking of
particular experiments at the point in the course when they
3.1 PEARL
are most relevant to what the students are studying. The PEARL project ran from 2000 to 2003 and the authors
Increasingly there are demands on engineering and were partners in it. PEARL researched and developed a
science courses to give their students experience of state- system to enable students to conduct real-world experi-
of-the-art laboratory facilities. These demands may come ments as an extension of computer-based learning (CBL)
from accrediting bodies or key prospective employers of the and distance learning systems. The objectives were to give
students on the course. Now this can be prohibitively high quality learning experiences in science and engineer-
expensive especially in the cases where this equipment ing education by bringing the teaching lab to the students;
would only be used for a small part of the year. One offering flexibility in terms of time, location, and special
solution here offered by the remote experiment approach is needs. This rationale extended Internet course delivery to
the possibility of sharing expensive equipment between include enabling students to work collaboratively on
different institutions. practical elements of their courses that would be tradition-
The third case can be addressed by providing experi- ally lab based. The project developed a modular system for
ments on something like a 24/7 basis remotely over the flexibly creating diverse remotely controlled experiments,
Internet. In this way, the same physical space can cope with integrating this with a collaborative working environment
much larger student numbers and a greater flexibility can and accessible user interfaces. The project evaluated the
be offered to the students as to when they undertake their pedagogic impact of this approach, validating its develop-
practical work. ments in different educational contexts and subject areas.
These included foundation level physical sciences (as part
2.2 Extending Access to Students with Disabilities of an open and distance learning introductory course); cell
Universities may wish to consider the additional potential biology (as part of a final year undergraduate course);
benefit of remote experiments of providing access for manufacturing engineering (postgraduate training); and
students with disabilities who may not be able to access a digital electronics (as part of undergraduate courses in
laboratory, or who cannot operate laboratory equipment. design and testing). Further information about the pedago-
Universities who wish to develop remote experiments may gical strategies developed in PEARL and their outcomes
find this a useful tool in making science and engineering may be found in [11].
courses more accessible. This may be particularly relevant The overall budget for PEARL was US $2 million with an
in countries which have introduced legislation to reduce effort of 30 person-years. It was coordinated by the Open
discrimination against people with disabilities, in particular University (the first author being project director) with the
students with disabilities. US and United Kingdom (and other partners being: University of Dundee; Trinity College
many other countries1) have such legislation. Clearly, Dublin, Faculty of Engineering of the University of Porto;
whether it is the primary aim of a university to adopt the and Zenon S.A. of Athens.
practice of using remote experiments to make courses more
accessible for students with disabilities or not, it is 3.2 MARVEL
important to provide ready access and to assure that the MARVEL was an education and training project funded by
user interface is usable to students who need to interact the European Commission’s Leonardo da Vinci programme
with their computer environment in variety of ways. This [12]. MARVEL aimed to implement and evaluate learning
includes students with disabilities. environments for Mechatronics in Vocational Training,
allowing students online access to physical workshops
and laboratory facilities from remote places. The project
3 AN OVERVIEW OF PEARL AND ITS SUCCESSORS merged real and virtual, as well as local and remote worlds
I keep six honest serving-men in real time, and led to evaluated working examples of
(They taught me all I knew); remotely accessible practical environments, together with
Their names are What and Why and When supporting e-learning and student assessment material, in
And How and Where and Who.
robotics, modular production systems, and process control.
.....
Rudyard Kipling With a duration of 30 months (ended in April of 2005),
MARVEL brought together partners from Germany, Portu-
The authors’ experience in remote laboratories begun in the gal, Scotland, Greece, and Cyprus. Instead of focusing on
late 1990s and led to a first project entitled Practical technological aspects, MARVEL concentrated on the
Experimentation by Accessible Remote Learning (PEARL), development of learning content (remote experiments),
at a time when a very significant effort was still required to
which shared the following modules embedded into a
overcome the technical difficulties that hampered remote
Moodle2 e-learning platform:
access to laboratory workbenches. Additional projects
followed, as the research effort focused successively on . a Flash Communications server to support colla-
technical, educational (content development and delivery), borative learning via videoconferencing;

1. See a list at http://www.w3.org/WAI/Policy/. 2. See http://moodle.org/.

344 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 2, NO. 4, OCTOBER-DECEMBER 2009

a proprietary scheduling/booking application that

. Tasks 1 and 2 proceeded simultaneously, and so did the
enabled the students to reserve one-hour slots in the development of the pedagogic evaluation methods and
remote labs; instruments involved in task 3. Halfway through the
. an underlying e-learning package that integrated the project, field trials were initiated, and data started to be
modules referred above and all pedagogical contents gathered. Analysis and reporting closed the project, which
that were necessary to carry out the required remote lasted from November 2006 to June 2008. The project
experiments. consortium included the Faculty of Engineering and the
The typical MARVEL remote experimentation scenario Faculty of Psychology and Educational Sciences of the
might be summarized as follows: University of Porto (Portugal), and several other schools
from Coimbra and Lisbon that offered their students remote
.The teachers drafted a metascript description (used access to the experiments.
by the students to start their work) and built a The technical work package included in the project work
corresponding workshop activity within Moodle plan played a minor role, and was basically intended to
(including the definition of deadlines and grading provide maintenance capability for the remote work-
schemes). benches. The real aim of the project consisted of dealing
. The remote lab equipment was set up to support the with teacher training and with the evaluation of pedagogi-
practical tasks required from the students and the cal effectiveness. A training program comprising three
corresponding interface panels were brought to- 5  2h-session training actions was developed to address
gether (e.g., using a set of PXI modules and the Pedagogical principles, E-learning via Moodle, and Online labs.
corresponding LabView scripts). The attendants to each session were proposed homework
. The teacher presented to each group of students the tasks estimated to require approximately the same time as
work to be done and the milestones and expected the in-class presentations. The Pedagogical principles and the
deliverables. E-learning via Moodle sessions were interleaved, to enable
. The work of the students was initiated and the the “practical” application of the “theoretical” concepts
teacher supported and supervised its development, presented in the pedagogical presentations. Teacher train-
assessing the intermediate documents and deciding ing took place between February and June 2006, involving
when to move on to the next phase of the workshop. 103 teachers, who were allowed to build individual training
The two last steps could either take place face-to-face or plans, according to their knowledge profile and application
online, using the videoconferencing server. All the back- goals. Interviews with the trainers and questionnaires filled
ground theoretical contents were made available within the by the trainees (teachers) were used to evaluate these
same Moodle site in the form of other resources/activities, training actions.
such as lessons, quizzes, assignments, a forum, etc. The evaluation of pedagogical effectiveness led to very
The sequence described above was perceived by the interesting results, and supported the conclusion that
students as a learning activity with an embedded remote students recognize the pedagogic benefits of remote labs.
experiment. The learning goal was stated in the metascript We also concluded that comprehension of the learning
provided by the instructor and the social constructivist process is an important dimension in the use of remote
approach ensured that at the end the students provided experiments, which is in itself worthy of consideration as a
evidence of achieving the intended learning goal. The research direction. The Labs-on-the-web project has shown
prototypes developed within MARVEL are all presented that there is still room for improvement, concerning
development and usage of remote labs to support practical
online at http://www.marvel.uni-bremen.de.
assignments in engineering courses.
3.3 LABS-ON-THE-WEB
3.4 Lego Mindstorm-Based Remote Labs
The Labs-on-the-web project was prepared in response to a
One of the findings of the stakeholder research undertaken at
call for proposals aiming to improve pedagogical success in
the end of the PEARL project was that cost was a major factor
higher education degrees [13]. The project rationale was likely to influence the widespread adoption of remote labs in
that web access to lab workbenches would facilitate higher education. Many educators considered making
experiments and other practical assignments proposed to facilities such as the remote controlled optical spectrometer
engineering students, enabling them to better understand and electron microscope remotely available as appropriate
and consolidate the underlying theoretical knowledge. The for learning but prohibitively expensive. Because of this, the
project comprised three main areas: work since PEARL at the OU has concentrated on realizing
low cost remote experiment facilities. The approach adopted
1. the technical work needed to set up a range of
is to build diverse lab jigs implemented with Lego Mind-
remote labs in various engineering degrees;
storm and basic webcams. Mindstorm consists of a pro-
2. teacher training, to ensure appropriate perception
grammable microcontroller embedded within a Lego brick
and use of the technology;
that can be connected to a range of sensors and actuators.
3. pedagogical evaluation (knowledge and skills, learn-
Examples of jigs created include:
ing processes, peer cooperation, teacher interaction),
including the development of the methods and . Programmable Robot;
instruments to be used on field trials, data gathering, . Light bench (not really sufficient precision);
and analysis. . Principals of flight;
COOPER AND FERREIRA: REMOTE LABORATORIES EXTENDING ACCESS TO SCIENCE AND ENGINEERING CURRICULAR 345

TABLE 1
Priority 1 Errors and Warnings

Fig. 1. A remote electronics workbench interface panel.

. Newton’s laws of motion;

. Measurement of g by falling/pendulum;
. Angles of refraction/refractive index.
The Lego Mindstorm is connected to a PC that acts as a
web server exposing its functionality to the world.
Dedicated remote clients are implemented in JAVA.

4 KEY LESSONS
The example projects that were summarized in the previous
sections were a valuable source of experience with regard to
technical, functional, and pedagogical aspects. A summary
of the lessons learned will now be presented, focusing on
what the authors consider to be the most relevant
conclusions derived from those projects.

4.1 Lesson #1: On the Importance of Removing/ documentation to advise developers with this respect [17],
Minimizing Accessibility Barriers but many remote laboratory environments designed with
The need to account for accessibility requirements when LabView suffer from this problem. An accessibility test
developing remote laboratories is a key lesson that deserves done with the Watchfire WebXACT3 tool [18] signaled a
to be considered first and foremost. Real barriers in virtual variety of quality and accessibility issues that are listed in
worlds are unfortunately too common, and particularly so Tables 1, 2, and 3, and grouped according to the three levels
in this application domain [14], [15]. Instrument control of priority defined by the W3C Web Content Accessibility
panels are normally designed to replicate the buttons and Guidelines (WCAG). Note: these relate to WCAG v1.0 which
displays found in standard workbench equipment, meaning pertained at the time this work was undertaken. Since
that interaction with the users is largely based on visual December 2008, these have been superseded by WCAG 2.0
information and positional control. The example shown in (http://www.w3.org/TR/WCAG).
Notice that many of the errors and warnings indicated in
Fig. 1 represents the interface to a remote electronics
these tables are not a consequence of the graphical
workbench with live video streaming from the workbench
programming environment, and were instead due to bad
desktop and from the laboratory where it is located.
programming practices. Addressing such problems may be
Graphical programming languages such as LabView [16]
perceived to effect time-to-market and other cost perfor-
are not necessarily an obstacle to the inclusion of accessi-
mance factors. However, addressing them is also often a
bility features in the implementation of a remote lab, but
market acceptance issue as well as an aspiration and legal
their inclusion is largely left to the designer/programmer.
obligation for many educational institutions.
The end result is in most cases highly unfavorable to users
with visual impairments or other special needs. National 3. This tool was acquired by IBM in February 2008 and is no longer
Instruments is aware of this problem and provides available since that date.
346 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 2, NO. 4, OCTOBER-DECEMBER 2009

TABLE 2 TABLE 3
Priority 2 Errors and Warnings Priority 3 Errors and Warnings

wasted. Lab scripts may offer the students a much richer

learning experience, if their teachers are aware of the
plethora of pedagogical benefits offered by remote experi-
ments. Leading the students to realize that the lab session
can adapt to their pace, instead of forcing them to work on a
predefined schedule; that they can continue an experiment
from home, if it was not possible to complete it at the lab;
that they can rehearse the lab assignment before going to
the workbench; that they can repeat part or a whole
experiment to confirm doubtful data; and that they will not
necessarily miss the assignment if illness or other reasons
prevented them from going to the lab, will likely lead to
much better knowledge retention rates and more satisfac-
tory learning experiences.
Lab assignments are an important activity within any
social constructivist learning model, not only because they
represent an exploratory approach to knowledge acquisi-
One of the lessons indicated by the results shown in tion, but also due to the fact that in most cases they are
Tables 1, 2, and 3 is that the programmers frequently skip carried out by groups of students, where collaboration
the introduction of alternative text to images to shorten the supersedes individual work. The design of remote experi-
development time (the error indicated in the beginning of ments as embedded learning objects within an e-learning
Table 1), therefore, preventing audio transcriptions that platform is, therefore, highly recommended. Moodle, which
would be vital to visually impaired users. has become a leading learning management system world-
wide, is based upon a social constructivist learning model,
4.2 Lesson #2: On the Importance of a Pedagogical and by that reason offers resources and activities which
Strategy have the potential to maximize the learning effectiveness of
Remote observation and control can be fun, but care must remote experiments. The workshop activity4 constitutes a
be taken to ensure that its enormous potential for
constructivist and collaborative learning strategies is not 4. See http://docs.moodle.org/en/Workshops.
COOPER AND FERREIRA: REMOTE LABORATORIES EXTENDING ACCESS TO SCIENCE AND ENGINEERING CURRICULAR 347

perfect example to illustrate how a remote experiment can

be transformed into an embedded learning object that
serves an educational purpose [12]. This activity uses peer
review techniques to take the students through various
stages during the development of educational content. The
teacher sets the time line underlying a sequence of tasks
that lead to a final outcome delivered and evaluated
according to a predefined grading strategy.
The lesson to learn with this respect is that remote
experiments should not be seen as a goal in themselves, but
rather as a building block within a much wider instructional
design context. Laboratory work, be it on-lab at the
workbench or done remotely, is meant to complement the
knowledge acquisition process, where e-learning platforms
and blended learning strategies play a vital role.

4.3 Lesson #3: Evaluation of Pedagogical

Effectiveness
A fundamental question when considering remote experi-
ments to support any science or engineering degree is—do
remote laboratories offer an added value in pedagogical
terms? It is not trivial to answer this question, and many
remote experimentation projects were carried out over the
years without the participation of educational science
partners. This is not necessarily a mistake, but any
technical project that seeks an educational objective will
never reach a real validation stage, if the evaluation of
pedagogical effectiveness is under the responsibility of the
engineering team.
An appropriate framework to evaluate the pedagogical
effectiveness of remote laboratories comprises a set of
instruments and procedures for capturing evaluation data,
including questionnaires, interviews with students and
teachers, and the observation of how students interact and
collaborate when carrying out remote lab assignments.
Their application to selected case studies provides a
wealth of information that must then be processed in
statistical and interpretative forms. The results obtained
are likely to contain some surprises, particularly when
seen through the eyes of academic staff that are not related
to educational sciences.
The evaluation framework developed within the LABS-
ON-THE-WEB project comprised an online questionnaire
Fig. 2. Average values obtained for the evaluation dimensions, before
that was applied to the students before and after doing each and after the remote experiments (153 valid questionnaires before, 169
remote experiment. This questionnaire comprised seven after, in a total of five courses).
possible choices (1-7 with the highest value indicating
strongest agreement) that were used to grade the following 4. teacher interaction (including items related to the
dimensions: importance of collaborative work between students
and teacher and the availability of the latter).
1. knowledge and skills (e.g., acquisition of new
This framework was used to evaluate the pedagogical
knowledge; development of ICT skills, including
Information and Communication Technologies; pos- effectiveness of a set of remote workbenches used in five
sible development of new alternatives/solutions); courses, involving a total number of students that ranged
2. learning process (e.g., understanding the theoretical from 153 (concerning the first stage, before the experiment)
concepts underlying the remote experiments and the to 169 (after). Fig. 2 shows the average results obtained for
cause/consequence relationship that explains a each of the four evaluation dimensions referred above.
given result); The results showed small positive effects from the
3. peer cooperation (assessing the importance, the introduction of remote experiments on the Knowledge and
existence, and the possibility of collaborative inter- Skills and Learning Processes dimensions. However, the
action among students); surprise findings were that there was actually a small but
348 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 2, NO. 4, OCTOBER-DECEMBER 2009

significant negative effect on Peer Cooperation and Interac- is possible to envisage functional features embedded into
tion with the Teacher. This was considered particularly the interface panels, which do not exist in the real
important, because one of the beliefs associated with offering workbench equipment). This section will discuss a number
remote experiments, is that they will promote collaborative of issues that are seen by the authors as relevant
learning practices. A possible explanation for these results improvements to be expected in future generations of
lies on students’ over expectations concerning teacher remote laboratories.
interaction. Many students that are online most of their
day (and night . . . ) look at remote laboratories as a 24-hour 5.1 Discussion #1: Embedded Assessment
Features
link to their teachers, and become somehow disappointed
when they realize that teacher support is not going to Students in a real lab are normally accompanied by a teacher
experience a major upgrade under this new umbrella. that observes their work and provides help whenever
The need for realistic management of expectations, necessary. The teacher divides his/her attention within the
concerning interaction among students and between stu- whole class, meaning that each group will sometimes have to
dents and teachers, is therefore a key lesson that will benefit wait, or eventually follow a trial-and-error path while trying
both sides alike. Much like the relationship between e- to solve a problem (be it with the equipment or with the
learning and in-class lectures, remote labs should not be experiment), until succeeding or receiving help. Even when
seen as a replacement for in-lab assignments, but rather as a this trial-and-error process leads to a catastrophic error,
complement that facilitates access to educational activities, there’s not necessarily a problem in this approach, which is
while preserving much of the asynchronous nature of e- in fact at the very core of any exploratory learning model.
learning, when it comes to teacher support. Chat rooms and However, a price has to be paid when the teacher’s attention
other synchronous communication resources can be ex- is shared by say eight or more groups in a lab class. There’s a
cellent in helping students, but they are also terribly time wealth of formative assessment information that may be
consuming, and this affects their acceptance by academic collected just by observing how a student adjusts an
staff pressed by many other duties. instrument. The problem is that a teacher will only be able
to observe one group at a time, meaning that only a small
4.4 Lesson #4: Ease of Automation or Remote fraction of that information will actually be captured, when a
Control single teacher accompanies a class with eight workbenches.
One of the key lessons learned in PEARL and other projects However, if formative assessment features are embedded
is that one cannot necessarily automate processes or make into the experiment interface panels, ubiquity of the observer
them remotely controllable in an easy or cost effective becomes possible. If ethical and transparency rules are
manner. We found that processes that are very easy for a properly addressed, remote lab assignments of this type will
human to conduct are sometimes the most difficult to offer the students a much richer learning experience.
automate. This affects decisions about what experiments to Embedding formative assessment features into experi-
offer remotely and thereby puts restrictions on particular ment interfaces raises a number of problems that extend from
experiments that support part of the curriculum being data capturing to data mining. A reference model comprising
made accessible remotely. success spaces and rules to identify meaningful student
actions will have to be defined, requiring a close cooperation
4.5 Lesson #5: Learning Objectives and Design between software and education science teams. The example
Decisions represented in Fig. 1, showing an experiment interface that
Another key lesson is that the remote experiment team includes a two-channel oscilloscope, might be used to
needs to consider learning objectives of the activity during illustrate this discussion. A teacher observing how a student
the process of considering remote implementation. It is adjusts the oscilloscope would easily assess if the he/she is
necessary to weigh up the costs of implementation against familiar with the instrument, whether the student seems to
the benefits of offering a facility. For example, in PEARL, handle the buttons randomly without achieving his/her goal
the Open University found that implementing a facility to and be able to advise or provide help. When using interface
allow students to insert the grating into the spectrometer, panels to a remote oscilloscope automated assessments can
which involve highly accurate robotic systems, would not potentially achieve the same thing, regardless of the number
be worth the cost because this part of the activity was not of online workbenches in use at any moment, and without
central to the learning objectives of the activity: students depending on the availability of the teacher.
needed to learn that it needs to be done, but it was not A fundamental question will, however, have to be
necessary for them to actually do it to learn this. addressed to enable embedded assessment features—what
student actions are relevant for assessment purposes? The
answer to this question involves the concept of success spaces,
5 DISCUSSION where it becomes possible to map student actions against
A major conclusion derived from the authors’ experience is specific skills or knowledge objects. In the case of the
that remote laboratories do not have to replicate their real oscilloscope shown in Fig. 1, if the student is asked to adjust
counterparts. Most remote laboratories are developed to the time base (to avoid a mismatch between the frequency of
resemble the real workbench hosting the experiment, and the signal under observation and the selected time scale),
there is nothing wrong in doing so, as long as we one possible success space is represented in Fig. 3a [19].
understand that physical resemblance does not implicate Fig. 3b represents a sequence of student actions, captured
an exact replica of functional aspects (e.g., in the sense that it from a start point (time base and amplitude) and including six
COOPER AND FERREIRA: REMOTE LABORATORIES EXTENDING ACCESS TO SCIENCE AND ENGINEERING CURRICULAR 349

noticed that and just wanted to try something else, or simply

failed to realize it and continue to try out different settings
randomly. Various other situations may be envisaged where
it is possible to question if the conclusions derived by this
method are meaningful. However, we should keep in mind
that a very large number of such atomic assessment exercises
may take place during a lab assignment lasting for one or two
hours, so it will be possible to look at the assessment results
from a statistical point of view for validation purposes.
Embedded assessment features are still far from reality,
particularly because there is a great deal of research needed
to define feasible sets of success spaces, meaningful knowl-
edge objects, and validation procedures. To complicate
things a little further, it quickly becomes clear that they will
vary widely from one remote experimentation area to
another. They may also vary dynamically within a given
experiment, if the challenges posed to the students evolve in
real time as the experiment progresses. The definition of a
framework for embedded assessment in remote laboratories
used within science and engineering courses is an open
field where much research and discussion will take place in
the following years.
5.2 Discussion #2: Instructional Design Support
Designing a lab assignment, and particularly the experi-
ment script, is no longer what it used to be. Offline
workbenches were far easier to set up, and the technical
skills required from the teachers and lab technicians were
largely restricted to the instruments present and the
experiment itself. Preparing the workbench was essentially
related to defining how to stack and lay out the necessary
instruments and equipment. The interface to an online
workbench must be equally simple to set up, or otherwise
Fig. 3. Success spaces used to plot student actions. (a) Success space
for adjusting time or amplitude. (b) Student actions mapped into the teachers and technicians will have to master Internet and
success space. firewall technologies, JAVA programming, web page
development, etc. Most of them will not be interested to
settings, until the time available for the assessment of this take that step. Specific technical training for academics and
knowledge object has expired. The boundaries of the success technicians is unavoidable, but remote laboratories will not
space, for this specific example, were defined by the convert teachers into web programmers. Instructional de-
maximum and minimum time and amplitude settings sign support will have to be devised, namely in the form of
needed to ensure an acceptable visualization of the input experiment interface design and production, integration
signals. This example is also useful to trigger some important into e-learning platforms, etc.
questions that are still open to discussion. The case of interface design and production is particu-
What student actions are relevant and deserve to be larly interesting for discussion within this context, since it is
logged into the assessment report, to be made available to relatively simple to envisage a library of instrument panels,
the student and to the teacher? If the student completed the communication gadgets (e.g., chat or videoconferencing
adjustment exercise within a standard time interval, there is windows), file transfer tools, etc.
nothing to log and no special action needs to be taken. A library manager tool might be offered to the teacher as
However, if he/she hadn’t yet entered the success space represented conceptually in Fig. 4a. The standard experi-
within the time limit set for this exercise, we’re likely in the ment interface, generated automatically when pressing the
presence of a student that is not yet sufficiently familiar “Generate interface” button, would ideally correspond to
with the operation of the oscilloscope, and this is mean- the layout represented in Fig. 4b and might be customized
ingful formative assessment information that deserves to be by dragging and dropping the rectangles corresponding to
logged. The same happens if the student has proven his/her each component.
knowledge in a very short time (e.g., <20 percent of the Figs. 5 and 6 show two examples of experiment
time limit), since we’re likely in the presence of a student interfaces generated by a prototype implemented from this
that may need a more challenging goal in order to feel conceptual specification [20]. The case study represented
motivated by the tasks proposed. here corresponds to a remote experiment dealing with
The reader may have noticed that in the case of Fig. 3b the structural fault detection on mixed-signal boards support-
student had actually entered the success space within the ing the IEEE 1149.1 and 1149.4 standards [21], [22]. The test
time limit set for this exercise. It is questionable if he/she program is specified in SVF [23] on the right side of the
350 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 2, NO. 4, OCTOBER-DECEMBER 2009

Fig. 6. Instruments and videoconferencing modules for the remote test

experiment.

and functional and portability issues, are reasonably

standardized, there is a wide variety of remote laboratories.
The development of tools for automatic generation of
experiment interfaces will not only facilitate the work of
teachers and technicians, promoting the widespread adop-
tion of the remote laboratories, but will also simplify the
development of remote laboratory grids, and content
sharing (of remote experiments) among institutions in
similar science and engineering areas.

5.3 Discussion #3: Pedagogical Strategies for

Collaborative Learning Based on Remote Labs
Remote laboratories and collaborative learning are not
necessarily linked. However, lab assignments (local or
remote) are usually carried out as group work, and therefore
offer an excellent opportunity to support collaborative
learning strategies. Their potential to support exploratory
learning models is also worth mentioning, since the lab
script may be explicitly designed to trigger group discussion
Fig. 4. Experiment interface panels: Conceptual design support. and the acquisition of knowledge needed to explain the
(a) Specification of the interface components. (b) Experiment interface experiment results. This social constructivist vision explains
generated according to (a). why some Moodle resources and activities (e.g., the workshop
activity referred in a previous section) simplify one of the
panel shown in Fig. 5, and the test setup and test results are main challenges faced by teachers—how to convert a remote
shown on the left. The oscilloscope, voltmeter, waveform experiment into a meaningful learning object. Pedagogical
generator, and videoconferencing channels are provided in strategies play a key role in what concerns the learning
a separate window to minimize vertical scrolling, as shown effectiveness of remote laboratories, but satisfactory solu-
in Fig. 6. tions/frameworks are still lacking. The fundamental pro-
Instructional design support in the form described in [20] blem is that remote experiments are still very frequently
is yet in the research and development stage. One of the offered as stand-alone activities, ignoring the potential that
reasons for this delay derives from the lack of de facto derives from being able to include in the same browser
standards concerning technologies and platforms to sup- window such powerful resources as the online workbench,
port the development of remote laboratories. Contrary to
the e-learning platform, search engines and online re-
the wider e-learning scenario, where enabling technologies,
sources, chat and videoconferencing, etc.
The development of a good pedagogical wrapper for
remote experiments calls for the cooperation of education
scientists and e-learning experts. This work is yet to be done,
particularly in what concerns the collaborative features and
how to embed them effectively into the remote experiment
script. Fig. 7 shows two examples of Moodle activities that
illustrate what we’re discussing in this section.

. The forum provides a discussion area where students

and teachers may discuss experiment procedures or
Fig. 5. Test program specification, test setup, and results for the remote results, clarify doubts, query each other, etc. A
test experiment. forum is one of the simplest collaborative activities
COOPER AND FERREIRA: REMOTE LABORATORIES EXTENDING ACCESS TO SCIENCE AND ENGINEERING CURRICULAR 351

Fig. 8. A remote lab located in the factory shop-floor.

technological and pedagogical aspects and is partly

responsible for the widespread acceptance of this platform).

5.4 Discussion #4: Remote Labs @ Industry

Remote labs are usually seen as online workbenches that are
located in the university campus, offering access to students
from their homes. However, there are other scenarios that
can be envisaged, offering pedagogical and other benefits
that go beyond the limitations of a strictly academic setting.
Remote access to sophisticated equipment located in the
factory shop-floor is a good example of such scenarios, as
illustrated in Fig. 8.
Fig. 7. Collaborative learning activities supported by Moodle. (a) A
In many cases, remote access to shop-floor equipment
Moodle Discussion forum. (b) A Moodle wiki. may already be in use inside the company. The SDRAM test
system illustrated in Fig. 9 is a good example of what has
within Moodle, yet few teachers will associate a just been said. Test engineers, particularly those working in
forum to each experiment, and those that do so will test program development, work from their desks instead of
frequently find the results disappointing for lack of walking to the factory shop-floor.
interest from the students. Due to their very high cost, SDRAM testers are an
. A wiki is another example of a powerful collabora- excellent example to illustrate the advantages of remote labs
tive activity supported by Moodle. Slightly more to industry [24]. Generally speaking, the benefits of
complex than a discussion forum, a wiki stores a envisaging remote access to shop-floor equipment may be
shared document that can be modified by the summarized as follows:
authorized participants. All modifications are auto-
matically registered, showing how the document . offers the students access to specialized knowledge
evolved. If the experiment script asks the students to that may not exist within the university;
present a theory explaining the observed results, a . represents an opportunity to work with sophisti-
wiki is an excellent choice for doing that work cated equipment that may be too expensive or too
collaboratively, and enables the teacher to follow the specific to exist in a university;
participation of each group member. However, . improves employability of the students, by offering
students will tend to ignore this activity, if they them skills that are directly relevant to companies,
don’t perceive a coherently integrated overall ped- and enabling the company to assess and select the
agogical strategy. students;
. enables the companies to have a say concerning
Collaborative activities offer an excellent framework to
academic curricula and required student skills;
improve the learning effectiveness of remote experiments, . offers the students a glimpse of their future profes-
but very little pedagogical research has been done so far in sional life before they leave the university (organiza-
this area. Remote laboratories are still very much a tional and cultural habits, time and activities
technological subproduct of the Internet revolution, and management, etc.);
most of them result from projects that addressed technical . last but not the least, remote labs at industry bridge
challenges, leaving the pedagogical aspects in the back- the gaps with academia and facilitate the deploy-
ground. The development of e-learning platforms suffers ment of joint educational and research programs.
less from this problem, not only because it started much In spite of the many advantages in this form of
longer ago, but also because some such platforms have an cooperation between academia and industry, good exam-
explicit underlying pedagogical vision (the social construc- ples of remote labs located at industry premises are still
tivist learning model of Moodle, for example, binds very scarce. There is yet a significant amount of work that
352 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 2, NO. 4, OCTOBER-DECEMBER 2009

There is a pedagogic advantage of remote labs in that

experiments can usually be offered at the time the students
undertake the related areas of study. This may be difficult
to quantify as a cost benefit but funding is often linked to
student completion rates and grades, so it may be possible.

6 CONCLUSIONS
In summary, this paper has given a case for the role of
remote experiments in extending access to teaching labs
and other practical work as part of principally science and
engineering education. Remote laboratories have come a
long way since their introduction in the late 1980s/early
1990s. The last decade witnessed a move from breaking
technological barriers to the enhancement of pedagogical
features, and the next generations will undoubtedly include
embedded tutoring and personal assessment features that
will improve their pedagogical effectiveness still further. In
spite of 20 years of activity and accumulated experience, it
is remarkable that some obstacles remain to the widespread
adoption of these solutions. The public perception of remote
laboratories has improved, but it is still important to stress
that in most cases we are talking about a complement to lab
classes, and not of their replacement. They enable the
students to rehearse, complete, repeat, or extend their lab
work, offering flexible access to lab spaces that would
otherwise remain out of reach when the lab is closed. At the
same time, they represent an extension to e-learning
Fig. 9. SDRAM test equipment and the test engineers working technologies, and in this way prevent Moodle and other
environment. (a) Advantest SDRAM tester. (b) Advantest development learning management systems from dying at the laboratory
tools.
door. The integration of remote experiments with the
remaining e-learning contents offers an enormous potential
needs to be done in this area, not so much in terms of
for improving the pedagogical success of science and
pedagogical or organizational research, but rather in terms
engineering students, and their contribution with this
of disseminating good practices, identifying good opportu-
respect will be even more important, as the next generations
nities, establishing the necessary cooperation channels, and
developing the corresponding educational programs. depart from near-replicas of workbench equipment and
evolve into adaptable learning companions, assessing and
5.5 Discussion #5: Cost Effectiveness tutoring students as they do their lab work. This paper has
In most cases it is more costly to offer an experiment also highlighted the potential for remote labs to give
remotely than face-to-face in the lab. This additional cost comprehensive access to practical work that might other-
needs to be justified. The justification normally comes from wise be denied them including students with disabilities.
meeting issues of access. This may be:

. access to expensive or safety critical equipment; ACKNOWLEDGMENTS

. access any time form anywhere to limited lab space A large number of people have been involved in the
and equipment; projects referred in this paper. Among many individuals
. improved access for disabled students; and institutions, the authors would particularly like to
. integration into distance learning programs; thank the support and contributions coming from Prof.
. sharing of lab facilities between institutions or across Isabel M. Figueiredo (Faculdade de Psicologia e Cincias da
sites at an institution. Educação da Universidade do Porto), Joaquim Faias (Escola
So, the additional cost of making any lab facility Superior de Tecnologia da. Saúde do Porto), Ana C. Leão
available online can, normally at least, be offset by the (Qimonda), Professor Eileen Scanlon and Dr. Chetz Colwell,
number of students that can use it in a given period. Many
both of the Institute of Educational Technology, Open
institutions are finding it increasing difficult to provide
comprehensive labs to all the students on their courses. University. This work was supported in part by grants
Remote labs can offer a solution to this maximizing the received from various national and international programs,
number of students that can use an item of equipment. Cost including the European Commission IST and Leonardo da
effectiveness is one criteria to evaluate in deciding what Vinci programmes, the Portuguese POCI 2010 programme
specific experiments to offer remotely and in their design; from the Ministério da Cincia, Tecnologia e Ensino Super-
not just the adoption of the overall approach. ior, and Open University Strategic Research Funds.
COOPER AND FERREIRA: REMOTE LABORATORIES EXTENDING ACCESS TO SCIENCE AND ENGINEERING CURRICULAR 353

REFERENCES [24] J.M. Ferreira and A.C. Leão, “An SDRAM Test Education Package
that Embeds the Factory Equipment into the E-Learning Server,”
[1] J.V. Nickerson, J.E. Corter, S.K. Esche, and C. Chassapis, “A Proc. Int’l Conf. Remote Eng. Virtual Instrumentation (REV ’08), June
Model for Evaluating the Effectiveness of Remote Engineering 2008.
Laboratories and Simulations in Education,” Computers &
Education, vol. 49, no. 3, pp. 708-725, Nov. 2007. Martyn Cooper received the BSc degree in
[2] E.D. Lindsay, S. Naidu, and M.C. Good, “A Different Kind of cybernetics and control engineering with sub-
Difference: Theoretical Implications of Using Technology to sidiary mathematics from the University of
Overcome Separation in Remote Laboratories,” Int’l J. Eng. Reading in UK. He has been a systems engineer
Education, vol. 23, no. 4, pp. 772-779, 2007. for over 28 years, the last 18 working on
[3] J. Ma and J.V. Nickerson, “Hands-On, Simulated, and Remote enabling technology for people with disabilities
Laboratories: A Comparative Literature Review,” ACM Computing and the last 11 of that particularly in access to
Surveys, vol. 38, no. 3, Sept. 2006. education. Since 2001, he is a senior research
[4] D. Gillet, A.V. Nguyen, and Y. Rekik, “Collaborative Web-Based fellow at the Open University (OU), where he is
Experimentation in Flexible Engineering Education,” IEEE Trans. the head of the Accessible Educational Media
Education, vol. 48, no. 4, pp. 696-704, Nov. 2005. (AEM) group in the Institute of Educational Technology (IET). He is
[5] E.D. Lindsay and M.C. Good, “Effects of Laboratory Access currently responsible for accessibility issues across teaching and
Modes upon Learning Outcomes,” IEEE Trans. Education, vol. 48, learning and infrastructure projects within the university as well as
no. 4, pp. 619-631, Nov. 2005. leading the groups externally funded research in this field. Since 1997,
[6] J.E. Corter, J.V. Nickerson, S.K. Esche, and C. Chassapis, he has been periodically used as an independent expert, in this area, by
“Constructing Reality: A Study of Remote, Hands-On, and the European Commission. He has participated in a range of learning
Simulated Laboratories,” ACM Trans. Computer-Human Interaction, technology standardization efforts intended to promote accessibility.
vol. 14, no. 2, Aug. 2007. This includes within IMS Global Learning Consortium, CEN-ISSS, and
[7] T. Abdel-Salam, P. Kauffman, and G. Crossman, “Does the Lack of the Dublin Core Metadata Initiative. He is currently leading the metadata
Hands-On Experience in a Remotely Delivered Laboratory Course and standardization work within the large-scale EU IST project EU4ALL.
Affect Student Learning?” European J. Eng. Education, vol. 31, no. 6, This project is developing technical and service frameworks for
pp. 747-756, Dec. 2006. accessible lifelong learning.
[8] F. Biocca, J. Burgoon, C. Harms, and M. Stoner, “Criteria and
Scope Conditions for a Theory and Measure of Social Presence,” José M.M. Ferreira received the BS, MSc, and
Proc. Presence, 2001. PhD degrees in electrical and computer engi-
[9] B. Fraser and H. Walberg, “Research on Teacher-Student Relation- neering from the University of Porto, Portugal, in
ships and Learning Environments: Context, Retrospect and 1982, 1987, and 1992, respectively. In 1982, he
Prospect,” Int’l J. Educational Research, vol. 43, pp. 103-109, 2005. joined the University of Porto (FEUP), where he is
[10] Z. Nedic and J. Machotka, “Remote Laboratory NetLab for currently an associate professor at the Depart-
Effective Teaching of 1st Year Engineering Students,” Int’l J. ment of Electrical and Computer Engineering. He
Online Eng., vol. 3, no. 3, 2007. also holds a position of professor II at the
[11] M. Cooper, A. Donnelly, and J.M. Ferreira, “Remote Controlled Buskerud College of Engineering, Kongsberg,
Experiments for Teaching over the Internet: A Comparison of Norway. His nonacademic experience includes
Approaches Developed in the Pearl Project,” Proc. 19th Ann. Conf. Texas Instruments and Efacec, where he worked as a design engineer in
Australasian Soc. for Computers in Learning in Tertiary Education, the area of industrial electronics. He is the author of one book on digital
2002. hardware design, coauthor of approximately 100 papers, and served as a
[12] J.M. Ferreira and D. Mueller, “The MARVEL EU Project: A Social reviewer for several journals and conferences. His academic interests are
Constructivist Approach to Remote Experimentation,” Proc. First related to research and education in digital hardware development and
Remote Eng. and Virtual Instrumentation Int’l Symp. (REV ’04), test, with an emphasis on e-learning and assessment of pedagogical
Sept. 2004. effectiveness, where he participated in various national and international
[13] J.M.M. Ferreira, E.M. Almeida, I.M. Figueiredo, and C.F. Leite, projects. He is responsible for the Teaching and Learning Laboratory of
“Labs-on-the-Web: A Multidisciplinary Project to Evaluate the the Faculty of Engineering of the University of Porto, which participates in
Pedagogic Effectiveness of On-Line-Labs,” Proc. European Distance various projects together with the Faculty of Psychology and Educational
and E-Learning Network Ann. Conf. (EDEN ’08), June 2008. Sciences of the same university.
[14] L. Nevile, M. Cooper, A. Heath, M. Rothberg, and J. Treviranus,
“Learner-Centred Accessibility for Interoperable Web-Based
Educational Systems,” Proc. 14th Int’l World Wide Web Conf.
(WWW ’05), May 2005.
[15] M. Cooper, “Making Online Learning Accessible—The Role of the
Educator and Issues for the Educational Institution—Reflections
on Experiences at the Open University,” Assoc. of Learning
Technology J., vol. 14, no. 1, pp. 103-115, Mar. 2006.
[16] NI LabView—The Software That Powers Virtual Instrumentation,
http://www.ni.com/labview, 2009.
[17] National Instruments, Accessibility Requirements Compliance
Matrix, www.ni.com/pdf/gsa/en/labview-508-accessibility-
requirements-compliance-matrix.pdf, 2009.
[18] J. Faias, J.M. Ferreira, and C. Leite, “Accessibility and Usability of
Online Labs: Real Barriers in a Virtual World,” Proc. Int’l Conf.
Remote Eng. Virtual Instrumentation (REV ’07), June 2007.
[19] M. Conradi, R. Langmann, A.M. Cardoso, and J.M.M. Ferreira,
“Embedding Formative Assessment Features into LabView
Interfaces,” Proc. Int’l Conf. Remote Eng. Virtual Instrumentation
(REV ’07), June 2007.
[20] J.R. Alves, G.R. Alves, P.A. Ferreira, and J.M.M. Ferreira, “REXIB:
Remote Experiments Interface Builder,” Int’l J. Online Eng., vol. 2,
no. 3, 2006.
[21] IEEE Std 1149.1-1990, IEEE Standard Test Access Port and Boundary-
Scan Architecture, 2009.
[22] IEEE Std 1149.4-1999, IEEE Standard for a Mixed-Signal Test Bus,
2009.
[23] Serial Vector Format Specification, www.jtagtest.com/pdf/
svf_specification.pdf, 2009.