Republic of Iraq

Ministry of Higher Education

& Scientific Research
AL-Nahrain University
Information Engineering College
Systems Engineering Department

Robot Simulation

4th year - Report

in
System Modeling & Simulation

By
Haider Hashim Dkhikh

Supervised by
Dr. Rajaa Aldeen Abd

July 2020
 Abstract

Simulation is well established in robotics and nearly every robot project benefits

from it. Thereby the understanding on what in-sights a simulation shall bring or how it

shall assist the development is totally different in every project. The main reason is that

simulation covers all aspects in the robot development process as for example learning

and tuning control parameters or testing the composition of algorithms in a robot

architecture. Further, the broad application field led to a large number of different

simulation environments where, depending on the application, some are designed to

simulate special purpose hardware and others aim to target a wide range of robotic

systems. In this report we present an overview on different use cases for simulation in

robotics. Dependent on the use cases we propose a categorization for robot

simulators/emulators and present some concrete environments.

Haider Hashim 2|Page

1. Introduction

Simulation has a long-standing tradition in robotics as a useful tool for testing

ideas on a virtual robot in a virtual setting before trying it on a real robot. However,
when robots became an affordable commodity for research groups in the late 80s and
early 90s, it became difficult to publish results that had not been obtained on real robots
and in simulation only; as a consequence, simulation went almost out of fashion and
was mainly used in specific sub-communities like swarm robotics and robot learning.
However, in the last decade, simulation in robotics is getting more attention again. One
good reason for this is that the computational power of computers has been increasing
significantly which makes it now possible to run computationally intensive algorithms
on personal computers instead of special purpose hardware.

Another reason is the increased effort of the game industry to create realistic
virtual realities in computer games. The creation of virtual worlds requires a huge
amount of processing power for graphical rendering and physics calculations. Thereby,
the game industry developed software engines which, at least in principle, seem
capable of providing high quality physic simulation and rendering software in the
robotics domain. Since the goals of computer gaming and robotics simulations are quite
similar the creation of a realistic virtual counterpart of a real world use case, robotics
simulation environments can reuse these simulation and graphics engines and profit
from the gain in processing power. The relationship between computer games and
scientific research is more deeply analyzed in [1].

Haider Hashim 3|Page

2. Use Cases and Objective for Robot Simulation
Below, we describe some motivations and benefits of using simulation in robotics;
some of those use cases also illustrate how simulators are applied in practice.

2.1 Speeding Up the Development Process:

At the start of a new project or the development of a new product, the hardware often
is not available. The time required for designing a robot, procuring off-the-shelf
components, manufacturing custom-designed parts, and assembling the overall system
can be significant. Rumors have it that this process can get so much delayed even in
otherwise well-managed European research projects that projects are at risk to miss
their original objectives because software development could start only after delivery
of the hardware. Simulation can be of great help. Just as electronics design nowadays
is able to develop graphics cards and mother-boards long before the actual GPUs or
CPUs become physically available, robotics needs the capability to develop software
functionality without using the actual target hardware. Thereby, the availability of
suitable simulators can speed up the development process enormously.

2.2 Producing Training Data for Offline Learning:

Most learning techniques require a large amount of training data, which often is
difficult and time-consuming to obtain using physical robots. Often, an appropriate
simulation environment can be used to produce such data much faster and in almost
arbitrary quantity. Also, in simulations it is often easier to generate such data with
sufficient coverage and even distribution across the data space than using real robots.
Even if the data produced by a simulation may be different from real-world data,
especially with respect to the amount and distribution of noise, it is often sufficient for

Haider Hashim 4|Page

performing a first learning phase and generate functionality, which can be fine-tuned
in a second learning phase using real-world data from a robot, however with much,
much less training cycles Tobe performed on the physical system [2].

2.3 Speeding Up Interactive Learning:

Some learning approaches make use of experience obtained from the interaction of
the robot with its environment, e.g. reinforcement learning, genetic algorithms, and
other evolutionary approaches. However, most of them need a large number of such
interactions before they converge or even something useful could be learned. For
example, Zagat presents an approach [3], where the control parameters for the four-
legged robot AIBO have been learned based on simulation which took600 learning
iterations, each taking 20 sec in real time. Often, the number of iterations required is
prohibitive for applying such approaches on real robots, and simulations, which
execute virtual interactions at rates often exceeding100 or 1000 times the interaction
rate of a physical robot, offer the only way to apply these approaches and get acceptable
learning effort.

2.4 Enabling Online Learning:

When online learning approaches are used, it is sometimes very beneficial to apply
learning experience acquired online to a number of similar virtual situations using
simulation. Vice versa, alternative actions than the one that was selected and executed
can be tried in simulation. Exploiting these opportunities often results in dramatic
speed-up of the learning process.

Haider Hashim 5|Page

2.5 Sharing Resources:
Due to the high investment necessary for sophisticated robot hardware platforms,
several researchers or developers may have to share the same hardware platform. This
makes the robot platform a potential bottleneck and often leads to resource contention,
especially when due dates or project reviews are approaching. The use of appropriate
simulators can remedy this situation.

2.6 Permitting Distributed Development:

Like in many large software projects, development of sophisticated software for
service robotics is nowadays often performed by spatially distributed groups of
researchers and developers. Examples include e.g. the recently completed German
project DESIRE as well as many European projects, e.g. CogX or Robocasting. If the
project has only a single platform available, simulation can serve here both for sharing
resources and permitting distributed development. Sometimes, several platforms
aroused, but use different hardware components or different configurations. By using
simulator models for each of these platforms and sharing them among the group, they
can help to ensure that all developed software runs on all platforms in the consortium.

2.7 Compensating Resource Unavailability:

The period an autonomous robot can be operated autonomously depends on its battery
power and, depending on robot and battery size, ranges usually between less than an
hour and few hours. The time period needed to recharge the batteries often exceed the
operation period by a factor of 2 to 4, during which the robot cannot be used for
experiments involving navigation. Maintenance or calibration work that occasionally
needs to be performed on the robot will further increase its unavailability. Simulators
can help a lot to compensate for the unavailability of the real hardware platform.
Haider Hashim 6|Page
2.8 Improving Safety and Security:
Simulators can also help to improve safety and security. Especially when dealing
with heavy or very fast robot equipment, programming errors could result in dangerous
or damaging behavior with severe consequences. Test-driving such equipment first in
a simulator can help to prevent such problems.

2.9 Increasing Thoroughness of Testing:

Finally, simulators can be instrumental to increase the thoroughness of testing robotic
equipment and applications, because they allow to safely assess a robot’s behavior in
unusual and extreme situations, which would otherwise be difficult to produce in real-
world situations or which would incur potentially severe damages.

3. Simulation challenges
simulation also faces some tough challenges. Creating the models of the
environments and the robots at a level of detail which allows simulation with believable
physics and/or photo-realistic graphics is a very challenging task. Running these
models in simulators can easily exceed the limits of even the most advanced
computational hardware. Thus, simulation model builders must make difficult tradeoffs
between model precision and runtime performance. Another challenging problem is
how to validate and debug models. There is little knowledge about systematic model
validation in the community, as this is rarely in the curriculum of robotics courses.
Complex simulation models can also be notoriously hard to debug, if model validation
yields significant defects and aberrations. Summarizing, there currently exists no
general-purpose simulator that would qualify for all of the above tasks. In the following
sections, different types of simulation environments and their properties will be
described.
Haider Hashim 7|Page
4. Simulators Used in Robotics (Application)
Since different research projects have different requirements with respect to the
features and capabilities of a suitable simulator, a number of different simulation
environments have been developed in robotics. Although many off them are somehow
available at no or nominal cost, many systems are not maintained or further developed
any more, sometimes already for many years. Such systems are difficult to use and
adapt to up-to-date requirements concerning the supported platform models and
graphics capabilities, and have little utility for new projects. The differences in the
requirements posed by particular projects are often related to the tradeoff to be chosen
between simulator performance and model precision. Since it is not possible by
principle to simulate a robot and its environment exactly as in the real world, different
simulation approaches have evolved that focus on different problem aspects.

4.1 Numerical Simulators

The simulators providing the highest precision with respect to physics are numerical
simulators. They are usually used to simulate single devices, like motors or circuits,
come with tools for the inspection of signals, and often do not support graphical
visualizations of the robot and its environment. A widely used commercial and very
versatile environment is Matlab/Simulink by MathWorks [4]. It allows the graphical
modeling of dynamical robot systems using block charts and the inspection and plotting
of the generated output signals.

Haider Hashim 8|Page

4.2 Algorithm Evaluation
Some simulators target a particular step in the robot development process, the selection
of an algorithm or computational approach for a particular robot functionality. In order
to make the right choice, we often need to compare and evaluate algorithms in specific
settings. An example would be choosing a path planning approach for a mobile
manipulator, built e.g. by combining one of several alternatives for a mobile base
(differential, omnidirectional) and a particular manipulator. For such uses of a
simulation environment, it should support the analytic evaluation of experiments by
appropriate statistical tools.

4.3 Robot System Evaluation

The objective of the simulators described in this section is to evaluate complete robot
systems. The simulated robot model can be controlled with the complete robot system
and control architecture. Therefore, these systems provide a richest of different sensors
like ultrasonic sensors, infrared sensors, laser scanners, force sensors, or cameras to
name only a few. Furthermore, different mobile robot bases and/or manipulators are
often available as predefined models and can be used or adapted to build a custom robot
model. Since the simulator and the robot control software should be interfaced in a
manner that requires as little change to the robot code as possible, integrated solutions
are provided, which combine a robot software framework and a simulation
environment.

Microsoft Robotics Developer Studio: (MRDS) provides a web service-

oriented software framework for robot development which is coupled with a graphical
programming tool and a 3D simulation environment. The simulation environment is
based on the commercial physic engine PhysX which allows hardware accelerated

Haider Hashim 9|Page

physic calculations. Simulations are visualized based on Microsoft XNA engine. The
environment allows for the import of Collada models, but is commercially published.

USARSim: has been developed at CMU for the RoboCup Rescue competition and
aims at the simulation of Urban Search and Rescue environments and robots
Automation. It is based on a commercial computer game engine name Unreal [11] [12].

4.4 Learning Algorithms

An application area where simulation is very beneficial is learning, e.g. to obtain
optimal control parameters for a walking robot. An essential requirement for simulators
used for learning is excellent runtime performance since learning algorithms often need
hundreds or thousands of iterations of the same experiment. Therefore, the simulator
must be capable to run experiments faster than real-time. Nevertheless, these
simulators often need to simulate physics so that the learned parameters are applicable
in real world. Therefore, the environment may have been precisely simulated, e.g.
when the robot has to learn how to climb stairs. The simulators discussed here often
provide 3D visualization capability, but usually the visualization can be completely
deactivated; for the learning out-come it is only important to record and archive the
Haider Hashim 10 | P a g e
parameters providing the best performance. A problem that simulation on this level
faces eminently is the so-called reality gap [14]. It describes the problem that behaviors
which are acquired in simulation may not behave exactly the same way or even
completely fail if transferred to the real world. This directly relates to the complexity
of the behavior or parameters that the robot shall learn.

UCHILSIM: is a simulation environment developed for the four-legged league

RoboCup competition at the University of Chile [3]. It uses the Open Dynamics Engine
(ODE) as physics engine. Models are defined in a customized Virtual Reality Modeling
language (VRML) format. Simulated robots can be controlled via an interface [15]
based on the Open-R API for the AIBO robots.

YARS: claims to be the fastest simulator for physical 3D simulation [17]. It targets
the application of evoutionary learning algorithms and is also based on the ODE
physics engine. Robot models can be described with the Robot Simulation Language.

Haider Hashim 11 | P a g e
4.5 Learning Team Behaviors
The most abstract types of simulators are used for analyzing the behavior of multirobot
teams or robot swarms. Since the precise simulation of the interaction of one or more
complex robot systems with a reasonably sophisticated environment is computationally
very challenging, these systems tend to use simplified robot and environment models.
For swarm behavior, robots are often represented just as boxes and with simplified
sensor models, which can be defined and configured without the support of
sophisticated modeling tools and complex description languages and formats.

Mason: is a simulator for multiagent simulation [18] and swarm robotics. It is one
of only a few simulation environments developed in Java. To achieve high
performance, an appropriate visualization level can be chosen: 3D, 2D.

BREVE: is a simulation environment for artificial life [19] and was not developed
specifically for robotics.

MuRoSimF: is a special simulation environment for robot teams, but with flexible
level of detail [20]. The simulation algorithms can be distributed over different CPUs.
It has been developed for the RoboCup Humanoid soccer competition.

Haider Hashim 12 | P a g e
5. Conclusions and Result
The available simulators for robotics all have very specific target uses. There is
no all-in-one solution which would be usable and suitable for any simulation-related
task during the robot development process. Sometimes it may be necessary to use
multiple simulation environments for a single project or robot development, each
targeted towards solving a particular problem on hand, and requiring a particular
tradeoff between model precision and performance to be chosen. The same holds true
for the techniques used to build simulators. Another major issue is the integration of
the simulation environment and the robot control software. Integrated solutions like
Player or the MRDS exist, but there are no standardized interfaces which a simulator
shall could support. Interfacing can be done via shared memory, custom API’s, custom
TCP/IP interfaces. Integration of simulators with commonly used software frameworks
like Player and ROS are getting more wide-spread. Furthermore, there are no standards
available for modeling robots and environments, or even data exchange. Often, it is
possible to store models in XML; standards developed elsewhere, like Collada and
VRML, are becoming more frequently used but are not capable to persist models in a
way that they can be exchanged between different simulation systems. The reason for
that is that these standards have not been specifically designed for robot simulations
and are missing important features like the description of sensor behaviors. Many
simulators evolve from particular research projects, and especially the RoboCup
community provides many contributions in this area. A drawback of this situation is a
lack of support and continuity in the development after some time, so that many good
simulators become less and less usable or never result in a stable version. Generally
speaking, for a specific task an appropriate simulator has to be carefully chosen.
Simulation can be a great help in the robot software development process, if the right
choice of simulator is made with the right tradeoff between performance and model
precision for the specific task.

Haider Hashim 13 | P a g e
References
[1]. M. Lewis and J. Jacobson, “Game Engines in Scientific Research,” in
Communications of the ACM , vol. 45, pp. 43–45, 2012.
[2]. T. Gabel, R. Hafner, S. Lange, M. Lauer, and M. Riedmiller, “Learning in the
RoboCup Simulation and Midsize League,” in Proceedings of the 7th Portuguese
Conference on Automatic Control (Controlo 2006), (Porto), Portuguese Society of
Automatic Control, 2011.
[3]. J. C. Zagal and J. R. del Solar, “UCHILSIM: A Dynamically and Visually Realistic
Simulator for the RoboCup Four Legged League,” in RoboCup 2004: Robot Soccer
World Cup VIII , vol. 3276/2005, pp. 34–45, Springer, 2005.
[4]. Mathworks, “Simulink.” [Online] http://www.mathworks.de/, February 2010.
[5]. P. W. Tuinenga, Spice: A Guide to Circuit Simulation and Analysis Using Spice
Upper Saddle River, NJ, USA: Prentice Hall PTR, 2012.
[6]. A. Miller and P. Allen, “GraspIt! A Versatile Simulator for Robotic Grasping,”
Robotics Automation Magazine, IEEE, vol. 11, pp. 110 – 122, Dec. 2014.
[7]. R. M. Murray, S. S. S. , and L. Zexiang, A Mathematical Introduction to Robotic
Manipulation. Boca Raton, FL, USA: CRC Press, Inc., 1994.
[8]. R. Diankov and J. Kuffner, “OpenRAVE: A Planning Architecture for
Autonomous Robotics,” Tech. Rep. CMU-RI-TR-08-34, Robotics Institute, Carnegie
Mellon University, Pittsburgh, PA, July 2018.
[9]. K. Dixon, J. Dolan, W. Huang, C. Paredis, and P. Khosla, “RAVE: A Real and
Virtual Environment for Multiple Mobile Robot Systems,” in Proceedings of the
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’99) vol.
3, pp. 1360–1367, October 2010.
[10]. N. Koenig and A. Howard, “Design and Use Paradigms for Gazebo, An Open-
Source Multi-Robot Simulator,” in In IEEE/RSJ International Conference on
Intelligent Robots and Systems, pp. 2149–2154, 2014.
[11]. S. Carpin, M. Lewis, J. Wang, S. Balakirsky, and C. Scrapper, “USARSim:a
Robot Simulator for Research and Education,” in International Conference on Robotics
and Automation , pp. 1400 – 1405, 2017.
[12]. C. Scrapper, S. Balakirsky, and E. Messina, “MOAST and USARSim: A
Combined C framework for the Development and Testing of Autonomous Systems,”
in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol.
6230of Presented at the Society of Photo-Optical Instrumentation Engineers
(SPIE)Conference, June 2016.

Haider Hashim 14 | P a g e
[13]. O. Michel, “Cyberbotics Ltd. Webots TM : Professional Mobile Robot
Simulation,”Int. Journal of Advanced Robotic Systems , vol. 1, pp. 39–42, 2004.
[14]. N. Jakobi, P. Husbands, and I. Harvey, “Noise and the Reality Gap: The Use of
Simulation in Evolutionary Robotics,” in Advances in Artificial Life, vol. 929/1995 of
Lecture Notes in Computer Science, pp. 704–720, Springer, 2010.
[15]. J. C. Zagal, I. Sarmiento, and J. R. del Solar, “An Application Interface for
UCHILSIM and the Arrival of New Challenges,” in RoboCup 2005: Robot Soccer
World Cup IX , vol. 4020/2006, Springer, 2006.
[16]. J. C. Zagal, J. R. del Solar, and P. Vallejos, “Back to Reality: Crossing the Re-
ality Gap in Evolutionary Robotics,” in IAV 2004 the 5th IFAC Symposium on
Intelligent Autonomous Vehicles , 2014.
[17]. K. Zahedi, A. von Twickel, and F. Pasemann, “YARS: A Physical 3D Simula-tor
for Evolving Controllers for Real Robots,” in SIMPAR ’08: Proceedings of the 1st
International Conference on Simulation, Modeling, and Programming for Au-
tonomous Robots (S. Carpin, ed.), pp. 75 – 86, Springer, 2008.
[18]. S. Luke, C. Cioffi-Revilla, L. Panait, K. Sullivan, and G. Balan, “MASON:
AMultiagent Simulation Environment,” Simulation , vol. 81, no. 7, pp. 517–527, 2005.
[19]. J. Klein, “BREVE: A 3D Environment for the Simulation of Decentralized
Systems and Artificial Life,” in Proceedings of the Eighth International Conference on
Artificial Life, pp. 329 –334, 2013.
[20]. M. Friedmann, K. Petersen, and O. von Styrk, “Simulation of Multi-Robot Teams
with Flexible Level of Detail,” in SIMPAR ’08: Proceedings of the 1st International
Conference on Simulation, Modeling, and Programming for Autonomous Robots
,(Berlin, Heidelberg), pp. 29 – 40, Springer-Verlag, 2015.

Haider Hashim 15 | P a g e