JOURNAL OF LATEX CLASS FILES, VOL. XX, NO.

XX, XXX XXX 1

How Simulation Helps Autonomous Driving:

A Survey of Sim2real, Digital Twins,
and Parallel Intelligence
Xuemin Hu, Shen Li, Tingyu Huang, Bo Tang, Senior Member, IEEE, Rouxing Huai,
and Long Chen, Senior Member, IEEE
arXiv:2305.01263v2 [cs.RO] 30 Jun 2023

Abstract—Safety and cost are two important concerns for the Reality Simulation
development of autonomous driving technologies. From the aca- Sim2real
demic research to commercial applications of autonomous driving Knowledge
distillation
Transfer
learning
Curriculum
learning

vehicles, sufficient simulation and real world testing are required. Meta
learning
Robust
RL
Domain
randomization

In general, a large scale of testing in simulation environment is Physical Parameter & Expert Knowledge
& Reality Sample
conducted and then the learned driving knowledge is transferred Vehicle Pre-trained
Deployment Strategies
to the real world, so how to adapt driving knowledge learned
in simulation to reality becomes a critical issue. However, the
Digital Twins
virtual simulation world differs from the real world in many Physical space Prediction & Analytics Based Control Virtual space
aspects such as lighting, textures, vehicle dynamics, and agents’ Physical Entity Digital Twin
behaviors, etc., which makes it difficult to bridge the gap between AR/MR
Interaction
Historical
the virtual and real worlds. This gap is commonly referred to as Real-time
sensor
data
AI
the reality gap (RG). In recent years, researchers have explored measurement Env Real-Time Data Env ML model
various approaches to address the reality gap issue, which can be
broadly classified into three categories: transferring knowledge
Parallel Intelligence
from simulation to reality (sim2real), learning in digital twins Physical societies Artificial societies
(DTs), and learning by parallel intelligence (PI) technologies.
In this paper, we consider the solutions through the sim2real, Mixed Big Data
DTs, and PI technologies, and review important applications and
innovations in the field of autonomous driving. Meanwhile, we Real dynamic Simulation
show the state-of-the-arts from the views of algorithms, models, system
Descriptive Predictive Prescriptive
system
and simulators, and elaborate the development process from Intelligence Intelligence Intelligence

sim2real to DTs and PI. The presentation also illustrates the far-
reaching effects and challenges in the development of sim2real, Fig. 1. Leveraging computer simulations to improve real-world autonomous
DTs, and PI in autonomous driving. driving performance using three types of techniques: sim2real, digital twins
(DTs), and parallel intelligence (PI). Sim2real mainly focus on improvement
Index Terms—autonomous driving, sim2real, digital twins, at the algorithm level. DTs are more concerned with scenario modeling and
parallel intelligence, reality gap. interaction. As a higher level technology, PI which has three-level functions
including description, prediction, and prescriptive tends to construct mixed
datasets and parallel computation of both artificial and physical scenarios.
I. I NTRODUCTION
UTONOMOUS driving, as an important part of intel-
A ligent transportation in the future, has great potential
in alleviating traffic congestion and avoiding traffic accidents
The application of autonomous driving in industry demands
significant effort, especially for the planning and control
caused by human factors. In the past decades, researchers’ tasks [6]–[9]. It is necessary to consider not only the safety
efforts in this direction have increased, and many academic performance at the algorithm level, but also the cost at the
achievements have been made in autonomous driving [1]–[5]. real vehicle level, such as the price of high-precision sensors,
radars, and cameras, as well as the collision damage of real ve-
This work was supported in part by the National Natural Science Foundation hicles, etc. The consequences of directly applying an immature
of China under Grant 62273135 and in part by the Natural Science Foundation
of Hubei Province in China under Grant 2021CFB460. (corresponding author:
algorithm in a real vehicle are immeasurable. Therefore, an
Long Chen) experimental process is usually conducted in the high-fidelity
Xuemin Hu, Shen Li, and Tingyu Huang are with the School of Artifi- simulator at first, and then deployed in the reality environment,
cial Intelligence, Hubei University, Wuhan, Hubei, 430062, China (e-mail:
[email protected], lishen [email protected], [email protected])
which can greatly reduce the research and development costs.
Bo Tang is with the Department of Electrical and Computer Engineer- However, the simulation and the reality environment could be
ing, Worcester Polytechnic Institute, Worcester, MA, 01609, USA. (e-mail: completely different, and there are always gaps between the
[email protected])
Rouxing Huai is with Beijing Huairou Academy of Parallel Sensing,
simulated and real worlds such as lighting, textures, vehicle
Beijing, 101499, China. (e-mail: [email protected]) dynamics, and agents’ behaviors, etc., which is usually named
Long Chen is with the State Key Laboratory of Management and Control “reality gap (RG)” [10].
for Complex Systems, Institute of Automation, Chinese Academy of Sciences,
Beijing, 100864, China, and also with the Waytous Inc. Beijing 100083, In order to bridge the gap between the simulation and reality,
China. (e-mail: [email protected]) many different methods have been proposed [11]. At a high
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 2

level, these methods are mainly divided into three categories: i) researchers utilize parallel intelligence (PI) [13] to reduce RG.
transferring knowledge from simulation to reality, ii) learning As a more advanced technology, PI combines the advantages
in digital twins, and iii) learning by parallel intelligence of both sim2real and DT methods and can achieve better
technologies, which are shown in Fig. 1. management and control for complex systems. Unlike DT,
Sim2real refers to the process of transferring the strategies which only have a descriptive function, PI has three functions
and knowledge learned from the simulated world to the real including description intelligence, prediction intelligence, and
word to bridge the RG. In the field of autonomous driving, the prescriptive intelligence, which are shown in the bottom part of
core idea of sim2real is to train autonomous driving systems in Fig. 1. In parallel intelligence techniques, researchers typically
simulation environment and then to apply them in real-world construct an artificial system which is mapped to a physical
vehicles. In order to address the RG caused by the factors system to learn knowledge and gives feedback to the physical
such as uneven sampling in the real world, too many physical system. The artificial system learns knowledge from the data
parameters, insufficient expert experience, and imperfect dy- collected from the two systems through computational exper-
namics models, etc., sim2real has gradually been developed iments for evaluation, and then the knowledge is applied to
using six kinds of methods, including curriculum learning, the real system through parallel execution with high real-time
meta-learning, knowledge distillation, robust reinforcement virtual-real interaction and online feedback.
learning, domain randomization, and transfer learning. The In these three methods, what researchers need to do is
top part of Fig. 1 shows these methods and their relationships. to construct a virtual world. Therefore, some researchers
Each method has its unique way to deal with the RG problem. devote themselves to developing simulators of autonomous
For example, randomization of environmental parameters is driving and robotics, such as AirSim [14], CARLA [15], etc.
proposed in domain randomization so that the parameters in The mismatch between real and simulated settings can be
the simulated world can cover those in the real world, making minimized by providing training data and experience in these
the strategies and knowledge learning from the simulated simulators, and robot agents can be deployed to the real world
world applicable in the real world. However, the computational through sim2real methods.
cost of sim2real is still a challenge, especially when dealing This study surveys methods, applications, and development
with complex and dynamic environment. The computational of bridging the reality gap in autonomous driving. To the
requirements for simulated complex environments are difficult best of our knowledge, this is the first survey to focus on
to meet, which limits the scalability of sim2real methods. dealing the RG from the perspectives of sim2real, digital twins,
Unlike sim2real methods, digital twin (DT)-based methods and parallel intelligence. In conclusion, our contributions are
aim to construct a mapping of real-world physical entities in summarized as following three aspects.
a simulation environment using the data from sensors and • A taxonomy of the literature is presented from the
physical models to achieve the role of reflecting the entire perspectives of sim2real, digital twins, and parallel in-
lifecycle process of corresponding physical entities [12]. In telligence, where particularly DT and PI methods are
autonomous driving, DTs are typically used for multi-scale reviewed to address the reality gap issue for the first time.
modeling of the environment and vehicles. As shown in the • Methods and applications of handling the reality gap
middle part of Fig. 1, data from real sensors are used to achieve between simulation and reality in autonomous driving are
motion planning of the twin body and physical vehicle in the comprehensively reviewed in this paper.
virtual scenes through data interaction between the driving • Discussions of key challenges and opportunities are pre-
data analysis model and virtual reality. Some additional expert sented in this paper, offering insights of developing new
experience guides the modeling process to further narrow sim2real, DT, and PI methods in autonomous driving.
the RG. Moreover, the DT system is continuously learned
The remainder of this paper is organized as follows. In
and optimized through real-time data updates and interactions
the Section II, we introduce the methodologies and appli-
between the virtual and real twins to keep improving accuracy
cations of sim2real. Technologies and applications of digital
of the model. To achieve better virtual simulation effects,
twins as well as the guiding technologies, AR and MR, in
researchers combine virtual reality technologies with DTs.
autonomous driving are introduced in Section III. In Section
Argument reality (AR) and mixed reality (MR) technologies
IV, parallel intelligence technologies for autonomous driving
can provide better interactivity and visualization for digital
are presented. In Section V, we introduce the simulators that
twins. Users can interact and control the DT model using AR
are used to implement the above related technologies. The
and MR technologies to achieve better physical simulation,
future works and challenges are summarized in Section VI,
visualization, and manipulation. In addition, AR and MR tech-
and the conclusions are drawn in the final section.
nologies can enhance DTs through real-time object tracking
and virtualization, resulting in better interactive experiences
and more advanced interactive applications. II. S IMULATION TO REALITY TRANSFER
Sim2real-based methods play a crucial role in adapting Autonomous driving algorithms require extensive testing
autonomous driving vehicles to the complexities of the real before commercial applications. For safety and cost consider-
world by transferring learned knowledge from simulated en- ation, most of the current researches on new algorithms, espe-
vironments, while DT-based methods allow autonomous driv- cially for reinforcement learning-based methods, have focused
ing vehicles to learn knowledge by synchronizing data from on simulation environment [16], [17]. However, vehicle agents
both the real and simulated worlds. In recent years, some are trained to near-human levels in the simulation environment,
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 3

their performance in the real world is still not comparable to

that in the simulation environment. To overcome this issue,
researchers have turned to sim2real transfer technologies,
where the models are trained well in the simulation and then
transferred to the real world. In this section, we present related
approaches and technologies to achieve this goal, including
curriculum learning, meta-learning, knowledge distillation,
robust reinforcement learning, domain randomization, and
transfer learning. Tab.1 presents some typical sim2real models
which are reviewed in this section.

A. Curriculum learning
Curriculum learning (CL) [37] is a training strategy in
which the model accumulates knowledge by initially learning
simple tasks before involving more complex ones. It optimizes
the sequence of accumulating the experience for agents to
speed up the training process and improve effectiveness. The
task of sequencing the samples is tedious, so Kumar et al. Fig. 2. Curriculum self-paced learning approach for object detection in
propose the concept of Self-Paced Learning [38], where the autonomous driving [39].
curriculum is dynamically determined to adjust to the learning
pace of the learner. A relevant method for object detection in
B. Meta-learning
autonomous driving tasks is proposed by Soviany et al. [39],
which is shown in Fig. 2. The model trains a target detector The concept of meta-learning [43], which means learning
on the source images that are usually from the simulation to learn, has gained significant attention in recent years. The
environment, and then allows it to learn tasks from easy to essence of meta-learning is that the model is expected to gain
hard levels by a self-paced learning method. Finally, it reaches prior experience in other similar tasks and then be able to learn
the goal of predicting the images in the real domain in order new knowledge more quickly. Meta-learning has been shown
to bridge the RG. However, adaptive methods need some to have good advantages in several scenarios, such as single-
prior knowledge and require manual design [40]. In recent task learning [44], multitask learning [45], few-shot scenarios
years, the method of combining of curriculum learning and [46], Neural Architecture Search (NAS) [47], etc. Finn et
reinforcement learning has been widely developed [41]. For al. [48] propose a model-agnostic meta-learning algorithm
example, Florensa et al. [42] propose a reinforcement learning- which is compatible with all gradient descent, named as Model
based curriculum learning method, which does not require Agnostic Meta Learning (MAML). The model requires only
priority knowledge and use inverse training, allowing the agent a small amount of data to achieve fast convergence, and it
to directly start the curriculum from an applicable initial state. addresses the previous drawback of focusing only on the
Nonetheless, this static approach sometimes does not work initialization parameters in the moment.
well in complex autonomous driving tasks. One of the challenges in traditional reinforcement learning
To address complex driving tasks in dynamic scenarios, is the inefficiency of data exploration. In order to solve the
Qiao et al. [18] propose a dynamic idea named Automatically problem, Sæmundsson et al. [49] refer meta-learning as a la-
Generated Curriculum (AGC). They use deep reinforcement tent variable model and enable the knowledge to be transferred
learning to develop a strategy and use candidate sets to gener- across the robotic system and automatically infer relationships
ate the curriculum, which can optimize the traffic efficiency at between tasks from the data based on probabilistic ideas. In
complex intersections and reduce the training time. However, contrast, Nagabandi et al. [22] propose an online adaptive
this approach of pre-training on other tasks and transferring the learning approach for high-capacity dynamic models, where
knowledge is not very effective in different driving situations. they implemented the algorithm on a vehicle to solve the
Bae et al. [19] propose a curriculum strategy with self-taught sim2real problem by using model-based reinforcement learn-
scoring functions, which effectively estimated the roll and ing with online adaptive methods while training a dynamic
sideslip angles of the vehicle in different driving situations. prior model using meta-learning. Jaafra et al. [23] propose an
Meanwhile, Song et al. [20] propose a high-speed autonomous adaptive meta-learning method based on an embedded adaptive
overtaking system based on three-stages curriculum learning, meta reinforcement learning method with a neural network
using the same task-specific curriculum learning to train end- controller, which enables fast and efficient iterations for chang-
to-end neural networks, and experimental results proved that ing tasks and extends RL to urban autonomous driving tasks
the method has better overtaking performance. Anzalone et in the CARLA simulator. In addition, to reduce the high cost
al. [21] propose a reinforcement learning method with five of the labeled datasets required to train a model with high
stages for end-to-end autonomous driving, following the idea performance, Kar et al. [24] propose the concept of Meta-Sim,
of curriculum learning to continuously increase the learning as shown in Fig. 3, which is a learning model for generating
difficulty of the agents at each stage to learn complex behav- synthetic driving scenarios with the goal of acquiring images
iors. through a graphics engine and their corresponding realistic
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 4

TABLE I
TYPICAL SIM2REAL MODELS IN AUTONOMOUS DRIVING.

Article Category Dataset/Simulator Implemented Tasks Description Year

Use the automatic curriculum
Curriculum Driving planning
Qiao et al. [18] NGSIM generation method, which reduces 2018
learning decisions at crossroads
training time and provides better results.
A curriculum strategy with task-specific scoring
Curriculum Estimation of side camber
Bae et al. [19] CarMaker and pacing functions is proposed to 2021
learning and side slip angle
reduce the side-slip angle side-tilt angle.
Curriculum Gran Turismo A high-speed automatic overtaking system
Song et al. [20] Overtaking a car 2021
learning Sport based on three-stage curriculum learning.
An end-to-end autonomous driving
Curriculum End-to-end competitive
Anzalone et al. [21] MuJoCo reinforcement learning process 2022
learning driving strategy
with five stages.
Simulation of dynamical An online adaptive learning method
Meta-
Nagabandi et al. [22] MuJoCo models online for high-capacity dynamic models 2018
learning
adaption of real scenarios to solve the simulation to reality problem.
A meta reinforcement learning approach to
Meta-
Jaafra et al. [23] CARLA Autonomous driving strategy embedding adaptive neural network controller 2019
learning
s on top of adaptive meta-learning.
Meta-Sim environment, where images
Meta- Autonomous driving scene and their corresponding realistic
Kar et al. [24] KITTI 2019
learning generation and rendering ground images are acquired
through a graphics engine.
Learning teacher’s intermediate
Knowledge Autonomous driving representations through
Saputra et al. [25] KITTI 2019
distillation trajectory prediction attentional imitation loss and
attentional cue training methods.
Let the student in the model try
Knowledge Driving
Zhao et al. [26] CARLA to imitate the potential space of 2019
distillation decision making
the teacher in the base teacher model.
Knowledge Point cloud map A knowledge distillation method
Zhang et al. [27] KITTI 2022
distillation feature extraction based on point cloud map.
Knowledge 3D image generation for A self-supervised knowledge
Sautier et al. [28] KITTI 2022
distillation multimodal autonomous driving distillation method.
Transformer-based voxel feature
Knowledge Semantic segmentation of auto- encoder for robust
Li et al. [29] KITTI 2022
distillation nomous driving radar data LIDAR semantic segmentation
in autonomous driving.
Robust Proposing constrained adversarial
Decision making on
He et al. [30] reinforcement SUMO strategies for self-driving 2022
highway entrance ramps
learning high-speed entrance ramps.
Risk-adversarial learning approach,
Robust
Control strategies incorporating risk-averse and
Pan et al. [31] reinforcement TORCS 2019
for dynamical models risk-seeking mechanisms acting on
learning
games between agents.
A new method of domain randomiza-
Domain Semantic segmentation of tion and pyramid consistency is
Yue et al. [32] GTA 2019
randomization autonomous driving scenarios proposed to learn models with high
generalization ability.
More complex road and high-speed
traffic situations are considered,
Domain ADAS and the sim2real transformation is
Kontes et al. [33] CARLA 2020
randomization obstacle-avoidance accomplished by training several
variants of the complex problem
using domain randomization.
Static domain randomization uses real
Domain ADAS
Pouyanfar et al. [34] KITTI data to solve the end-to-end collision- 2019
randomization obstacle-avoidance
free depth drive problem.
A continuous end-to-end transfer
Transfer End-to-end autonomous driving
Kim et al. [35] Kinect learning approach which uses 2017
learning with lane semantic segmentation
two transfer learning steps.
Adding the idea of robust RL to
transfer learning, learning both
Transfer Autonomous driving
Akhauri et al. [36] deepDrive parameters and strategies, and transferring 2020
learning domain transfer
the correspondence between the two
to the execution environment.
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 5

models. To address this issue, Xu et al. [52] introduce the

conditional adversarial networks that enable the network to
automatically learn to transfer knowledge from the teacher to
student with smaller losses, thus make teacher-student models
to learn better from real data. However, most of the teacher-
student models are dependent on the teacher’s predictions
without considering ground truth error. In this case, Saputra
et al. [25] suggest transferring knowledge only when the
teacher is trusted, where the trust score is determined by the
Fig. 3. Synthetic datasets generation for driving scenarios based on the Meta-
teacher’s attrition. Fig. 4 shows the teacher-student structure
Sim method [24]. with a distillation rate of 92.95% for trajectory prediction
in autonomous driving. The teacher consists of a feature
extractor network and a pose regressor network. By using the
“dark knowledge” of the teacher, the student can transfer the
knowledge from simulation to reality. Similarly, Zhao et al.
[26] make the student in the model mimic the potential space
of the teacher, and potentially represent the side information
of the semantic segmentation in the teacher model, where the
student tries to get a representation of the teacher input related
to driving. Experimental results show that the model needs less
labeling to achieve better results.
Object detection and semantic segmentation in point clouds
obtained by LiDAR are important tasks for autonomous driv-
ing, and some researchers utilize the method of knowledge
distillation to handle these tasks. Zhang et al. [27] propose a
knowledge refinement method based on point cloud graphs.
The method extracts local geometric structures of the point
cloud using dynamic graph convolution and weighted learning
strategy to improve the efficiency of knowledge extraction,
which is applied in sim2real for autonomous driving. Never-
theless, annotating some point cloud data requires significant
cost. Sautier et al. [28] use a self-supervised knowledge distil-
lation approach without any point cloud and image annotation,
where the super pixels are used to extract features. To improve
the robustness, Li et al. [29] propose a robust LIDAR semantic
segmentation method with Transformer-based Voxel Feature
Encoder (TransVFE), where the encoder can learn features
Fig. 4. Network architecture of knowledge distillation for the student (right) better by modeling and preserving local relationships between
and teacher (left) in [25]. points.

images that bridge the distribution gap between the simulation D. Robust reinforcement learning
and reality. Meta-Sim also optimizes a meta objective by
Robust reinforcement learning (RL) was proposed as a new
automatically learning to synthesize labeled datasets related
paradigm for RL in [53] to represent the insensitivity of
to downstream tasks.
control systems to characteristic perturbations. The approach
is defined based on H ∞ control theory and explicitly considers
C. Knowledge distillation input disturbances and modeling errors to achieve robustness
The concept of knowledge distillation was first introduced in RL against uncertainties that affect actions and states in the
by Hinton et al. [50]. The main idea is that small student system, such as physical parameter changes.
models usually learn from and are supervised by large teacher In the process of reinforcement learning research, classical
models for knowledge transfer. Knowledge distillation, as an techniques for improving robustness are shown to only prevent
effective technique for deep neural model compression, has common situations and inconsistent for different simulation
been widely used in different areas of artificial intelligence environments, so robust adversarial reinforcement learning
[51], including speech recognition, visual recognition, and (RARL) [54] is proposed to improve the agents’ behaviors
natural language processing. by training the adversary to effectively prevent against system
The knowledge transfer from the teacher to student is a perturbations. As shown in Fig. 5, a constrained policy for
critical component of knowledge distillation, but it is also highway entrance ramps in autonomous driving is proposed
a challenge to entirely learn from the real data for teacher by He et al. [30]. They model the environment of a highway
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 6

Fig. 6. Framework for collision-free autonomous driving based on domain

randomization [34].

Fig. 5. Decision making framework based on robust RL for highway on-ramp authors train a generic model for different target domains.
merging in autonomous driving [30]. For some parameters of interest such as friction and mass,
the method proposed in [59] trains a generic control strategy
through a dynamic model with calibrated parameters, and then
intersection as an adversarial agent to constrain vehicle behav- analyzes the parameters using an online recognition system.
iors, and use adversarial agent and white-box adversarial attack Besides, Yue et al. [32] propose a new method of domain
techniques to simulate or generate adversarial environmental randomization and pyramid consistency to learn models with
disturbances and observational perturbations so as to bridge high generalization ability using consistency forced training
the RG. In addition, authors also propose an observation- of crossing-domain. The overall experiments show the effec-
based adversarial reinforcement learning that influences the tiveness of domain randomization is much better than existing
observations by incorporating a Bayesian optimization black- methods.
box attack method to make the agent efficiently approximate Kontes et al. [33] consider more complex road and high-
the optimal adversarial perturbation strategy [55]. Pan et al. speed driving situations by training several variants of the
[31] also propose a risk-adversarial learning approach, which complex problem, in which authors use domain randomization
incorporates a risk-averse mechanism by playing a game to complete the sim2real transfer and transfer the learned
between risk-averse and risk-seeking agents while modeling driving strategies to real vehicles. In addition, to perform
risk as a value function. Experimental results show that this randomized simulation of visual features in the source domain,
method triggers much fewer catastrophic events by agents than the method in [60] generates robust motion trajectories by
classical reinforcement learning methods. randomized dynamics models and acts on real robots to guide
Most of the existing literature refer RARL as a zero-sum them to perform learning multiple motor skills. Pouyanfar
simultaneous game with a Nash equilibrium, which may ignore et al. [34] propose a dynamic domain randomization, which
the sequential manner of RL deployment, produce overly differs from the static domain randomization and contains the
conservative agents, and lead to training instability. To handle randomization of dynamic objects. The proposed framework
this issue, Huang et al. [56] present a new sequential robust RL based on domain randomization for collision-free autonomous
formulation for both single-agent robot control and multi-agent driving is shown in Fig. 6. It generates a simulated world by
highway merging in the task, where they propose a general and simulating vehicle driving to collect images and corresponding
Stackelberg game model called RRL-Stack and developed the steering angles, and eventually predicts future steering angles
Stackelberg policy gradient algorithm to solve the RRL-Stack, in real images by scenario randomization, which solves the
which is better than the general RL method. end-to-end collision-free driving problem using the real data.

E. Domain randomization F. Transfer learning

Domain randomization is a method to solve the data aug- Transfer learning [61] is a popular approach that has been
mentation problem for sim2real transfer. The idea is to train developed to address the challenges associated with the high
the data in the virtual domain, obtain the data distribution cost and time needed to label data in machine learning
under various conditions in the virtual domain by sampling a applications, especially supervised learning. The main idea
set of random parameters, and then get the data results in the of transfer learning is to transfer labeled data or knowledge
real domain and adjust the parameters to learn the sampling between related domains in order to improve the learning
method that can be better generalized to the real domain in effect of the target domain or tasks, in which the model can
order to reduce the RG. transfer from the existing labeled data (source domain data)
In the fields of autonomous driving and robotics, it is to unlabeled data (target domain data). Transfer learning in
often necessary to deal with RGB images in a simulation autonomous driving is mainly used to implement the process
environment. Tobin et al. [57] directly transfer the strategy of transfer knowledge learned from a source domain to another
on simulated RGB images to the real world by domain target domain, such as transferring driving strategies learned
randomization without real images for the first time. From in a simulation environment to a real environment.
the view of a stochastic algorithm, the method in [58] shows The high cost of information acquisition for autonomous
how the source domain can be properly randomized, by which driving has led to the consideration of collecting data in a
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 7

Fig. 7. Transfer learning-based driving model including two stages with the
transferred weights technique [36].

simulation environment and transferring them to the real world

in a reasonable manner. Isele et al. [62] model intersection
simulations and transfer their learned information to the real
world, which demonstrates the robustness of the transfer
process. Kim et al. [35] propose a continuous end-to-end
transfer learning approach that uses two transfer learning steps
to transfer the environment and information to the nearby value Fig. 8. Comparison of digital twins with other simulation methods [66].
(a) Iterative convergence, switching between physical and digital space, (b)
domains of the vehicle to implement the learning process in a Model-based design, digital and incremental exchanges, and (c) Digital twin,
stepwise approach. visual synchronization of data.
To deal with the data acquisition problem, Tzeng et al. [63]
propose an improved adaptive approach that does not require
manual labeling of the data and effectively compensates for physical entity (physical twin), and both of them are intercon-
the domain bias generated in transfer learning by weakly nected by real-time data exchange [67], [68]. Conceptually,
pairing the source-target approach. Gradient strategies have the digital twin technology is an extension of augmented
proven to be very effective in solving complex behavioral reality and mixed reality. Meanwhile, parallel technologies
planning, and the multi-task policy gradient learning approach provides the theoretical basis for the development of DTs. Fig.
further improves the learning efficiency by using the transfer 8 illustrates the differences between the digital twin method
learning process among different tasks [64]. The same idea of and its previous methods in autonomous driving, such as
robust RL can also be added to transfer learning by learning the iterative method (a typical method for creating virtual
both parameters and policies in the process and transferring environment and tuning models) and the model-based design
the corresponding relation between them to the execution method [66]. For a more comprehensive decomposition of
environment [36], whose network architecture is shown in DTs, Tao et al. [69] propose the five-dimensional digital twins
Fig. 7. The experiment conducted in the scene of high-speed model, which is presented as Eq. 1.
intersections is divided into two stages. In the first stage, the
simulated data are trained, and the transfer learning weight is MDT = (P E, V M, Ss, DD, CN ) (1)
used to train the real data in the second stage. To address the
where PE, VM, Ss, DD, and CN denote the physical entity,
problem of the large network size that leads to poor program
virtual model, service, data, and connection, respectively.
performance, Sharma et al. [65] use a pre-trained segmentation
The AR, MR, PI and other technologies mentioned above
network, DeconvNet, where a variety of synthetic data and real
have a profound impact on the simulation work and are
data are used. The model achieves good results in simulating
inspirational for the applications of DTs in the autonomous
off-road driving environment.
driving field. Nowadays, there has been little work to summa-
rize DTs and VR-related technologies in autonomous driving.
III. D IGITAL TWINS TECHNOLOGIES AND APPLICATIONS Therefore, we review the methods and technologies from
AR/MR, and PI to DTs and their related applications in
While much progress has been made in sim2real in au- autonomous driving in this paper.
tonomous driving, transferring driving strategies from virtual
simulation to the real world remains a challenge. It is time
consuming and expensive to tune related algorithms to validate A. AR/MR-based methods and applications
design requirements in real physical environment with various Augmented reality is a technology that enhances the per-
scenarios. They are trying to leverage more comprehensive ception of the physical world by integrating virtual objects
simulations to reduce the number of places requiring physical into image sequences obtained from various cameras, allowing
tuning in industry. users to interact with these objects in real-time [70]. The
Virtual reality simulation has become a promising direction definition of AR has three characteristics including tracking,
in various fields. The main idea of DTs, as an important display, and interaction [71]. The three characteristics also
embodied part of VR, is a virtual copy or model of any define the technical requirements of the AR system, namely
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 8

that it needs a display that can display real and virtual

images, a tracking system that can find the position of users’
viewpoint and fix the virtual image in the real world, and
a computer system that can interact with the user in real-
time. Mixed reality, which includes both augmented reality and
augmented virtual, introduces realistic scene information in
the virtual environment through immersive ways, accompanied
by interactive feedback networks to enhance environmental
realism [72]. MR emphasizes the integration of real objects
into the virtual environment, so the simulation ability in the
framework is more crucial compared with AR. It creates a
Fig. 9. A windshield display interface prototype in a virtual environment for
space where virtual and real elements coexist, and both can an emergency scenario [74]. Left: the display showing the viewport space and
be allowed to interact with each other. As an off-line version dynamic elements. Upper right: the prototype running in a virtual environment.
of the digital twin system, MR can bridge the gap between Upper left: an example of requiring an emergency response.
the simulation and reality by enabling seamless interaction
between physical and virtual objects that exist in a physical A typical approach of building MR systems with high-
or virtual environment [73]. fidelity driving scenarios is to use high-precision simulators.
In recent years, AR and MR have been used and developed Tettamanti et al. [81] propose a vehicle traffic simulation sys-
in many fields such as assisted driving and autonomous driv- tem, which connects the SUMO simulator with real vehicles
ing. Lindemann et al. [74] design a windshield for AR effect online to simulate various scenarios and provides data for
display, which shows the perception and decision making vehicles in a virtual environment when testing autonomous
of the autonomous driving system. As shown in Fig. 9, driving methods. On top of that, Szalai et al. [76] propose
the method provides the driver with information detected by a mixed reality virtual environment to verify the feasibility
the self-driving vehicle in the environment, such as other of building virtual environment through game engines such as
vehicles, obstacles, road information, and the driving decisions SUMO and Unity3D, combined with a well-established system
for assisting driving. The selected diversity and real-time architecture. Mitchell et al. [82] propose an integrated system
interactive response ability of AR and MR in the virtual envi- for MR with multiple intelligent agents, in which they add
ronment provide an excellent testing and validation platform real vehicles to the environment and set up online learning
for autonomous driving, which could be used to reduce the strategies for the agents. The obstacle vehicles in the virtual
impact of RG. Su et al. [75] propose an optimized sensor environment are assigned simple tracing algorithms. This
array combined with the magnetic peg tracking technique in system provides a safe environment for testing autonomous
AR to track the position and direction of magnetic pegs in driving models in the real world. Through the 3D location
a magnetic field, which improves the positioning accuracy of information provided by MR devices such as Hololens, Moezzi
an automated guided vehicle (AGV). In [76], the MR system et al. [83] solve the simultaneous localization and map building
uses sensors to capture vehicle state data and build a virtual problems and achieve path planning for intelligent vehicles.
space in order to provide a large number of optional complex MR can also be used as a system environment for DTs
environments for testing autonomous driving algorithms. Marc to digitally model unstudied objects and generate digital twin
et al. [77] propose a classical MR framework for autonomous entities in a mixed reality environment. Tu et al. [84] propose
driving based on LiDAR. The real sensors and sensor models a method to enhance the mobility of the industrial crane digital
both perceive their respective environments, and then the data twin through spatial registration and tracking in an MR system.
perceived by each are fused into a mixed reality module to To enhance the real-time performance and accuracy of the
test the autonomous driving system that reacts to the real and system distance measurement, Choi et al. [85] propose an MR
virtual worlds. integration system combining deep learning and DTs, where
The video perspective technologies can be extended to assist the deep learning-based instance segmentation is applied in
autonomous driving to enhance drivers’ state perceptions, like RGB and 3D point cloud data to improve registrations between
highlighting pedestrians and vehicle density [78]. Through the real robot and its digital twin. Lalik et al. [86] build a
human-machine interface and internet of vehicles, Wang et digital twin system integrated with existing control devices
al. [79] propose an augmented reality-based assisted driving through human-robot interaction to improve the accuracy of
system for multi-vehicle interconnection. This method obtains robot motion, which is similar to the hardware-in-the-loop
the road, vehicle, and pedestrian information from multiple technology.
connected vehicles, collaboratively calculates and simultane-
ously displays this information through AR to assist drivers in
driving. At the same time, AR makes it possible to visualize B. Digital twins in autonomous driving
and interact with the data in DTs. Williams et al. [80] use The concept of digital twins has gained significant atten-
an AR-assisted DT solution, where digital data from the DT tion in Industry 4.0 due to its competitive and economic
model are provided as a visual presentation through an AR advantages. In the field of autonomous driving, digital twin
device, and the DTs are used as a database to learn to generate technologies provide a safe and diverse testing environment
motion commands for the vehicles in the virtual environment. with reducing its time cost by at least 80%. At the same
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 9

TABLE II
AUTONOMOUS DRIVING APPLICATIONS STATISTICS FOR DIGITAL TWINS.

Article Methodologies Twin objects Applications Year

Laschinsky et al. [89] MBD Vehicles ADAS design and test 2010

Shikata et al. [90] MBD Vehicles EV charging design and test 2019

ADAS test,
Szalai et al. [76] MBD, VPG Vehicles, traffic 2020
mixed-reality application

Dygalo et al. [91] MBD Vehicles ADAS recognition 2020

Model-based RL
Wu et al. [92] MBD Environment 2021
in autonomous driving

Yu et al. [87] MBD, CPS Vehicles, environment ADAS design and test, V2X 2022

Schwarz et al. [93] CPS Vehicles, environment ADAS design and test 2010

Eleonora et al. [94] CPS Vehicles AGV logistics action test 2017

Chen et al. [95] CPS Drivers Drivers’ safety behaviors analysis 2018

Rassõlkin et al. [96] CPS, MBD Vehicles ADAS test 2019

Ge et al. [97] CPS Vehicles ADAS Design and test, V2X 2019

Veledar et al. [98] CPS Vehicles ADAS design and test 2019

Multi-sensor fusion for

Liu et al. [99] CPS Vehicles 2020
vehicle recognition, ADAS

Liu et al. [100] CPS Vehicles ADAS design and test 2021

Culley et al. [101] VPG, MBD Vehicles, environment ADAS design and test 2020

Fremont et al. [102] VPG Environment, vehicles Formal test based on scenario 2020

Reinforcement learning
Voogd et al. [103] Transfer learning Vehicles 2022
autonomous driving

MBD: model-based design, CPS: cyber-physical systems, VPG: virtual proving ground.

time, it solves two problems of traditional simulators [87], control. Researchers typically design task-specific digital twin
[88]: 1) simulation testing is not equivalent to real end-to-end systems based on the above points. The digital twin system
testing, and 2) factors such as weather, climate, and lighting for automated guided vehicles enables the autonomous trans-
are not fully covered. The digital twin paradigm is able to portation of driven robots among three predefined tracks of
generate high-fidelity complex virtual environments and sensor processing stations through digital twin modeling, where the
modeling that approximates real data to realize a complete, digital twin is only used as a dynamic simulation of discrete
comprehensive, accurate, and reliable digital twin entity. Tab.2 events without involving complex dynamic driving control
provides the review of digital twins for autonomous driving techniques [94]. Rassõlkin et al. [96] propose a digital twin
applications. system for autonomous electric vehicles. They use matlab
In the field of autonomous driving, a digital twin system to model the sensors and vehicle models of the ISEAUTO
is usually constructed with three parts [87]: 1) the digital vehicle and then combine data from the real devices and virtual
twin of the sensor model, 2) the digital twin of 3D digital sensors with a machine learning program.
maps, and 3) the logical twin corresponding to traffic flow In recent years, many different frameworks have been pro-
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 10

This method greatly enhances the traffic flow of autonomous

driving and improves the effectiveness of the algorithm. A
framework for the application of vehicle edge computing in
the DT is proposed to model the physical vehicle environment
through edge heterogeneous information obtained by vehicles
in the DT, combining with communication techniques of
vehicle-to-infrastructure (V2I) [108].
The high-fidelity simulator LGSVL [109] for autonomous
driving provides an end-to-end, full-stack simulation, through
which users can create required digital twin entities, including
customizable vehicle sensors and core modules, to achieve
the requirements of autonomous driving algorithms. Liu et al.
[99] propose a digital twin system based on the Unity game
engine, which depicts the road traffic situation by integrating
physical target detector data and digital twin information. They
demonstrate and evaluate the approach in the Unity engine,
and the results show that their method combined with the
digital twin system can significantly improve driving safety.
In reinforcement learning, DT technologies are used to
improve the ability of transferring learned strategies and
bridge the gap between different domains by the systematic
and simultaneous combination of data from traffic scenarios
in both virtual and real worlds. Voogd et al. [103] apply
domain randomization and adaptive techniques at each stage
Fig. 10. Block diagram of data flow based on the digital twin model. The of the development cycle of digital twins to make the transfer
green, blue, and yellow lines indicate the physical entity data stream, the DT learning process more robust. The multi-modal fusion of driver
stream, and the simulation evaluation data stream, respectively [104]. digital twins (DDTs) is proposed in [110], which emphasizes
the ability of the driver’s intrinsic personality and extrinsic
physiological level to bridge the gap between the real au-
posed for the digital twin in autonomous driving. These digital tonomous driving and fully digital systems.
twin frameworks are generally divided into physical, data, and
virtual parts, and the users can add additional modules in order IV. PARALLEL INTELLIGENCE TECHNOLOGIES
to accomplish different tasks. Xiong et al. [104] present a
Sim2real-based methods transfer the learned knowledge
physical entity simulation framework, as shown in Fig. 10,
from the simulated to real worlds, and DT-based methods
which consists of three parts: the physical entity, the digital
allow autonomous driving vehicles to learn knowledge by
twin component, and the simulation evaluation component.
synchronizing data from both the real and simulated worlds
The physical entity part is responsible for receiving real-world
in digital twins. Both of them can handle the RG problem
data and executing relevant operations. The simulation evalua-
to some extent, but they still face some challenges such as
tion component is responsible for processing the received real
the difficulty in modeling complex systems. The RG is very
and virtual data, and the digital twin component performs the
large in some complex scenarios, to which the dynamic control
simulation verification of the DT vehicles. The Society for
simulation in DT and sim2real methods cannot adapt. In this
Automotive Engineering (SEA) also proposes a framework
case, it is an effective way to solve the RG problem to
consisting of four modules: data, integration, analysis, and
construct an artificial system, realize the dynamic interaction
digital twin. Each module corresponds to a different task and
of computation, physics and society, and deal with uncertainty
meets the requirements of digital twin components, which di-
through parallel computation of artificial and real systems.
rectly interact with the physical system to achieve autonomous
driving tasks [105].
Vehicle to everything (V2X) based on vehicle networking A. Parallel intelligence concept and ACP method
is also playing an important role in the study of the digital The parallel intelligence concept, which was first proposed
twins for autonomous driving. China Academy of Information by Wang [111], provides an effective idea to connect the
and Communication (CAICT) develops a digital twin system physic world and artificial world and achieves a great success
that uses the V2X communication technologies to achieve in many fields in recent years. This concept is based on the
high-fidelity transmission and simulation, which is used in an cyber-physical-social systems (CPSS) that is the evolution of
autonomous driving testing system [97]. Niaz et al. [106] then cyber-physical-systems (CPS). CPSS enables virtual and real
confirm that the V2X-based DT system supports low latency systems to interact, feedback, and promote each other. The
connectivity for autonomous driving tests. Wang and Tu [107] real system provides valuable datasets for the construction and
propose a method that can automate traffic modeling to auto- calibration of the artificial system, while the artificial system
matically capture and reconstruct the traffic in special areas. guides and supports the operation of the real system, thus
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 11

B. Parallel learning and parallel vision

After proposing the concept of parallel intelligence, re-

searchers attempt to extend parallel intelligence into the ma-
chine learning field and construct a novel theoretical frame-
work called parallel learning [122]. The framework of parallel
learning is illustrated in Fig. 11, which can be divided into
three interrelated stages: descriptive learning, predictive learn-
ing, and prescriptive learning. Descriptive learning involves the
extraction of knowledge and experience from extensive artifi-
cial data within an artificial system. Predictive learning aims to
establish mappings from observed information to unknown in-
formation. Prescriptive learning further optimizes the learning
strategy by leveraging these two processes while providing
Fig. 11. Parallel learning framework [122]. guidance to enhance the system’s performance. Specifically,
this process addresses the issue of inadequate recorded data
resulting from manual operations by employing predictive
learning with a substantial number of generated artificial data.
achieving self-evolution. Alam and Saddik [112] propose a
Descriptive learning is subsequently employed to enhance the
cloud-based cyber-physical system (C2PS) intelligent interac-
acquired knowledge and experience. The framework provides
tion controller, which incorporates cloud technologies in a
a new approach of handling the problems of reality gap and
cyber-physical system. The architecture uses Bayesian net-
data imbalance from the real and simulated scenarios for
works to systematically and dynamically consider the context
machine learning-based autonomous driving methods.
for prototype telematics-based driver assistance applications in
the vehicle Cyber-Physical Systems (VCPS). Bhatti et al. [113] Computer vision is one of the most significant technologies
propose a framework for an IoT-based digital twin for electric in machine learning and autonomous driving. Based on the
vehicles, where they explain the detailed roles of IoT and the ACP theory, Wang et al. [130] propose the concept of parallel
DT technologies in autonomous navigation control, advanced vision to address the challenge of extracting features from
driver assistance systems and vehicle health detection for complex environments in traditional computer vision methods.
electric vehicles. As shown in Fig. 12, the approach entails constructing artificial
To implement the parallel intelligence concept, the ACP scenes that resemble real driving scenarios and automatically
method, an organic combination of artificial societies (A), obtaining accurate labeling information while generating large-
computational experiments (C) and parallel execution (P), scale and diverse datasets. Subsequently, computational ex-
is proposed by Wang. et al. [114], which has made great periments are conducted in these artificial scenes, including
achievement in many fields. The ACP method consists of two stages: “learning and training” and “experimentation and
three main steps: 1) modeling complex systems using artificial evaluation”. In the learning and training stage, the model is
systems, 2) using computational experiments to train and eval- trained using the generated large-scale and diverse datasets
uate complex systems, and 3) setting the interaction between from both artificial and real worlds. In the experimentation
the real physical system and the virtual artificial system so and evaluation stage, the model performance is assessed in the
as to realize effective parallel control and management of two environments using the datasets. Computer vision models
the complex system. Unlike DT, which is still in the partial run in parallel in real and artificial driving scenarios, constantly
computational experiment stage of the ACP method and only fusing both of the two scenarios to improve the performance of
establishes a descriptive artificial system without prediction knowledge transfer in complex environment to better reduce
and guidance functions, providing real-time monitoring and RG. On this basis, parallel vision paves the way for vision
adjustment services for specific systems, PI provides a com- technologies in autonomous driving.
plete modeling and description of the ACP method with three
functions of description intelligence, prediction intelligence,
and prescriptive intelligence, so it can achieve better manage-
ment and control of complex systems.
In recent years, researchers applied the ACP and parallel
intelligence methods in handling knowledge transferring from
the simulation to reality worlds. The parallel intelligence
concept has also evolved in several technologies, such as
parallel learning [115], parallel vision [116], parallel control
[117], [118], parallel testing [119], parallel driving [120],
[121], and parallel planning [115], etc. The survey of these
methods is shown in Table III. Fig. 12. Parallel vision framework [122].
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 12

TABLE III
COMPARISON OF SELECTED VIRTUAL-TO-REAL WORKS ON CONTROL OF AUTONOMOUS DRIVING WITH REAL DATA.

Article Virtual Methodologies Highlights Year

A parallel learning framework for
Li et al. [122] Software Defined Artificial Systems parallel learing 2017
extracting data and integrating learning

Parallel vision for long-term

Zhang et al. [123] Pedestrians datasets parallel vision 2020
learning in an ever-changing environment

Background subtraction algorithm

Zheng et al. [124] Changedetection dataset parallel vision 2020
based on parallel vision and GANs

Cooperative control of
Wang et al. [125] iHorizon parallel driving 2017
multi-vehicle autonomous driving

Hybrid virtual-reality data for autonomous

Chen et al. [126] PED(end-to-end driving data set) parallel driving 2019
driving performance improvement

Parallel driving with safety

Liu et al. [127] PanoSim parallel driving 2020
and smart connectivity

Parallel drive system

Chen et al. [128] PDOS(ubiquitous operating system) parallel driving 2022
for autonomous driving tasks

Closed-loop parallel test system

Li et al. [119] Customizing environment parallel testing 2019
for more challenging tests

End-to-end parallel planning

Chen et al. [115] GTA5/ETS2 parallel planning 2019
for emergency scenarios

Parallel planning system

Liu et al. [129] Customizing the UAV environment parallel planning 2022
for UAV clusters

Parallel traffic
Wang et al. [117] ATS parallel control 2010
management framework

C. Parallel driving and parallel testing methods based on parallel driving can learn drivers’ behav-
ior in response to different reality and simulation scenarios
The development of autonomous driving poses significant through the integrated multi-ADAV modules so as to bridge
challenges to current vehicle and transportation systems. Wang the RG.
et al. [125] propose three elements required for future con- In the research of parallel driving, Wang et al. [125] propose
nected autonomous driving: physical vehicles, human drivers, the initial concept of parallel testing, in which the cyclic
and cognitive attributes. Based on the ACP theory, the authors updating method is used to address the RG problem and
develop projecting the three elements to three parallel worlds verify the performance of autonomous driving. As shown in
and creating a parallel driving framework. As shown in Fig. Fig. 14, the cyclic updating method of co-evolution between
13, the three parallel worlds are interconnected. Each vehicle the real testing ground and parallel virtual testing ground
in the real world is assigned controlling functions as well as enhances the interaction of virtual reality and authenticity of
an ADAV module which is responsible for communication the scenarios, so the test in virtual testing ground can show the
with the artificial world and other vehicles. This module also performance of autonomous driving. Following this research,
provides driving information for human drivers. Facilitated by Li et al. [119] complete the framework of parallel testing
the ADAV module, parallel driving enables joint responses to which emphasized the real-time data interaction between real
complex autonomous driving scenarios, with human drivers and virtual scenarios and the diversity of testing scenarios.
conditionally participating to ensure system safety. Further-
more, Liu et al. [120] integrate “digital quadruplets” including
the physical vehicle, the descriptive vehicle, the predictive D. Parallel planning and parallel control
vehicle, and the prescriptive vehicle in parallel driving. Based Planning is one of the important parts for autonomous
on the description of digital quadruplets, three virtual vehicles, driving systems. To address the problem of emergency traffic
which are defined as three “guardian angels” for the physical scenarios in the real world for self-driving vehicles, Chen et
vehicle, play different roles to make intelligent vehicles safer al. [131] propose the method of parallel planning. As shown
and more reliable in complex scenarios. Autonomous driving in Fig. 15, the method involves modeling emergency traffic
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 13

theory. In recent years, parallel control is used in transportation

systems to construct a parallel transportation management
system and verify its effectiveness in complex transportation
systems [117]. Based on the synthetical data and real-world
observations, parallel control enhances control effectiveness
and improves the development of autonomous driving systems.

V. AUTONOMOUS DRIVING SIMULATORS

It requires significant costs to directly test autonomous
driving algorithms in real world scenarios, and the data of
rare events in the real world are difficult to collect. Some
researchers devote themselves to construct parallel datasets
[133], but the collected off-line datasets cannot meet the
requirements of real-time interactive testing. In this case,
low-cost testing of autonomous driving systems, simulation
Fig. 13. Framework of parallel driving. (RD: real driving; DC: driver systems are often built to simulate testing processes. Au-
cognition; CPSS services including three components: people (social web),
place/location (geo web), and technology (sensors, Internet of Things, etc); tonomous driving simulators need to be able to build realistic
HD: human driver; RV: real vehicle; ADAV: artificial driver and artificial road environment as well as vehicle models, sensors, such as
vehicle; and SA: situation awareness) [125]. cameras, and LiDAR.
Simulators are usually developed based on game engines
with certain rendering capabilities, and most are static sim-
scenarios from the artificial traffic world, in which emergency ulators such as Unreal, and Unity, which have limited con-
traffic scenarios are generated for learning autonomous driving trol over characters and weather. However, simulators that
algorithms. The gained knowledge can enable correctly and focus on the task of driving vehicles usually need to build
timely planning in real-world scenarios when emergent events dynamic scenes. For the standard of scene simulation, the open
occur. Parallel planning integrates parallel perception, deep series developed by the Association for Standardization of
learning, and end-to-end autonomous driving into a frame- Automation and Measuring Systems(ASAM) is developing as
work, and incorporates knowledge guidance of emergency a guide for autonomous driving scenarios. A typical software
avoidance from expert strategies in the planning process, based on OpenScenario is RoadRunner Scenario, which is an
which combines with synthesized data from both virtual and interactive dynamic scenario simulator. Besides, CARLA [15]
real worlds. Based on this mode, it can effectively apply the and LGSVL [109], etc. can also be used to build driving
knowledge learned in the artificial world, especially for the scenarios based on OpenScenario for autonomous driving
driving strategies for emergency scenarios, in the real world tasks. In addition, some simulation environments improved
and enhance the applicability of artificial driving behaviors to based on robot simulators like Gazebo [134] can achieve more
real-world situations. details at the unit level such as vehicle models, pedestrian
Parallel control is a knowledge-based automated learning modeling, and higher accuracy for sensors, but not realistic
way that aims to achieve automation of intelligent systems enough for 3D scene implementation. Currently, some high-
and simplification of complex systems. Wang et al. [132] fidelity simulators have been developed as virtual environment
first propose parallel control framework based on the ACP for digital twins in autonomous driving, such as AirSim [14],
CarSim [141], and SUMO [142]. Tab.3 provides the review of
the simulators.

VI. P ROBLEMS AND CHALLENGES

Although the methods of sim2real, DTs and PI have made
great achievements in the field of autonomous driving, it
still needs to further improve existing limitations and explore
unknown technologies in the process of developing a reliable,
Fig. 14. Realization of parallel testing [125].
safe and comfortable autonomous driving method in the real
world. It is necessary for researches to focus the work on
sim2real, DT, and PI technologies and consider how to bridge
the RG, in order to promote the development of extensible
and safe autonomous driving. At present, there are four main
challenges in this field.
(1) Current methods and technologies are designed for spe-
cific scenarios, but their performance applied in other different
tasks is not satisfactory. There is a lack of a general task-
Fig. 15. Framework of parallel motion planning. [131]. independent method to ensure the effectiveness and portability
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 14

TABLE IV
SIMULATOR AND RELATED DESCRIPTIONS FOR AUTONOMOUS DRIVING.

Platform Descriptions Advantages

AirSim [14] High fidelity platform developed by unreal engine High-fidelity environment and car models

Gazebo [134] Open source robot simulator Strong physical modeling capabilities

CARLA [15] Unreal engine open source autopilot simulator High-fidelity sensor data

High-precision maps and interfaces to support

LGSVL [109] Autonomous robot car simulator for unity game engine
multiple autonomous driving algorithm engines

Torcs [135] Vehicle driving games High portability and modifiability

Lightweight, allowing intensive learning

MetaDrive [136] Open source driving simulation platform
training using a large number of random maps

SUMO [137] Open source traffic simulation platform Generate complex traffic systems

Generate complex traffic systems

SUMMIT [138] CARLA-based traffic environment extension
and can import the SUMO mapping system

AutoWare [139] Open-source autonomous driving platform Mature development framework

Apollo [140] Large-scale datasets consisting of video and 3d point clouds Data for high precision attitude information

of the sim2real process. In this case, we need to consider a to improve the generalization of the transfer process.
way to quantify and represent the reality gap in the current (5) Parallel intelligence technologies improve autonomous
autonomous driving field and compare different environments driving systems’ performance, handle big data, and solve
and applications, which is conducive to further research. the RG problem through multi-unit parallel computing and
Moreover, quantifying RG is deriving relevant algorithms to execution. With the expansion of the computing scale and
reduce RG and achieve the purpose of leaping from the virtual increasement of functional requirements in the autonomous
to real world. driving field, parallel intelligence technologies need to meet
(2) As a solution for training generalized reinforcement the requirements of ever-increasing computing scale and com-
learning models, sim2real needs a lot of data to support its plexity.
training process. We expect to use more real data, but real (6) Parallel intelligence can help autonomous driving sys-
data are difficult to obtain compared with simulated data. Real tems learn knowledge from the data generated by the physical
data are often difficult and challenging to collect, classify, and and artificial systems, but the generated data exhibit charac-
generalize to a standard distribution. teristics of multi-modality, high massiveness and redundancy.
Therefore, how to perform efficient data analysis to refrain
(3) The digital twin is nowadays mainly used as an applied
from useless data and make a precise representation of the
technology in the fields of robotics and autonomous driving
physical system are problems for PI-based autonomous driving
to achieve relevant tasks directly by building digital twin
systems.
entities or environments. However, the specific methodology
and interpretation of related characteristics of digital twins are
scarce in relevant literature. It is still a difficult task to achieve VII. C ONCLUSION
a comprehensive digital twin. With the help of meta-learning, Autonomous driving researches require a significant amount
the digital twin can obtain prior learning experience through of real-world data to train a reliable and robust algorithm. On
the previous twin in the processes of environment interaction one hand, using real data in extreme and scarce scenarios often
or simulated object synchronization, leading to the iterative leads to a high cost, so simulated data are typically used to
optimization process. meet the requirement. On the other hand, there is always a
(4) The models and algorithms of DT technologies are sep- gap between the simulated and real worlds, so it is necessary
arated, and there is no comprehensive method to combine and to further investigate methods for transferring knowledge from
evaluate them. For the model, environmental parameters, and simulation to reality. In this paper, we comprehensively review
some random noise in DTs, there is a lack of a synchronous state-of-the-arts of sim2real, DTs, and PI. In sim2real, we
sim2real method that can be considered as a dynamic process mainly introduce the methodologies and related applications
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 15

from the view of different categories, while we focus on the [15] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, V. Koltun, Carla: An
virtual technologies about the DT dependency, and then intro- open urban driving simulator, in: Conference on robot learning, PMLR,
2017, pp. 1–16.
duce the common frameworks and applications of autonomous [16] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa,
driving. Subsequently, the parallel intelligence thesis and D. Silver, D. Wierstra, Continuous control with deep reinforcement
technologies including parallel learning, parallel vision, and learning, arXiv preprint arXiv:1509.02971 (2015).
[17] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley,
parallel driving, etc. in autonomous driving are reviewed. To D. Silver, K. Kavukcuoglu, Asynchronous methods for deep rein-
demonstrate the simulation environment for realizing sim2real, forcement learning, in: International conference on machine learning,
DT and PI methods, we also summarize existing autonomous PMLR, 2016, pp. 1928–1937.
[18] Z. Qiao, K. Muelling, J. M. Dolan, P. Palanisamy, P. Mudalige,
driving simulators. In addition, we present existing challenges Automatically generated curriculum based reinforcement learning for
for the future development of autonomous driving in sim2real, autonomous vehicles in urban environment, in: 2018 IEEE Intelligent
DTs and PI in this paper. Vehicles Symposium (IV), IEEE, 2018, pp. 1233–1238.
[19] J. Bae, T. Kim, W. Lee, I. Shim, Curriculum learning for vehicle lateral
stability estimations, IEEE Access 9 (2021) 89249–89262.
R EFERENCES [20] Y. Song, H. Lin, E. Kaufmann, P. Dürr, D. Scaramuzza, Autonomous
overtaking in gran turismo sport using curriculum reinforcement learn-
[1] X. Hu, L. Chen, B. Tang, D. Cao, H. He, Dynamic path planning ing, in: 2021 IEEE international conference on robotics and automation
for autonomous driving on various roads with avoidance of static (ICRA), IEEE, 2021, pp. 9403–9409.
and moving obstacles, Mechanical systems and signal processing 100 [21] L. Anzalone, P. Barra, S. Barra, A. Castiglione, M. Nappi, An end-
(2018) 482–500. to-end curriculum learning approach for autonomous driving scenar-
[2] W. Zhang, A robust lateral tracking control strategy for autonomous ios, IEEE Transactions on Intelligent Transportation Systems 23 (10)
driving vehicles, Mechanical Systems and Signal Processing 150 (2022) 19817–19826.
(2021) 1–15. [22] A. Nagabandi, I. Clavera, S. Liu, R. S. Fearing, P. Abbeel, S. Levine,
[3] L. Chen, Y. Li, C. Huang, B. Li, Y. Xing, D. Tian, L. Li, Z. Hu, C. Finn, Learning to adapt in dynamic, real-world environments
X. Na, Z. Li, S. Teng, C. Lv, J. Wang, D. Cao, N. Zheng, F.-Y. Wang, through meta-reinforcement learning, arXiv preprint arXiv:1803.11347
Milestones in autonomous driving and intelligent vehicles: Survey of (2018).
surveys, IEEE Transactions on Intelligent Vehicles 8 (2) (2023) 1046– [23] Y. Jaafra, A. Deruyver, J. L. Laurent, M. S. Naceur, Context-aware
1056. doi:10.1109/TIV.2022.3223131. autonomous driving using meta-reinforcement learning, in: 2019 18th
[4] L. Chen, Y. Li, C. Huang, Y. Xing, D. Tian, L. Li, Z. Hu, S. Teng, C. Lv, IEEE International Conference On Machine Learning And Applications
J. Wang, D. Cao, N. Zheng, F.-Y. Wang, Milestones in autonomous (ICMLA), IEEE, 2019, pp. 450–455.
driving and intelligent vehicles—part 1: Control, computing system [24] A. Kar, A. Prakash, M.-Y. Liu, E. Cameracci, J. Yuan, M. Rusiniak,
design, communication, hd map, testing, and human behaviors, IEEE D. Acuna, A. Torralba, S. Fidler, Meta-sim: Learning to generate
Transactions on Systems, Man, and Cybernetics: Systems (2023) 1– synthetic datasets, in: Proceedings of the IEEE/CVF International
17doi:10.1109/TSMC.2023.3276218. Conference on Computer Vision, 2019, pp. 4551–4560.
[5] L. Chen, S. Teng, B. Li, X. Na, Y. Li, Z. Li, J. Wang, D. Cao, N. Zheng, [25] M. R. U. Saputra, P. P. De Gusmao, Y. Almalioglu, A. Markham,
F.-Y. Wang, Milestones in autonomous driving and intelligent vehi- N. Trigoni, Distilling knowledge from a deep pose regressor network,
cles—part ii: Perception and planning, IEEE Transactions on Systems, in: Proceedings of the IEEE/CVF international conference on computer
Man, and Cybernetics: Systems (2023) 1–15doi:10.1109/TSMC. vision, 2019, pp. 263–272.
2023.3283021. [26] A. Zhao, T. He, Y. Liang, H. Huang, G. Van den Broeck, S. Soatto,
[6] X. Hu, B. Tang, L. Chen, S. Song, X. Tong, Learning a deep Sam: Squeeze-and-mimic networks for conditional visual driving pol-
cascaded neural network for multiple motion commands prediction in icy learning, in: Conference on Robot Learning, PMLR, 2021, pp. 156–
autonomous driving, IEEE Transactions on Intelligent Transportation 175.
Systems 22 (12) (2022) 7585–7596. [27] L. Zhang, R. Dong, H.-S. Tai, K. Ma, Pointdistiller: structured knowl-
[7] Y. Liu, X. Ji, K. Yang, X. He, X. Na, Y. Liu, Finite-time optimized edge distillation towards efficient and compact 3d detection, arXiv
robust control with adaptive state estimation algorithm for autonomous preprint arXiv:2205.11098 (2022).
heavy vehicle, Mechanical Systems and Signal Processing 139 (2020) [28] C. Sautier, G. Puy, S. Gidaris, A. Boulch, A. Bursuc, R. Marlet, Image-
1–22. to-lidar self-supervised distillation for autonomous driving data, in:
[8] S. Teng, L. Chen, Y. Ai, Y. Zhou, Z. Xuanyuan, X. Hu, Hierarchical Proceedings of the IEEE/CVF Conference on Computer Vision and
interpretable imitation learning for end-to-end autonomous driving, Pattern Recognition, 2022, pp. 9891–9901.
IEEE Transactions on Intelligent Vehicles 8 (1) (2023) 673–683. [29] J. Li, H. Dai, Y. Ding, Self-distillation for robust lidar semantic seg-
doi:10.1109/TIV.2022.3225340. mentation in autonomous driving, in: Computer Vision–ECCV 2022:
[9] S. Teng, X. Hu, P. Deng, B. Li, Y. Li, Y. Ai, D. Yang, L. Li, Z. Xu- 17th European Conference, Tel Aviv, Israel, October 23–27, 2022,
anyuan, F. Zhu, L. Chen, Motion planning for autonomous driving: The Proceedings, Part XXVIII, Springer, 2022, pp. 659–676.
state of the art and future perspectives, IEEE Transactions on Intelligent [30] X. He, B. Lou, H. Yang, C. Lv, Robust decision making for autonomous
Vehicles (2023) 1–21doi:10.1109/TIV.2023.3274536. vehicles at highway on-ramps: A constrained adversarial reinforcement
[10] A. Kadian, J. Truong, A. Gokaslan, A. Clegg, E. Wijmans, S. Lee, learning approach, IEEE Transactions on Intelligent Transportation
M. Savva, S. Chernova, D. Batra, Sim2real predictivity: Does evalua- Systems (2022).
tion in simulation predict real-world performance?, IEEE Robotics and [31] X. Pan, D. Seita, Y. Gao, J. Canny, Risk averse robust adversarial
Automation Letters 5 (4) (2020) 6670–6677. reinforcement learning, in: 2019 International Conference on Robotics
[11] K. Kang, S. Belkhale, G. Kahn, P. Abbeel, S. Levine, Generalization and Automation (ICRA), IEEE, 2019, pp. 8522–8528.
through simulation: Integrating simulated and real data into deep [32] X. Yue, Y. Zhang, S. Zhao, A. Sangiovanni-Vincentelli, K. Keutzer,
reinforcement learning for vision-based autonomous flight, in: 2019 B. Gong, Domain randomization and pyramid consistency: Simulation-
international conference on robotics and automation (ICRA), IEEE, to-real generalization without accessing target domain data, in: Pro-
2019, pp. 6008–6014. ceedings of the IEEE/CVF International Conference on Computer
[12] L. Edington, N. Dervilis, A. B. Abdessalem, D. Wagg, A time- Vision, 2019, pp. 2100–2110.
evolvingdigitaltwintoolforengineeringdynamics applications, Mechani- [33] G. D. Kontes, D. D. Scherer, T. Nisslbeck, J. Fischer, C. Mutschler,
cal Systems and Signal Processing 188 (2023) 1–16. High-speed collision avoidance using deep reinforcement learning and
[13] X. Wang, L. Li, Y. Yuan, P. Ye, F.-Y. Wang, Acp-based social domain randomization for autonomous vehicles, in: 2020 IEEE 23rd
computing and parallel intelligence: Societies 5.0 and beyond, CAAI international conference on Intelligent Transportation Systems (ITSC),
Transactions on Intelligence Technology 1 (4) (2016) 377–393. IEEE, 2020, pp. 1–8.
[14] S. Shah, D. Dey, C. Lovett, A. Kapoor, Airsim: High-fidelity visual [34] S. Pouyanfar, M. Saleem, N. George, S.-C. Chen, Roads: Randomiza-
and physical simulation for autonomous vehicles, in: Field and Service tion for obstacle avoidance and driving in simulation, in: Proceedings
Robotics: Results of the 11th International Conference, Springer, 2018, of the IEEE/CVF Conference on Computer Vision and Pattern Recog-
pp. 621–635. nition Workshops, 2019, pp. 0–0.
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 16

[35] J. Kim, C. Park, End-to-end ego lane estimation based on sequential IEEE/RSJ International Conference on Intelligent Robots and Systems
transfer learning for self-driving cars, in: Proceedings of the IEEE (IROS), IEEE, 2015, pp. 5307–5314.
conference on computer vision and pattern recognition workshops, [61] K. Weiss, T. M. Khoshgoftaar, D. Wang, A survey of transfer learning,
2017, pp. 30–38. Journal of Big data 3 (1) (2016) 1–40.
[36] S. Akhauri, L. Y. Zheng, M. C. Lin, Enhanced transfer learning for [62] D. Isele, A. Cosgun, Transferring autonomous driving knowledge
autonomous driving with systematic accident simulation, in: 2020 on simulated and real intersections, arXiv preprint arXiv:1712.01106
IEEE/RSJ International Conference on Intelligent Robots and Systems (2017).
(IROS), IEEE, 2020, pp. 5986–5993. [63] E. Tzeng, C. Devin, J. Hoffman, C. Finn, P. Abbeel, S. Levine,
[37] Y. Bengio, J. Louradour, R. Collobert, J. Weston, Curriculum learning, K. Saenko, T. Darrell, Adapting deep visuomotor representations with
in: Proceedings of the 26th annual international conference on machine weak pairwise constraints, in: Algorithmic Foundations of Robotics
learning, 2009, pp. 41–48. XII: Proceedings of the Twelfth Workshop on the Algorithmic Foun-
[38] M. Kumar, B. Packer, D. Koller, Self-paced learning for latent variable dations of Robotics, Springer, 2020, pp. 688–703.
models, Advances in neural information processing systems 23 (2010). [64] H. B. Ammar, E. Eaton, P. Ruvolo, M. Taylor, Online multi-task
[39] P. Soviany, R. T. Ionescu, P. Rota, N. Sebe, Curriculum self-paced learning for policy gradient methods, in: International conference on
learning for cross-domain object detection, Computer Vision and Image machine learning, PMLR, 2014, pp. 1206–1214.
Understanding 204 (2021) 103166. [65] S. Sharma, J. E. Ball, B. Tang, D. W. Carruth, M. Doude, M. A. Islam,
[40] T. Matiisen, A. Oliver, T. Cohen, J. Schulman, Teacher–student cur- Semantic segmentation with transfer learning for off-road autonomous
riculum learning, IEEE transactions on neural networks and learning driving, Sensors 19 (11) (2019) 2577.
systems 31 (9) (2019) 3732–3740. [66] C. Schwarz, Z. Wang, The role of digital twins in connected and
[41] S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, P. Stone, automated vehicles, IEEE Intelligent Transportation Systems Magazine
Curriculum learning for reinforcement learning domains: A framework 14 (6) (2022) 41–51.
and survey, The Journal of Machine Learning Research 21 (1) (2020) [67] M. Singh, E. Fuenmayor, E. P. Hinchy, Y. Qiao, N. Murray, D. Devine,
7382–7431. Digital twin: Origin to future, Applied System Innovation 4 (2) (2021)
[42] C. Florensa, D. Held, M. Wulfmeier, M. Zhang, P. Abbeel, Reverse 36.
curriculum generation for reinforcement learning, in: Conference on [68] Y. Vered, S. J. Elliott, The use of digital twins to remotely update
robot learning, PMLR, 2017, pp. 482–495. feedback controllers for the motion control of nonlinear dynamic
[43] J. Schmidhuber, Evolutionary principles in self-referential learning, systems, Mechanical Systems and Signal Processing 185 (2023) 1–17.
or on learning how to learn: the meta-meta-... hook, Ph.D. thesis, [69] T. Fei, X. Bin, Q. Qinglin, C. Jiangfeng, J. Ping, Digital twin modeling,
Technische Universität München (1987). Journal of Manufacturing Systems 64 (2022) 372–389.
[44] S. Thrun, L. Pratt, Learning to learn: Introduction and overview, [70] B. Mark, C. Adrian, L. Gun, et al., A survey of augmented reality,
Learning to learn (1998) 3–17. Foundations and Trends® in Human–Computer Interaction 8 (2-3)
[45] M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, (2015) 73–272.
T. Schaul, B. Shillingford, N. De Freitas, Learning to learn by gradient [71] R. T. Azuma, A survey of augmented reality, Presence: teleoperators
descent by gradient descent, Advances in neural information processing & virtual environments 6 (4) (1997) 355–385.
systems 29 (2016). [72] E. Costanza, A. Kunz, M. Fjeld, Mixed reality: A survey, Springer,
[46] J. Snell, K. Swersky, R. Zemel, Prototypical networks for few-shot 2009.
learning, Advances in neural information processing systems 30 (2017). [73] W. Hoenig, C. Milanes, L. Scaria, T. Phan, M. Bolas, N. Ayanian,
Mixed reality for robotics, in: 2015 IEEE/RSJ International Conference
[47] E. Real, A. Aggarwal, Y. Huang, Q. V. Le, Regularized evolution
on Intelligent Robots and Systems (IROS), IEEE, 2015, pp. 5382–5387.
for image classifier architecture search, in: Proceedings of the aaai
[74] P. Lindemann, T.-Y. Lee, G. Rigoll, An explanatory windshield display
conference on artificial intelligence, Vol. 33, 2019, pp. 4780–4789.
interface with augmented reality elements for urban autonomous driv-
[48] C. Finn, P. Abbeel, S. Levine, Model-agnostic meta-learning for fast
ing, in: 2018 IEEE International Symposium on Mixed and Augmented
adaptation of deep networks, in: International conference on machine
Reality Adjunct (ISMAR-Adjunct), IEEE, 2018, pp. 36–37.
learning, PMLR, 2017, pp. 1126–1135.
[75] S. Su, X. Zeng, S. Song, M. Lin, H. Dai, W. Yang, C. Hu, Positioning
[49] S. Sæmundsson, K. Hofmann, M. P. Deisenroth, Meta reinforce- accuracy improvement of automated guided vehicles based on a novel
ment learning with latent variable gaussian processes, arXiv preprint magnetic tracking approach, IEEE Intelligent Transportation Systems
arXiv:1803.07551 (2018). Magazine 12 (4) (2018) 138–148.
[50] G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural [76] M. Szalai, B. Varga, T. Tettamanti, V. Tihanyi, Mixed reality test
network, arXiv preprint arXiv:1503.02531 (2015). environment for autonomous cars using unity 3d and sumo, in: 2020
[51] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with IEEE 18th World Symposium on Applied Machine Intelligence and
deep convolutional neural networks, Communications of the ACM Informatics (SAMI), IEEE, 2020, pp. 73–78.
60 (6) (2017) 84–90. [77] M. R. Zofka, M. Essinger, T. Fleck, R. Kohlhaas, J. M. Zöllner,
[52] Z. Xu, Y.-C. Hsu, J. Huang, Training shallow and thin networks for The sleepwalker framework: Verification and validation of autonomous
acceleration via knowledge distillation with conditional adversarial vehicles by mixed reality lidar stimulation, in: 2018 IEEE Interna-
networks, arXiv preprint arXiv:1709.00513 (2017). tional Conference on Simulation, Modeling, and Programming for
[53] J. Morimoto, K. Doya, Robust reinforcement learning, Neural compu- Autonomous Robots (SIMPAR), IEEE, 2018, pp. 151–157.
tation 17 (2) (2005) 335–359. [78] X. Gao, X. Wu, S. Ho, T. Misu, K. Akash, Effects of augmented-reality-
[54] L. Pinto, J. Davidson, R. Sukthankar, A. Gupta, Robust adversarial rein- based assisting interfaces on drivers’ object-wise situational awareness
forcement learning, in: International Conference on Machine Learning, in highly autonomous vehicles, in: 2022 IEEE Intelligent Vehicles
PMLR, 2017, pp. 2817–2826. Symposium (IV), IEEE, 2022, pp. 563–572.
[55] X. He, H. Yang, Z. Hu, C. Lv, Robust lane change decision making [79] Z. Wang, K. Han, P. Tiwari, Augmented reality-based advanced driver-
for autonomous vehicles: An observation adversarial reinforcement assistance system for connected vehicles, in: 2020 ieee international
learning approach, IEEE Transactions on Intelligent Vehicles (2022). conference on systems, man, and cybernetics (SMC), IEEE, 2020, pp.
[56] P. Huang, M. Xu, F. Fang, D. Zhao, Robust reinforcement learning as a 752–759.
stackelberg game via adaptively-regularized adversarial training, arXiv [80] R. Williams, J. A. Erkoyuncu, T. Masood, et al., Augmented reality as-
preprint arXiv:2202.09514 (2022). sisted calibration of digital twins of mobile robots, IFAC-PapersOnLine
[57] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, P. Abbeel, 53 (3) (2020) 203–208.
Domain randomization for transferring deep neural networks from sim- [81] T. Tettamanti, M. Szalai, S. Vass, V. Tihanyi, Vehicle-in-the-loop test
ulation to the real world, in: 2017 IEEE/RSJ international conference environment for autonomous driving with microscopic traffic simula-
on intelligent robots and systems (IROS), IEEE, 2017, pp. 23–30. tion, in: 2018 IEEE International Conference on Vehicular Electronics
[58] F. Sadeghi, S. Levine, Cad2rl: Real single-image flight without a single and Safety (ICVES), IEEE, 2018, pp. 1–6.
real image, arXiv preprint arXiv:1611.04201 (2016). [82] R. Mitchell, J. Fletcher, J. Panerati, A. Prorok, Multi-vehicle mixed-
[59] W. Yu, J. Tan, C. K. Liu, G. Turk, Preparing for the unknown: Learning reality reinforcement learning for autonomous multi-lane driving, arXiv
a universal policy with online system identification, arXiv preprint preprint arXiv:1911.11699 (2019).
arXiv:1702.02453 (2017). [83] R. Moezzi, D. Krcmarik, H. Bahri, J. Hlava, Autonomous vehicle
[60] I. Mordatch, K. Lowrey, E. Todorov, Ensemble-cio: Full-body dy- control based on hololens technology and raspberry pi platform: An
namic motion planning that transfers to physical humanoids, in: 2015 educational perspective, IFAC-PapersOnLine 52 (27) (2019) 80–85.
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 17

[84] X. Tu, J. Autiosalo, A. Jadid, K. Tammi, G. Klinker, A mixed reality [104] H. Xiong, Z. Wang, G. Wu, Y. Pan, Design and implementation of
interface for a digital twin based crane, Applied Sciences 11 (20) (2021) digital twin-assisted simulation method for autonomous vehicle in car-
9480. following scenario, Journal of Sensors 2022 (2022).
[85] S. H. Choi, K.-B. Park, D. H. Roh, J. Y. Lee, M. Mohammed, [105] S. Almeaibed, S. Al-Rubaye, A. Tsourdos, N. P. Avdelidis, Digital twin
Y. Ghasemi, H. Jeong, An integrated mixed reality system for safety- analysis to promote safety and security in autonomous vehicles, IEEE
aware human-robot collaboration using deep learning and digital Communications Standards Magazine 5 (1) (2021) 40–46.
twin generation, Robotics and Computer-Integrated Manufacturing 73 [106] A. Niaz, M. U. Shoukat, Y. Jia, S. Khan, F. Niaz, M. U. Raza,
(2022) 102258. Autonomous driving test method based on digital twin: A survey, in:
[86] K. Lalik, S. Flaga, A real-time distance measurement system for a 2021 International Conference on Computing, Electronic and Electrical
digital twin using mixed reality goggles, Sensors 21 (23) (2021) 7870. Engineering (ICE Cube), IEEE, 2021, pp. 1–7.
[87] B. Yu, C. Chen, J. Tang, S. Liu, J.-L. Gaudiot, Autonomous vehicles [107] S.-H. Wang, C.-H. Tu, J.-C. Juang, Automatic traffic modelling for
digital twin: A practical paradigm for autonomous driving system creating digital twins to facilitate autonomous vehicle development,
development, Computer 55 (9) (2022) 26–34. Connection Science 34 (1) (2022) 1018–1037.
[88] E. Salvato, G. Fenu, E. Medvet, F. A. Pellegrino, Crossing the reality [108] X. Xu, K. Liu, P. Dai, B. Chen, Enabling digital twin in vehicular edge
gap: A survey on sim-to-real transferability of robot controllers in computing: A multi-agent multi-objective deep reinforcement learning
reinforcement learning, IEEE Access 9 (2021) 153171–153187. solution, arXiv preprint arXiv:2210.17386 (2022).
[89] Y. Laschinsky, K. von Neumann-Cosel, M. Gonter, C. Wegwerth, [109] G. Rong, B. H. Shin, H. Tabatabaee, Q. Lu, S. Lemke, M. Možeiko,
R. Dubitzky, A. Knoll, Evaluation of an active safety light using E. Boise, G. Uhm, M. Gerow, S. Mehta, et al., Lgsvl simulator: A
virtual test drive within vehicle in the loop, in: 2010 IEEE International high fidelity simulator for autonomous driving, in: 2020 IEEE 23rd
Conference on Industrial Technology, IEEE, 2010, pp. 1119–1112. International conference on intelligent transportation systems (ITSC),
[90] H. Shikata, T. Yamashita, K. Arai, T. Nakano, K. Hatanaka, H. Fu- IEEE, 2020, pp. 1–6.
jikawa, Digital twin environment to integrate vehicle simulation and [110] Z. Hu, S. Lou, Y. Xing, X. Wang, D. Cao, C. Lv, Review and
physical verification, SEI Technical Review 88 (2019) 18–21. perspectives on driver digital twin and its enabling technologies for
[91] V. Dygalo, A. Keller, A. Shcherbin, Principles of application of virtual intelligent vehicles, IEEE Transactions on Intelligent Vehicles (2022).
and physical simulation technology in production of digital twin of [111] F.-Y. Wang, Parallel system methods for management and control of
active vehicle safety systems, Transportation research procedia 50 complex systems, CONTROL AND DECISION. 19 (2004) 485–489.
(2020) 121–129. [112] K. M. Alam, A. El Saddik, C2ps: A digital twin architecture reference
[92] J. Wu, Z. Huang, P. Hang, C. Huang, N. De Boer, C. Lv, Digital twin- model for the cloud-based cyber-physical systems, IEEE access 5
enabled reinforcement learning for end-to-end autonomous driving, in: (2017) 2050–2062.
2021 IEEE 1st International Conference on Digital Twins and Parallel [113] G. Bhatti, H. Mohan, R. R. Singh, Towards the future of smart electric
Intelligence (DTPI), IEEE, 2021, pp. 62–65. vehicles: Digital twin technology, Renewable and Sustainable Energy
[93] C. Schwarz, K. Moran, Digital map enhancements of electronic stabil- Reviews 141 (2021) 110801.
ity control, SAE International, Warrendale, PA, USA, SAE Tech. Paper [114] F.-Y. Wang, Toward a revolution in transportation operations: Ai for
(2010) 0148–7191. complex systems, IEEE Intelligent Systems 23 (6) (2008) 8–13.
[94] E. Bottani, A. Cammardella, T. Murino, S. Vespoli, et al., From the [115] L. Chen, X. Hu, B. Tang, D. Cao, Parallel motion planning: Learning
cyber-physical system to the digital twin: the process development for a deep planning model against emergencies, IEEE Intelligent Trans-
behaviour modelling of a cyber guided vehicle in m2m logic, XXII portation Systems Magazine 11 (1) (2018) 36–41.
Summer School Francesco TurcoIndustrial Systems Engineering (2017) [116] K. Wang, C. Gou, N. Zheng, J. M. Rehg, F.-Y. Wang, Parallel vision for
1–7. perception and understanding of complex scenes: methods, framework,
[95] X. Chen, E. Kang, S. Shiraishi, V. M. Preciado, Z. Jiang, Digital and perspectives, Artificial Intelligence Review 48 (2017) 299–329.
behavioral twins for safe connected cars, in: Proceedings of the [117] F.-Y. Wang, Parallel control and management for intelligent trans-
21th ACM/IEEE international conference on model driven engineering portation systems: Concepts, architectures, and applications, IEEE
languages and systems, 2018, pp. 144–153. transactions on intelligent transportation systems 11 (3) (2010) 630–
[96] A. Rassõlkin, T. Vaimann, A. Kallaste, V. Kuts, Digital twin for 638.
propulsion drive of autonomous electric vehicle, in: 2019 IEEE 60th [118] J. Lu, Q. Wei, T. Zhou, Z. Wang, F.-Y. Wang, Event-triggered near-
International Scientific Conference on Power and Electrical Engineer- optimal control for unknown discrete-time nonlinear systems using
ing of Riga Technical University (RTUCON), IEEE, 2019, pp. 1–4. parallel control, IEEE Transactions on Cybernetics (2022).
[97] Y. Ge, Y. Wang, R. Yu, Q. Han, Y. Chen, Research on test method [119] L. Li, X. Wang, K. Wang, Y. Lin, J. Xin, L. Chen, L. Xu, B. Tian, Y. Ai,
of autonomous driving based on digital twin, in: 2019 IEEE Vehicular J. Wang, et al., Parallel testing of vehicle intelligence via virtual-real
Networking Conference (VNC), IEEE, 2019, pp. 1–2. interaction, Science robotics 4 (28) (2019) eaaw4106.
[98] O. Veledar, V. Damjanovic-Behrendt, G. Macher, Digital twins for [120] T. Liu, X. Wang, Y. Xing, Y. Gao, B. Tian, L. Chen, Research
dependability improvement of autonomous driving, in: Systems, Soft- on digital quadruplets in cyber-physical-social space-based parallel
ware and Services Process Improvement: 26th European Conference, driving, Chinese Journal of Intelligent Science and Technology 1 (1)
EuroSPI 2019, Edinburgh, UK, September 18–20, 2019, Proceedings (2019) 40–51.
26, Springer, 2019, pp. 415–426. [121] J. Yang, X. Wang, Y. Zhao, Parallel manufacturing for industrial
[99] Y. Liu, Z. Wang, K. Han, Z. Shou, P. Tiwari, J. H. Hansen, Sensor metaverses: A new paradigm in smart manufacturing, IEEE/CAA
fusion of camera and cloud digital twin information for intelligent Journal of Automatica Sinica 9 (12) (2022) 2063–2070.
vehicles, in: 2020 IEEE Intelligent Vehicles Symposium (IV), IEEE, [122] L. Li, Y.-L. Lin, D.-P. Cao, N.-N. Zheng, F.-Y. Wang, Parallel learning-
2020, pp. 182–187. a new framework for machine learning, Acta Automatica Sinica 43 (1)
[100] S. Liu, B. Yu, J. Tang, Q. Zhu, Towards fully intelligent transportation (2017) 1–8.
through infrastructure-vehicle cooperative autonomous driving: Chal- [123] W. Zhang, K. Wang, Y. Liu, Y. Lu, F.-Y. Wang, A parallel vision
lenges and opportunities, in: 2021 58th ACM/IEEE Design Automation approach to scene-specific pedestrian detection, Neurocomputing 394
Conference (DAC), IEEE, 2021, pp. 1323–1326. (2020) 114–126.
[101] J. Culley, S. Garlick, E. G. Esteller, P. Georgiev, I. Fursa, I. Van- [124] W. Zheng, K. Wang, F.-Y. Wang, A novel background subtraction
der Sluis, P. Ball, A. Bradley, System design for a driverless au- algorithm based on parallel vision and bayesian gans, Neurocomputing
tonomous racing vehicle, in: 2020 12th International Symposium on 394 (2020) 178–200.
Communication Systems, Networks and Digital Signal Processing [125] F.-Y. Wang, N.-N. Zheng, D. Cao, C. M. Martinez, L. Li, T. Liu,
(CSNDSP), IEEE, 2020, pp. 1–6. Parallel driving in cpss: A unified approach for transport automation
[102] D. J. Fremont, E. Kim, Y. V. Pant, S. A. Seshia, A. Acharya, X. Bruso, and vehicle intelligence, IEEE/CAA Journal of Automatica Sinica 4 (4)
P. Wells, S. Lemke, Q. Lu, S. Mehta, Formal scenario-based testing of (2017) 577–587.
autonomous vehicles: From simulation to the real world, in: 2020 IEEE [126] L. Chen, Q. Wang, X. Lu, D. Cao, F.-Y. Wang, Learning driving models
23rd International Conference on Intelligent Transportation Systems from parallel end-to-end driving data set, Proceedings of the IEEE
(ITSC), IEEE, 2020, pp. 1–8. 108 (2) (2019) 262–273.
[103] K. Voogd, J. P. Allamaa, J. Alonso-Mora, T. D. Son, Reinforcement [127] T. Liu, X. Yang, H. Wang, X. Tang, L. Chen, H. Yu, F.-Y. Wang, Digital
learning from simulation to real world autonomous driving using digital quadruplets for cyber-physical-social systems based parallel driving:
twin, arXiv preprint arXiv:2211.14874 (2022). From concept to applications, arXiv preprint arXiv:2007.10799 (2020).
JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. XX, XXX XXX 18

[128] L. Chen, Y. Zhang, B. Tian, Y. Ai, D. Cao, F.-Y. Wang, Parallel driving Shen Li received the B.S. degree in Computer
os: A ubiquitous operating system for autonomous driving in cpss, Science and Technology from Liaoning University in
IEEE Transactions on Intelligent Vehicles 7 (4) (2022) 886–895. 2018. From September 2022 to now, he is pursuing
[129] H. Liu, Y. Sun, J. Cao, S. Chen, N. Pan, Y. Dai, D. Pan, Study on his Master’s degree in School of Artificial Intelli-
uav parallel planning system for transmission line project acceptance gence, Hubei University, Wuhan, China. His areas
under the background of industry 5.0, IEEE Transactions on Industrial of interest include deep learning and autonomous
Informatics 18 (8) (2022) 5537–5546. driving.
[130] K. Wang, C. Gou, F. Wang, Parallel vision: An acp-based approach to
intelligent vision computing, Acta Automatica Sinica 42 (10) (2016)
1490–1500.
[131] L. Chen, X. Hu, W. Tian, H. Wang, D. Cao, F.-Y. Wang, Parallel
planning: A new motion planning framework for autonomous driving, Tingyu Huang received the B.S. degree in Com-
IEEE/CAA Journal of Automatica Sinica 6 (1) (2018) 236–246. munication Engineering from Hunan Institute of
[132] F.-Y. Wang, On the modeling, analysis, control and management of Science and Technology in 2022. From September
complex systems, Fuza Xitong yu Fuzaxing Kexue(Complex Systems 2022 to now, she is pursuing her Master’s degree in
and Complexity Science) 3 (2) (2006) 26–34. School of Artificial Intelligence, Hubei University,
[133] L. Chen, Q. Wang, X. Lu, D. Cao, F.-Y. Wang, Learning driving models Wuhan, China. Her areas of interest include deep
from parallel end-to-end driving data set, Proceedings of the IEEE learning and autonomous driving.
108 (2) (2020) 262–273. doi:10.1109/JPROC.2019.2952735.
[134] N. Koenig, A. Howard, Design and use paradigms for gazebo, an
open-source multi-robot simulator, in: 2004 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No.
04CH37566), Vol. 3, IEEE, 2004, pp. 2149–2154. Bo Tang is an Associate Professor in the Depart-
[135] B. Wymann, E. Espié, C. Guionneau, C. Dimitrakakis, R. Coulom, ment of Electrical and Computer Engineering at
A. Sumner, Torcs, the open racing car simulator, Software available at Worcester Polytechnic Institute. Prior to this, he
http://torcs. sourceforge. net 4 (6) (2000) 2. was an Assistant Professor in the Department of
[136] Q. Li, Z. Peng, L. Feng, Q. Zhang, Z. Xue, B. Zhou, Metadrive: Electrical and Computer Engineering at Mississippi
Composing diverse driving scenarios for generalizable reinforcement State University. He received the Ph.D. degree in
learning, IEEE transactions on pattern analysis and machine intelli- electrical engineering from University of Rhode Is-
gence (2022). land (Kingstown, RI) in 2016. His research interests
[137] D. Krajzewicz, Traffic simulation with sumo–simulation of urban lie in the general areas of bio-inspired artificial
mobility, Fundamentals of traffic simulation (2010) 269–293. intelligence (AI), AI security, edge AI, and their
[138] P. Cai, Y. Lee, Y. Luo, D. Hsu, Summit: A simulator for urban driving applications in Cyber-Physical Systems (e.g., wire-
in massive mixed traffic, in: 2020 IEEE International Conference on less networks, autonomous vehicles, and power systems). He is currently an
Robotics and Automation (ICRA), IEEE, 2020, pp. 4023–4029. Associate Editor for IEEE Transactions on Neural Networks and Learning
[139] S. Kato, S. Tokunaga, Y. Maruyama, S. Maeda, M. Hirabayashi, Systems.
Y. Kitsukawa, A. Monrroy, T. Ando, Y. Fujii, T. Azumi, Autoware on
board: Enabling autonomous vehicles with embedded systems, in: 2018 Rouxing Huai currently an Advisory Scientist at
ACM/IEEE 9th International Conference on Cyber-Physical Systems Beijing Huairou Academy of Parallel Sensing, is a
(ICCPS), IEEE, 2018, pp. 287–296. senior researcher in mechatronics, intelligent sys-
[140] X. Huang, X. Cheng, Q. Geng, B. Cao, D. Zhou, P. Wang, Y. Lin, tems, color optics, and parallel optical fields. He
R. Yang, The apolloscape dataset for autonomous driving, in: Proceed- received his PhD in Computer and Systems En-
ings of the IEEE conference on computer vision and pattern recognition gineering from USA in 1990 and worked at the
workshops, 2018, pp. 954–960. University of Arizona for 21 years in Robotics
[141] R. F. Benekohal, J. Treiterer, Carsim: Car-following model for simu- and Intelligent Systems for Manufacturing, Space
lation of traffic in normal and stop-and-go conditions, Transportation Exploration, and Optical Applications.
research record 1194 (1988) 99–111.
[142] D. Krajzewicz, J. Erdmann, M. Behrisch, L. Bieker, Recent de-
velopment and applications of sumo-simulation of urban mobility, Long Chen is currently a Professor with State Key
International journal on advances in systems and measurements 5 (3&4) Laboratory of Management and Control for Complex
(2012). Systems, Institute of Automation, Chinese Academy
of Sciences, Beijing, China. His research interests
include autonomous driving, robotics, and artificial
intelligence, where he has contributed more than 100
publications. He received the IEEE Vehicular Tech-
nology Society 2018 Best Land Transportation Paper
Award, the IEEE Intelligent Vehicle Symposium
2018 Best Student Paper Award and Best Workshop
Paper Award, the IEEE Intelligent Transportation
Systems Society 2021 Outstanding Application Award, the IEEE Conference
on Digital Twin and Parallel Intelligence 2021 Best Paper and Outstanding
Xuemin Hu is currently an Associate Professor Paper Award, the IEEE International Conference on Intelligent Transportation
with School of Artificial Intelligence, Hubei Univer- Systems 2021 Best Paper Award. He serves as an Associate Editor for
sity, Wuhan, China. He received the B.S. degree in the IEEE Transaction on Intelligent Transportation Systems, the IEEE/CAA
Biomedical Engineering from Huazhong University Journal of Automatica Sinica, the IEEE Transaction on Intelligent Vehicle and
of Science and Technology and the Ph.D. degree the IEEE Technical Committee on Cyber-Physical Systems.
in Signal and Information Processing from Wuhan
University in 2007 and in 2012, respectively. He was
a visiting scholar in the University of Rhode Island,
Kingston, RI, US from November 2015 to May
2016. His areas of interest include computer vision,
machine learning, motion planning, and autonomous
driving.In this paragraph you can place your educational, professional back-
ground and research and other interests.