See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/351871154

Systematic Modeling Approach for Environmental Perception Limitations in

Automated Driving

Preprint · May 2021

CITATIONS READS

0 350

3 authors, including:

Ahmad Adee Roman Gansch

Bosch Bosch
7 PUBLICATIONS   12 CITATIONS    25 PUBLICATIONS   265 CITATIONS   

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Ahmad Adee on 13 September 2021.

The user has requested enhancement of the downloaded file.

 Systematic Modeling Approach for Environmental
Perception Limitations in Automated Driving
Ahmad Adee Roman Gansch Peter Liggesmeyer
Corporate Research, Corporate Research, Software Engineering Institute
Robert Bosch GmbH, Robert Bosch GmbH, University of Kaiserslautern,
Renningen, Germany Renningen, Germany Kaiserslautern, Germany
[email protected] [email protected] [email protected]

Abstract—Highly automated driving (HAD) vehicles are com- situational awareness is critical to safety and the situational
plex systems operating in an open context. Complexity of these awareness is derived from complex sensors and processing
systems as well as limitations and insufficiencies in sensing algorithms [2].
and understanding the open context may result in unsafe and
uncertain behavior. The safety critical nature of the HAD vehicles Evaluating a perception system (sensor and its processing
demands to model limitations, insufficiencies and triggering algorithm) in terms of their limitations, capabilities or inherent
conditions to argue safe behavior. uncertainties is not a straightforward task. A perception system
Standardization activities such as ISO/PAS 21448 provide cannot be characterized based on a rudimentary set of safety
guidelines on the safety of the intended functionality (SOTIF) and
requirements or key performance indicators (KPIs), as the
focus on the performance limitations and triggering conditions.
Although, SOTIF provides a non-exhaustive list of scenario performance of such system depends on many influencing
factors that may serve as a starting point to identify and factors. For example, functional performance of a LIDAR
analyze performance limitations and triggering conditions, yet based perception system may depend on the spatial distribution
no concrete methodology is provided to model these factors. of detection, reflection, weather and road conditions.
We propose a novel methodology to model triggering condi-
tions and performance limitations in a scene to assess SOTIF. We
Modeling the dependencies and influencing factors of the
utilize Bayesian network (BN) in this regard. The experts provide perception system to assess performance limitations and con-
the BN structure and conditional belief tables are learned using sequently the relevant uncertainties is important for SOTIF
the maximum likelihood estimator. We provide performance argumentation [3]. Such models can provide valuable insights
limitation maps (PLMs) and conditional performance limitation on the functional performance of the system during develop-
maps (CPLMs), given a scene. As a case study, we provide PLMs
and CPLMs of LIDAR in a defined scene using real world data.
ment. ISO/PAS 21448 [2] provides a list of such dependencies
in terms of scenario factors but does not provide concrete steps
Index Terms—SOTIF, autonomous vehicle safety, safety of the to model these scenario factors.
intended functionality, Bayesian networks, parameter learning Probabilistic graphical models (PGMs) [4] in general and
Bayesian networks (BNs) [5] in particular have rapidly gained
I. I NTRODUCTION popularity in the dependability research [6], [7]. The BN
Highly automated driving (HAD) vehicles are complex is a directed acyclic graph (DAG) that consists of nodes
systems operating in an open context [1]. The complexity and edges. Every node is a random variable (X1 , . . . , Xn ),
and open context nature may result in unsafe and uncertain which represents an element of the system or its context. The
behavior due to limitations and insufficiencies in sensing and edges represent a directed relationship between two nodes
understanding the operational environment [1]. Modeling such and run from the parent node (pa) towards the child node
limitations and insufficiencies requires the consideration of all (ch). Together, nodes and edges represent the structure of the
possible scenarios and factors influencing the HAD vehicle probabilistic network (Fig. 1). The strength of these depen-
performance. The international organization for standardiza- dencies are governed by conditional probability distributions
tion (ISO) published the publicly available specification (PAS), Pr(ch | pa) [4]. Mathematically, the BN can be written as
ISO/PAS 21448 road vehicles safety of the intended function- follows.
ality (SOTIF) [2]. The goal of the SOTIF guidelines is to n
Y
identify the performance limitations and triggering conditions Pr(X1 , . . . , Xn ) = Pr(Xi | pa(Xi )) (1)
that may lead to potentially hazardous behavior. Specifically, i
SOTIF is applied to the intended functionality where proper
BN is effective in modeling uncertainty and probability reason-
The research leading to these results has received funding from the ing of a system. It exploits the dependence relationship through
European Unions Horizon 2020 research and innovation program under the the local conditions in the model to perform uncertainty
Marie Skłodowska-Curie grant agreement No 812.788 (MSCA-ETN SAS).
This publication reflects only the authors view, exempting the European Union analysis for prediction, classification and causal inference of
from any liability. Project website: http://etn-sas.eu/. influencing factors.
 In this publication, we formulate a model using BN for
known triggering conditions and performance limitations in
a given scene. A human expert provides the SOTIF relevant
scenario factors and models the causal relations among them
using a BN structure. We perform parameter learning of
BN to quantify the dependencies in the model. In order to
explain the performance limitations and triggering conditions
effects on SOTIF, posterior probability analysis and causal Truncation Reflection Occlusion
inference is conducted. We construct performance limitation Road= Road =
Weather Dry
maps (PLMs) and conditional performance limitation maps Illumination Road Wet

(CPLMs) using these analyses. Causal inference identifies Cloudy 0.7 0.3
Weather Rainy 0.5 0.5
the most contributing influencing factors on performance. To-
gether, PLMs, CPLMs and causal inference provide valuable Fig. 1: An example of grid map and scene modeling attributed
insights on the SOTIF. This may help the analyst in the safety to the cells: LIDAR detections are discretized in grid cell
case generation, identification of the performance limitations, around the field of view. Four LIDARs are attached at the
generation of targeted test, validation and verification cam- roof of the HAD vehicle for detection. Bottom part shows
paigns and influencing factors, which in turn can help in a Bayesian network along with conditional belief table for
defining refinement measures. Summarizing, we provide the Pr(Road | W eather).
following contributions.
• We introduce a method to model known triggering con-
Start
ditions and performance limitations in a scene.
• We introduce PLMs as the representation of SOTIF
SOTIF relevant

Refinement
metric. scenario
• We introduce CPLMs to quantify the effects of triggering factors
conditions and influencing factors on SOTIF.
• We implement the methodology and provide PLMs and Model of Data
causal acquisition
CPLMs of LIDARs case study while utilizing real world relations and pre-
data. (BN Structure) processing
Human
Database
The publication is structured as follows: Sec. II presents Expert
the proposed methodology. Sec. III briefly describes the setup Parameter
Analyze
learning
used for data acquisition. Sec. IV provides the application
of proposed methodology on LIDAR perception. In sec. V,
results of the implementation are evaluated. Sec. VI provides End
the evaluation of the approach and robustness of the results.
Sec. VII provides an overview on the state of the art. Finally, Fig. 2: Flowchart describing the flow of the proposed method-
in sec. VIII we discuss conclusion and future work. ology. SOTIF relevant scenario factors and expert knowledge
are encoded into scene model defined by the BN structure.
II. P ROPOSED M ETHODOLOGY Data is gathered accordingly and learning of parameters is
We introduce a modeling methodology using BN to identify, performed.
model and quantify performance limitations as well as trigger-
ing conditions in a scene. The experts provide the structure of
BN while the conditional belief tables (CBTs) are learned from setup among other. For example, consider the following two
real sensor data. Fig. 2 shows the flowchart of the methodology descriptions.
we adopt in this publication. A detailed explanation of the 1) Context: Highway, Perception: Radar based, Studied be-
steps proposed in the flowchart (Fig. 2) follows. havior: False Positives.
2) Context: Urban, Perception: LIDAR based, Studied be-
A. SOTIF Relevant Scenario Factors havior: Position Trueness.
The first step towards modeling relevant SOTIF scenario Both description may lead to different scenario factors. In
factors is the identification of performance limitations and the former, the human expert might be interested in steel
triggering conditions in a given scene [2]. SOTIF provides bridges, tin cans and other such instances while in the latter
a dynamic element and scenery centric non-exhaustive list of the factors of interest may include weather conditions, exhaust
scenario factors [2]. Although this list can be a starting point, gases and reflections. The process is similar to hazard iden-
yet identification of triggering conditions and performance tification and risk assessment (HARA) from ISO 26262 [8],
limitations is dependent on many other aspects including the but does not explicitly considers malfunctioning behavior of
context of driving, perception system in question and existing components. It assesses the intended functionality of HAD
vehicle functions especially where situational awareness is between undesired behavior, triggering conditions of the scene,
critical to safety. We utilize the scenario factors from ISO/PAS and performance limitations and provide propositions e.g. the
21448 [2] as well as expert opinion, previous data and existing proposition p1 : high occlusion may result in higher FNs. We
setup (constraint on data acquisition and/or data labels) to then construct BN with arcs representing the dependencies
model the scene in our methodology (Fig. 2). and nodes representing the undesired behavior, triggering
SOTIF related undesired behavior (e.g. braking when not conditions and performance limitations derived from these
required and vice versa) may originate from FP and FN propositions e.g., the proposition p1 is modeled as an explicit
detections [2]. Since we are more focused at the perception node (Fig. 3). The resulting BN structure asserts that a child
level of the functionality, we only consider FN and true pos- node is governed by a causal mechanism that probabilistically
itive (TP) of the perception system. The overall methodology determines its value based on the values mechanism of its
we define in this publication, is however generic and can parents [4]. The stochastic attribute of such models helps
be applied to complete functional chain (sense, plan, decide modeling aleatory uncertainty [12].
and act) of the system under study. The choice of undesired
C. Data Acquisition and Pre-processing
behavior is highly dependent on the system under study, the
scene model and metrics that can support the safety case. Dataset D acquired and utilized in our methodology consists
Apart from TP and FN, SOTIF related undesired behavior of fully observed instances of the network variables.
such as FP, positional error, contour matching, classification D = ξ[1] . . . ξ[M ] (2)
as well as regression quality can also be modeled to assess the
Where ξ[.] represents a data instance and M represents the
performance limitation and the effects of triggering conditions
number of instances in D.
on the functional performance. As an example, for the second
We calculate SOTIF related undesired behavior for each data
case in which LIDAR based perception system is analyzed
instance, if the undesired behavior is not labeled. For example,
in the context of urban driving, the expert may provide the
data instances may not be labeled with FNs. However, this is
following factors.
an ad-hoc step for data processing that may or may not be
• Occlusion: In urban driving, there may be a relatively
required, depending upon the available dataset.
higher probability of occlusion occurrence as parked cars, In order to fully grasp the effects of SOTIF relevant scenario
trees may occlude objects. factors (conditional dependencies in BN) and performance
• FN/FP rate: The overall FN/FP rate in the urban context
limitations around the HAD vehicle, we discretize the spatial
of driving. distributions of detections in a grid map (Fig. 1). Modeling
• Weather conditions: Different weather may effect the
spatial distribution of triggering conditions and performance
LIDAR performance. limitations in a grid map is important for the following reasons.
• Reflection from objects: Reflection from different objects
1) Scenario factors are spatially distributed e.g. in a weather
(buses windows) effects the FP rate.
situation involving dense fog the FN rate of the grid cells
• Illumination: Higher illumination may increase the re-
farther from the HAD vehicles will be different than the
flection from objects.
nearer ones, for some perception systems.
The above-mentioned factors are non-exhaustive. Scenario 2) Safety criticality is variable around the vehicle in the
factors are provided and refined based on the expert opinion sense that events nearer to the HAD vehicle are generally
and ability for data acquisition. The resulting factors then can considered more critical.
be used to model the causal relation. Data instances thus can be spatially associated around the
B. Model of the Causal Relation HAD vehicle to fully associate the observed instances with
Modeling the qualitative and casual relations amongst the their respective detection points in space. In this way, a
scenario factors, triggering conditions and performance lim- grid map is created around the HAD vehicle to represent
itations is a significant component of this methodology. We SOTIF relevant perception metrics/properties (Fig. 1). For the
utilize BN structure for this purpose. Traditionally, the BN construction of grid map, a coordinate system (e.g. Cartesian
structure modeling is based either on the expert knowledge [9] or polar) is selected as well as the grid size. Each grid cell
or on the learning from data (structure learning) [10]. However, is then represented by a separate BN and its corresponding
in structure learning from data the number of graph candi- CBTs (Fig. 1). The structure of each BN is kept constant in
dates grow exponentially with the number of variables in the this work.
data [11]. Discerning true graph by using observational data Suppose the data instances are distributed into N number
alone from other graphs that model the same set of conditional of grid cells (thus N number of BNs) based on the Cartesian
independencies is also challenging. Due to these challenges, (x, y) or polar (r, θ) coordinates of detection. The dataset
we opt for the former technique in this work. (Eq. 2) can be re-written as.
Scene description, which include SOTIF relevant scenario Dk = ξ k [1] . . . ξ k [M k ]∀k ∈ K (3)
factors and corresponding undesired behavior(s) constitute the
Where K is a set as follows.
nodes of the BN structure. As a first step towards derivation of
the structure, the experts establish hierarchical dependencies K = {1, 2, . . . , N } (4)
Here k represent k th grid cell and BN. A. SOTIF Relevant Scenario Factors
D. Parameter Learning The experts provide different factors that may effect the
Once BN structure (Sec. II-B) is determined and corre- LIDAR perception system performance (Sec. II-A). Based on
sponding data is acquired (Sec. II-C), the CBTs can be the expert inputs, SOTIF scenario factors and availability of
learned. We determine the CBTs and thus the strength of the data acquisition setup, we include the nodes shown in Fig. 3
dependencies by utilizing the maximum likelihood estimator as SOTIF relevant scenario factors.
(MLE) [4]. We perform non-parametric learning, not assuming Truncated or occluded objects may only produce sparse
prior probabilities. Given a variable X with parents U, we will point measurements. The occlusion and truncation both repre-
k
have a parameter θx|u for each combination of xk ∈ V al(X) senting the visibility state of an object is defined analogously
and uk ∈ V al(U) for a CBT. The likelihood function for such to the KITTI benchmark [13]. Weather conditions may effect
case is as follows. the road conditions and light intensity that in turn can effect
k
Y Y Y M k [u,x] reflection on road. Especially heavy rain may cause flooding
LX (θX|U : Dk ) = k
θx[m]u[m] = θk x|u on road, which in turn can decrease the TP in detections [14].
m u∈V al(U) x∈V al(X)
(5) We use FN and TP rate to represent the SOTIF measure as
k
Here θx|u represents the parameter to be learned, k represents they are considered adequate measures for SOTIF analysis [2].
the kth BN around the HAD vehicle and m represents the
mth data instance in the dataset. Maximizing the likelihood B. Model of the Causal Relation
function from Eq. 5 results in the learned parameter.
Based on the propositions from the previous section, the BN
k M k [u, x] structure is developed. The effects discussed in the previous
θx|u = (6)
M k [u] section can be encoded in the following simple propositions.
Here M k [u, x] represents the combined occurrence of u and a) Proposition 1: Truncation and occlusion in detection
x for the k th BN. Eq. 6 defines the MLE. may influence FN and TP.
E. Refinement b) Proposition 2: Weather conditions may effect road
conditions and scene illumination, which in turn can effect
The aim of refinement steps is to improve the BN (both
the TP/FN rate.
structure and CBTs), so that exhaustive and complete models
for SOTIF can be produced. We believe that this a hybrid c) Proposition 3: Road condition and scene illumina-
approach (involving experts while partially automating the tion can effect reflection in the scene, which in turn can effect
approach) may provide the most suitable results. Every step the TP/FN rate.
explained in the previous sections and depicted in the Fig. 2 The resulting BN structure is shown in Fig. 3. The BN model
is subject to iterative refinement based on the analyses and contains seven nodes. Once the CBTs are established, the BN
obtained results. This includes additions/deletion of scenario can be updated with new information.
factors, restructuring of the BN structure or acquisition of more
data. False Negative/ True Positive
Yes
III. E XPERIMENTAL S ETUP No

The experimental setup consists of two Hesai Pandar 64 and Occlusion

two Velodyne Ultra Puck VLP-32C LIDAR sensors installed Truncation Reflection Fully visible (0)
Yes (0) Yes Partly occluded (1)
on the roof corners of a car (Fig. 1). The recorded data consists No (1) No Largely occluded (2)
of different labels including bounding boxes, pose, visibility Unknown (3)
state and vehicle activity among others surrounding 360° of the Illumination
Road
Low light
HAD vehicle. The data was collected mostly on the highway Day
Wet
Dry
in the nearby regions of Stuttgart, Germany. The data consists Tunnel light
of around twenty thousand instances. A deep neural network Weather
(DNN) was trained and used as the processing algorithm. Cloudy
Two experts from the field with substantial experience in the Sunny
Rainy
LIDAR based perception systems provided their opinions on Clear
LIDAR insufficiencies, triggering conditions and limitations Not Defined
based on the observations in the data acquisition process and Fig. 3: BN based on the SOTIF relevant scenario factors
experiences with the LIDAR based perception systems. and expert knowledge describing the causal structure used
IV. I MPLEMENTATION in our implementation. False negative and true positive are
selected alternatively. Color coding is provided to support the
In this section, we demonstrate the application of our
subsequent figures of the analysis.
methodology on the LIDAR sensing dataset discussed in the
previous section.
C. Data Acquisition and Pre-processing
The dataset we use in this paper provides detection and
corresponding ground truth (separate datasets). One Bounding
box is labeled for each object detection and its corresponding
ground truth. All relevant nodes (Fig. 3) are labeled except TP
and FN. In order to evaluate TP and FN for each data instance
we use mean squared error (MSE).
n
1X
M SE = (Yi − Ŷi )2 (7)
n i=0
Where n represents number of samples, Yi represents the
ground truth and Ŷi represents the detection. We execute Eq. 7 False Negative
for individual detections and find a corresponding sample in
Yes
the ground truth using x and y values. All those data instances No
from detection which return a data instance from ground truth
are considered TP, while all those data points from ground Fig. 4: Performance limitation map for FN rate in the described
truth that do not return a corresponding value from detection scene and available data used for learning. Better performance
are considered to be FN. Data instances with |x| > 140 meters of LIDAR is be observed near the HAD vehicle.
and |y| > 50 meters are not considered as the defined optimal
range of LIDAR.
Resolution of grid cells in the grid-map is an interesting
aspect as it directly influences the TP and FN rate. This phe-
nomenon is analogous to discretization of continuous spatial
distribution as a BN node [15]. As coarsening may result is
less precise and accurate while refinement may result in pre-
cise and less accurate CBTs [16], a well thought discretization
is required. Both, static and dynamic discretization can be
performed in this regard [16]. Based on the availability of
data for each cell and complete representation of all the nodes
of the BN structure (Fig. 3), we use x = 20 and y = 10 meters
accordingly, in this publication.
True Positive
D. Parameter Learning
Yes
We perform parameter learning for individual BN (repre- No
senting a grid cell) using its corresponding data instances and
Fig. 5: Performance limitation map for TP in the described
Eq. 6. After the establishment of the BN structure and learning
scene. Better performance of LIDAR is observed near the
of the distribution parameters (CBTs), the BN can be used
HAD vehicle.
as an effective tool for analysis and estimation. The resultant
PLMs and CPLMs can be used as metrics to assess the SOTIF,
identify triggering conditions as well as provide safety cases 1) We observe better detection capabilities near the HAD
and validation targets. vehicle.
E. Refinement 2) Both TP and FN rate are symmetrically distributed across
X and Y axes with slightly higher FN rate (and lower
We provide preliminary refinement steps in the results
TP rate) in front and on the right side of the HAD vehicle.
(Sec.V).
By using the PLMs (Fig. 4, 5), the uncertainty of the scene
V. R ESULTS can be represented with a PLM, which can be expressed as a
In this section, we present the results obtained by applying quantitative evaluation of the safety of intended functionality
our methodology. (SOTIF) for a given scene and system under consideration.

A. Performance Limitation Map B. Conditional Performance Limitation Map

Grid maps pertaining to TP and FN are shown in Fig. 4, 5. Another interesting analysis result comes in the form of
Essentially PLMs represent the marginalized posterior proba- a CPLM, conditioned on individual or multiple nodes of
bility distributions (Pr(ch)) of specific nodes. The heat map the scene. This corresponds to conditional probability of a
reference is reversed for TP to keep the same color for unde- child (ch) node given its parent(s) (pa) node (Pr(ch | pa)).
sired probabilities. We can infer the following conclusions. The pa can be selected individually or in combination. In
 Occlusion Illumination False Negative
Fully visible (0) False Negative Low light
Partly occluded (1) Day Yes
Largely occluded (2) Yes Tunnel light No
Unknown (3) No
Fig. 8: Causal inference map for FN when illumination (day).
Fig. 6: Conditional performance limitation map (CPLM) for Illumination state ”day” has varied effects on the detection.
FN (Yes) conditioned on Occlusion (fully visible) in the
described scene. CPLM for FN describes that occlusion (fully
visible) scenes may not causes higher FN rate.

Occlusion
Fully visible (0) False Negative
Partly occluded (1)
Largely occluded (2) Yes
Unknown (3) No
Occlusion
Fully visible (0) False Negative Fig. 9: Causal inference map for FN when occlusion (largely
Partly occluded (1) occluded). Largely occluded states have considerable effects
Largely occluded (2) Yes
Unknown (3) No on the detections away from the HAD vehicle.
Fig. 7: Conditional performance limitation map (CPLM) for
FN conditioned on Occlusion (largely occluded) in the de-
C. Causal Inference
scribed scene. CPLM for FN describes a higher FN rate for
occlusion (largely occluded) scenes. The strength of BN to provide backward propagation of
evidence [4] provides an added advantage of performing
causal inference. In other words, casual inference estimates
the strength of a parent node on the child node or any other
the light of ISO/PAS 21448, it can be seen as how trig-
node in the structure(Pr(pa | ch or any)). For example, given
gering conditions influence the performance [2]. We provide
the BN, consider the following query.
CPLMs of FN conditioned on occlusion (Fig. 6,7). Evidently,
a) Query: What causes the FN rate?
occlusion=largely occluded scenes have higher probabilities
This query can be answered by setting the FN in the
of FNs than occlusion=f ully visible scenes. We can infer
grid map to Y es. Fig. 8,9 shows the causal inference of
the following conclusions.
Pr(Illumination= Day | F N = Y es) and Pr(Occlusion =
1) Largely occluded scenes have higher FN rate than fully largely occluded | F N = Y es) respectively. We can infer the
visible scenes for LIDARs, given the data. following conclusions.
2) The average Pr(F N | Occlusion) rate is symmetrically 1) Occlusion = largely occluded has higher impact than
distributed across X and Y axes with slightly higher FN illumination = day on FN.
rate in front and on the right side of the HAD vehicle.
Such results may directly indicate the relevant triggering
TABLE I: Evaluation of FN rate when new evidence arrives. oriented or expert elicited CBTs. It may happen that not all the
Only weather, occlusion, road and reflection nodes are consid- influencing factors are encoded in the BN structure and data
ered evidence nodes. Instead of grid map, overall prediction does not represent the true relative frequency of phenomena.
rate is calculated. Dataset that does not well represent the open context may
result in error prone PLMs and CPLMs.
Evidence FN (Yes) accuracy
Weather 75.4102%
B. Rare Event Problem
Occlusion 71.5815%
Road 76.1166% This concerns the well-known rare event occurrence fre-
Reflection 76.3445% quency problem and its representation. This problem arises
when there are important states of nodes which occur with
lower frequency e.g. illumination : tunnel light is expected
conditions of performance limitations and may provide a way to occur with lower frequency than illumination : day. From
forward for informed improvement in the design of the system the SOTIF standpoint, these states can also be safety critical.
from the SOTIF viewpoint. For example, based on the CPLM Evaluating robust CPLM for such states becomes challenging
(Fig. 7) and a benchmark for FN rate for each cell (given and is subject to perturbations. Such states can be artificially
a constant severity and controllability), we may infer that inserted in the data but the resultant marginalized probabilities
largely occluded scenes are risk factors to SOTIF. However, will not be the true representation of the real world.
defining a benchmark is out of the scope of this work. In this regard, a relative representation of each state frequency
in the data explicitly modeled in the results can be a promising
D. Refinement direction.
Some of the refinement steps proposed by the experts are.
• More data is required for truncation node in order to
C. Training and Test Data
establish or negate a causal relation. Test dataset may inappropriately be segregated from the
• Abrupt zero values in regions where surrounding grid training data. Generally, test dataset should not be correlated
cells have relatively higher values (Fig. 6) are observed with training dataset. However, in reality, highly correlated
for occlusion (f ully visible). These cells require further dataset is used because data is recorded at the same locations
analysis and data instances for robust results. and it is recorded sequentially. This may lead to overestimated
The refinement steps are non-exhaustive and provision of an accuracy of the PLM.
exhaustive list of steps is out of the scope of this work.
D. Data Abstraction and ODD Taxonomy
VI. E VALUATION Every scene is defined based on some abstraction. This
We perform evaluation of our learned PLM using the test is analogous to the data discretization problem in BN [15].
dataset. We predict the FN by essentially setting the weather, Different abstractions may result in different maps e.g. a
occlusion, road and reflection states from the test dataset as lower and more specific abstraction of illumination node
evidence and predicting the FN. The results are then compared will be the values of light intensities instead of states such as
with the FNs computed by using Eq. 7. Tab. I shows the day, a further lower abstraction might be taking a continuous
results of our evaluation. We observe that substantial accuracy light intensities distribution. Such distribution may result in
in the results can be achieved. However, like any other different maps hence challenging the robustness of the results.
data oriented implementation, measuring the true underlying Since these abstractions can be governed by operational design
parameter distribution (or CBTs) is a challenging task. In domain (ODD) taxonomies, a well-established ODD taxonomy
general, a parameter learning algorithm for BN extracts the can be used as the benchmark for data abstraction for analyses.
joint relative frequency if they have conditional relation in Moreover, dynamic discretization can also be used in this
their structure e.g. X | Y [4]. As the real CBTs are unknown, regard [16].
the method approximates it using the dataset D. The resulting
VII. R ELATED W ORK
CBTs represent the characteristics of real and unknown CBTs.
Special care must be taken for tasks that are safety critical in In recent years, extensive research has been done on the
nature. In the following, we discuss some of the assumptions topic of SOTIF and scenario based safety of HAD vehi-
that the parameter learning of BNs are based on which may cles [17]. However, to the best of authors’ knowledge, existing
challenge the robustness of the results. approaches lack in the systematic identification, modeling,
quantification and analysis of SOTIF relevant scenario factors.
A. Representation of the Open Context Berk et al. [18] formalize the reliability-based validation of the
The first and foremost assumption taken in any model is that environment perception for safe automated driving and discuss
it is considered as a good approximation of open context. In the associated challenges. The work focuses on the perception
the specific case of BN, the structure represents the causal failure rate λper and discusses the false negative (FN) and
model and the CBT represents the relative occurrence as false positive (FP) as uncertainties. The implementation also
the approximation of open world phenomena for both data provides qualitative and semi-quantitative analyses of sensor
perception reliability. Ali et al. [19] analyze the hazards arising triggering conditions. The implementation can also be ex-
due to variabilities in collaborative cyber physical systems tended to test perception system based on same sensors with
(CPSs). Environmental, infrastructural, spatial and temporal different governing algorithms.
variabilities are considered as factors causing uncertainties.
R EFERENCES
They also develop a fault traceability graph to trace the faults
considered by multiple hazard analyses in the collaborative [1] S. Burton, I. Habli, T. Lawton, J. McDermid, P. Morgan, and Z. Porter,
“Mind the gaps: Assuring the safety of autonomous systems from an
CPSs with variability. Edward Schwalb [20] provides a prob- engineering, ethical, and legal perspective,” Artificial Intelligence, vol.
abilistic framework for incrementally bounding the residual 279, p. 103201, 2020.
risk associated with autonomous drivers and enabling the [2] International Organization for Standardization, “ISO/PAS 21448:2019
Road vehicles – Safety of the intended functionality,” p. 54, 2019.
quantifying progress. The work introduces continuous moni- [3] A. Adee, P. Munk, R. Gansch, and P. Liggesmeyer, “Uncertainty Rep-
toring by autonomous driver for imminent hazards and selects resentation with Extended Evidential Networks for Modeling Safety of
actions that maximizes the time to materialization (TTM) the Intended Functionality (SOTIF),” in European Safety and Reliability
Conference (ESREL2020), 2020, pp. 4148–4156.
of these hazards. The approach also enables implementing [4] D. Koller and N. Friedman, Probabilistic graphical models: principles
the continuous expansion of SOTIF through measurement of and techniques. MIT press, 2009.
improvements from regressions using posterior probabilities. [5] J. Pearl, Probabilistic reasoning in intelligent systems: networks of
plausible inference. Elsevier, 2014.
Finally, Kramer et al. [21] provide integrated method for safety [6] P. Weber, G. Medina-Oliva, C. Simon, and B. Iung, “Overview on
assessment of automated driving functions, which covers the bayesian networks applications for dependability, risk analysis and
aspects of functional safety and SOTIF, including identification maintenance areas,” Engineering Applications of Artificial Intelligence,
vol. 25, no. 4, pp. 671–682, 2012.
and quantification of hazardous scenarios. They also provide a [7] B. Cai, X. Kong, Y. Liu, J. Lin, X. Yuan, H. Xu, and R. Ji, “Application
functional insufficiency and causal chain analysis technique to of bayesian networks in reliability evaluation,” IEEE Transactions on
identify and model SOTIF related hazards. Similar methodol- Industrial Informatics, vol. 15, no. 4, pp. 2146–2157, 2018.
[8] International Organization for Standardization, ISO 26262-1:2011 Road
ogy is also presented in another literature [22]. However, the vehicles Functional safety, 2011.
work provides a more theoretical view of the problem. [9] M. J. Flores, A. E. Nicholson, A. Brunskill, K. B. Korb, and S. Mascaro,
“Incorporating expert knowledge when learning bayesian network struc-
ture: a medical case study,” Artificial intelligence in medicine, vol. 53,
VIII. C ONCLUSION AND F UTURE W ORK no. 3, pp. 181–204, 2011.
[10] M. Scanagatta, A. Salmerón, and F. Stella, “A survey on bayesian
We presented a method to develop performance limitation network structure learning from data,” Progress in Artificial Intelligence,
maps (PLMs) as well as conditional performance limitations vol. 8, no. 4, pp. 425–439, 2019.
[11] R. Kabli, F. Herrmann, and J. McCall, “A chain-model genetic algorithm
maps (CPLMs) under the scene model to study safety of the for bayesian network structure learning,” in Proceedings of the 9th
intended functionality (SOTIF). We identify the relevant trig- annual conference on genetic and evolutionary computation, 2007, pp.
gering conditions, which are provided by experts and reasoned 1264–1271.
[12] R. Gansch and A. Adee, “System theoretic view on uncertainties,” in
through data. The methodology encodes the parameter learning 2020 Design, Automation Test in Europe Conference Exhibition (DATE),
for Bayesian network (BN) for the implementation. 2020, pp. 1345–1350.
This methodology particularly argues SOTIF under man- [13] J. Fritsch, T. Kuehnl, and A. Geiger, “A new performance measure and
evaluation benchmark for road detection algorithms,” in International
ageable effort. In its core, the provided methodology enables Conference on Intelligent Transportation Systems (ITSC), 2013.
the analyst to identify performance limitations under various [14] C. Goodin, D. Carruth, M. Doude, and C. Hudson, “Predicting the
triggering conditions, their causal relations and limitations, Influence of Rain on LIDAR in ADAS,” Electronics, vol. 8, no. 1, p. 89,
2019.
conditioned on various phenomena critical under SOTIF. This [15] N. X. Vinh, M. Chetty, R. Coppel, and P. P. Wangikar, “Data dis-
further assists the analyst to establish mitigation strategies cretization for dynamic bayesian network based modeling of genetic
for the identified performance limitation under triggering networks,” in International Conference on Neural Information Process-
ing. Springer, 2012, pp. 298–306.
conditions. In order to argue the adequacy of the approach, [16] N. Fenton and M. Neil, Risk assessment and decision analysis with
LIDAR performance was studied given a scene. The scene Bayesian networks. Crc Press, 2018.
was modeled using a BN structure and parameter learning [17] S. Riedmaier, T. Ponn, D. Ludwig, B. Schick, and F. Diermeyer,
“Survey on scenario-based safety assessment of automated vehicles,”
was performed using real world data to elicit conditional belief IEEE Access, vol. 8, pp. 87 456–87 477, 2020.
tables (CBTs). [18] M. Berk, O. Schubert, H.-M. Kroll, B. Buschardt, and D. Straub,
“Assessing the safety of environment perception in automated driving
We also evaluated the accuracy of learned BNs to demon- vehicles,” apr 2020.
strate the predictive capabilities. We achieved roughly 75% [19] N. Ali, M. Hussain, and J. E. Hong, “Analyzing safety of collaborative
when predicting the FN rate on the training data. We then cyber-physical systems considering variability,” IEEE Access, vol. 8, pp.
162 701–162 713, 2020.
discussed the robustness concerns of the safety methods in [20] E. Schwalb, “Analysis of Safety of The Intended Use (SOTIF),” 2019.
particular when data is used for parameter learning. [21] B. Kramer, C. Neurohr, M. Büker, E. Böde, M. Fränzle, and W. Damm,
In future, we intend to explore how the robustness concerns “Identification and quantification of hazardous scenarios for automated
driving,” in Model-Based Safety and Assessment, M. Zeller and K. Höfig,
of BNs can be addressed and mitigated. We particularly intend Eds. Cham: Springer International Publishing, 2020, pp. 163–178.
to provide methods focused on uncertainty measures and [22] C. Neurohr, L. Westhofen, M. Butz, M. H. Bollmann, U. Eberle, and
confidence intervals for CBTs and probabilities. Moreover, R. Galbas, “Criticality analysis for the verification and validation of
automated vehicles,” IEEE Access, vol. 9, pp. 18 016–18 041, 2021.
we also intend to model combined CPLMs for heterogeneous
perception systems in order to identify and analyze common

View publication stats