Sensitivity Analysis of

Ordinary Differential Equation Models

Methods by Morris and Sobol’ and Application in R

Frank Weber, Stefan Theers, Dirk Surmann, Uwe Ligges, and Claus Weihs

May 23, 2018

Contents

1 Introduction 2

2 Sensitivity Analysis Methods 3

2.1 Morris Screening – A Derivative-Based Method . . . . . . . . . . . . . . . . . . . 3
2.2 Sobol’ Sensitivity Analysis – A Variance-Based Method . . . . . . . . . . . . . . 5

3 The R Package sensitivity 7

3.1 Morris Screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Sobol’ Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 The R Package ODEsensitivity 17

4.1 Example Case 1: The Lotka-Volterra Equations . . . . . . . . . . . . . . . . . . . 18
4.2 Example Case 2: Low Frequency Oscillations . . . . . . . . . . . . . . . . . . . . 19
4.3 Morris Screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.3.1 General ODE Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.3.2 Low Frequency Oscillations . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.4 Sobol’ Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4.1 General ODE Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4.2 Low Frequency Oscillations . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Conclusion 31
5.1 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

References 32

1
Sensitivity Analysis of ODE Models 1 Introduction

1 Introduction

The goal of sensitivity analysis is to examine how sensitive a mathematical model responds
to variations in its input variables (Confalonieri et al., 2010). This incorporates, according to
Confalonieri et al. (2010), the identification of relevant model inputs, model balance, model
simplification and general model building. Two major approaches to sensitivity analysis can be
distinguished: local and global sensitivity analysis. Local sensitivity analysis investigates the
model response when only one parameter is varied when holding all other parameters at constant
central values. Global sensitivity analysis investigates the model response when every existing
parameter in the model is varied (see Saltelli et al., 2010 or Confalonieri et al., 2010). Although
local methods are easier to implement, today global sensitivity analysis techniques are more
common as their results do not depend on central values. Confalonieri et al. (2010) list three
classes of global sensitivity analysis techniques: regression, screening and variance-based methods.
The most well-known representatives of the two latter ones will be treated here.

This report will focus on the sensitivity analysis of ordinary differential equation (ODE) models
since they can be used to model so-called Low Frequency Oscillations (LFOs). LFOs are permanent
complex valued voltage oscillations with a frequency of up to 2 Hz occurring in electrical systems
like the European electrical transmission system. As an energy network is an “electro-mechanical
system” with power produced and consumed by mechanical systems, Surmann et al. (2014) point
out that a mechanical system of connected harmonic oscillators is suitable for modeling LFOs.
Mathematically, this model is based on a system of ordinary differential equations.

In R (a software environment for statistical computing by R Core Team, 2016) sensitivity analyses
(in a general manner) can be performed using the package sensitivity (Pujol et al., 2016). Specifically
for the sensitivity analysis of ODE models, the package ODEsensitivity (Theers et al., 2016) has
been created. It also provides support for the package ODEnetwork (Surmann, 2015) to simplify
the sensitivity analysis of ODE networks that are used to simulate LFOs. ODEsensitivity mainly
relies on functions implemented in sensitivity with adaptions to the special needs when analyzing
ODE models.

First, two sensitivity analysis methods (one screening and one variance-based method) are described
in general in section 2. In section 3, the key functions of the sensitivity package are presented, with
a focus on the changes that were necessary. The package ODEsensitivity is described in section 4.
Section 5 contains a conclusion and an outlook.

2
Sensitivity Analysis of ODE Models 2 Sensitivity Analysis Methods

2 Sensitivity Analysis Methods

2.1 Morris Screening – A Derivative-Based Method

Screening methods are probably the easiest way to analyze the sensitivity of a model function
f: Rk → R to variations of the k inputs xi ∈ R (with i = 1, . . . , k). The following assumptions
by Morris (1991) will be made here, too:

1. f is at least once differentiable.

2. For given inputs xi , the model function f evaluates without any stochastic errors (making f
uniquely determined).

3. Each input xi is a realization of a random variable Xi that is uniformly distributed on

[0, 1], in short: Xi ∼ U (0, 1). Thus, the vector x := (x1 , . . . , xk )T is always a point in the
k-dimensional unit hypercube Ω := [0, 1]k . For violations of this assumption, see the note at
the end of this subsection.
∂f
Since f is at least once differentiable, partial deviations ∂xi (x) can be analyzed to estimate the
sensitivity of f intuitively. See Table 1 for further details concerning the interpretation.

Table 1: Motivation of Morris’s elementary effects (cf. Morris, 1991).

∂f
∂xi (x) effect of xi on f
= 0 negligible
= const. 6= 0 linear & additive
= g(xi ) nonconst. nonlinear
= g(xj1 , xj2 , . . .) nonconst. with j1 , j2 , . . . 6= i interactions with other inputs

This makes Morris (1991) formulate his “discretized approach”: Consider a regular k-dimensional
p-level grid (p ∈ N>0) with
1 2
 k
x ∈ 0, , ,...,1 =: Ω̃ ⊆ Ω.
p−1 p−1

Then Morris’s elementary effects can be defined directly in the form of partial derivatives, see
Definition 1:

Definition 1. Morris’s Elementary Effects.

The elementary effect of the i-th input variable xi on f is defined as

f (x + ∆ ei ) − f (x)
di (x) := ∼ Fi (1)
∆

3
Sensitivity Analysis of ODE Models 2 Sensitivity Analysis Methods

Rk the i-th unit vector and Fi the discrete distribution of

n o
(p−1)−1
with ∆ ∈ 1 2
p−1 , p−1 , . . . , p−1 , ei ∈
the di (x), i = 1, . . . , k.

In order to generate a random sample of those elementary effects, Morris (1991, section 3) proposes
an economical design by constructing r “trajectories” in Ω̃, each one consisting of k + 1 points
(including the random starting point). This design is a so-called one-factor-at-a-time (OAT, in
the design of experiments also abbreviated as OFAT) design because for every trajectory, each
of the k directions is altered once and thus, “two successive points differ by one factor only”
(Pujol, 2009, p. 3). This means that every trajectory delivers one elementary effect for each input
xi . Consequently, this approach requires r · (k + 1) model evaluations. The typical number of
trajectories is around r ∈ [10, 50], but for ODE models, we recommend a much higher value for r
(see subsection 4.3). Campolongo et al. (2007) proposed a space-filling optimization of Morris’s
OAT design: They adduce that the trajectories sampled in Morris’s approach might not cover the
region of interest Ω optimally. For this reason, they generate a lot more trajectories, for example
r∗ ∈ [500, 1000], and finally select those r trajectories with a maximum dispersion. This requires
an appropriate measure of distance between two trajectories:

Definition 2. Distance Measure d(m, l).

The distance between two trajectories m and l is defined as

0,

m=l
d(m, l) := r if (2)
(l,t) 2
 
 k+1 k+1 (m,s)
m 6= l
Pk
−
P P

s=1 t=1 i=1 xi xi ,

(m,s)
with xi the i-th component (i.e. input factor) of the s-th point in Ω of the m-th trajectory.

Note that the use of the Euclidean metric in Definition 2 is arbitrary. Putting the selection process
more precisely, the user sets an r ∈ N and looks out for the combination of r trajectories with a
maximum sum of squared d(m, l) values (i.e. a brute force search).

Another kind of sampling design was developed by Pujol (2009) and is based on simplexes.
However, this design type is out of scope of this report, so please refer to the original article by
Pujol (2009) for more details. We just note here that for the simplex-based design, the model is
assumed to be “linear without interactions at the scale of a simplex” (Pujol, 2009, p. 8). When
using the R package sensitivity, the size of the simplexes can be determined by the user, though.
Both sampling design types (Morris’s OAT design and Pujol’s simplex-based design) are supported
by the sensitivity package and can be combined with the space-filling optimization by Campolongo
et al. (2007).

4
Sensitivity Analysis of ODE Models 2 Sensitivity Analysis Methods

The actual sensitivity measures summarize the discrete distribution Fi from equation (1): Let
µi and σi denote estimates for the expected value and the standard deviation of Fi , respectively.
Then, µi is a measure for the overall influence of input xi on the output and σi reflects the linearity
of the influence (Fruth, 2015, p. 20). As can be seen in the third and fourth case in Table 1, “A
value [of σi ] close to 0 suggests linear behavior, a high value nonlinear or interaction behavior”
(Fruth, 2015, p. 20). Additionally to µi and σi , Campolongo et al. (2007) propose another measure
of influence for non-monotonic objective functions because a problem called “effects of opposite
signs” might occur for such functions. If we consider an input variable for which the objective
function is not monotone, the corresponding elementary effects might have opposite signs. This
results in µi being close to zero even though xi might have a large impact on f . Campolongo
et al. (2007) use the absolute values of the elementary effects to resolve this problem of “effects of
opposite signs”. Let the distribution of the absolute values of the elementary effects be named
Gi with an estimated expected value µ∗i . Then, using µ∗i instead of µi is a better measure for
the “overall influence” of xi on f , as shown in section 3.1. σi is used unrevisedly as an index for
the higher order effects of xi . In the following, µi , µ∗i and σi will be called “Morris sensitivity
indices”.

Usually the domain of the input variables are different intervals than [0, 1]. The derived trajectories
for the k-dimensional unit hypercube Ω = [0, 1]k need to be scaled to the real domains of the input
variables to evaluate the model function at points where it is well-defined. Scaling to the unit
hypercube affects the calculation of the elementary effects and has to be decided case by case (see
subsection 3.1 for a more detailed explanation). If the assumption of a uniform distribution on the
domain intervals doesn’t hold, the Morris screening method cannot be used and the variance-based
Sobol’ method should be considered instead (see subsection 2.2).

2.2 Sobol’ Sensitivity Analysis – A Variance-Based Method

An alternative to the quite intuitive screening methods are variance-based methods with their
key concept to partition the model output’s variance like in an ANOVA. In particular, the
Sobol’ approach (Sobol’, 1990) enjoys great popularity today and serves as a benchmark for
other sensitivity analysis methods. As stated in Fruth (2015), Sobol’ indices offer an access to
interactions and can be interpreted clearly.

The Sobol’ sensitivity measures are based on the functional ANOVA decomposition (see Theorem 1,
adapted from Fruth, 2015). Define xI for an index set I ⊆ {1, . . . , k} as the vector of all xi with
i ∈ I (and still xi ∈ R).

5
Sensitivity Analysis of ODE Models 2 Sensitivity Analysis Methods

Theorem 1. Functional ANOVA Decomposition.

Let x := (x1 , . . . , xk )T be a k-dimensional random vector with independent components and
f: Rk → R, f ∈ L2 with L2 the space of square integrable functions. Then f holds
k
f (x) = f∅ + f{i} (xi ) + f{i,j} (xi , xj ) + . . . + f{1,...,k} (x1 , . . . , xk ) (3)
X X

i=1 i<j

with the 2k summands

f∅ := E f (x) ,

f{i} (xi ) := E f (x)|xi − f∅ ,

h i
f{i,j} (xi , xj ) := E f (x)|xi , xj − f{i} (xi ) + f{j} (xj ) + f∅ ,

..
. .

The decomposition in equation (3) is unique if E fI (xI ) = 0 for every index set I ⊆ {1, . . . , k}

(I 6= ∅) and if the “non-simplification condition” is fulfilled

E fI (xI )|xJ = 0


for every J ⊂ I ⊆ {1, . . . , k}.

In this case, the summands in equation (3) are uncorrelated and

d
D := Var f (x) = Var f∅ + Var f{i} (xi ) + Var f{i,j} (xi , xj )


X
X

i=1 i<j

+ . . . + Var f{1,...,k} (x1 , . . . , xk ) .

The term f∅ is a constant, f{i} (xi ) is called the i-th first order effect and f{i,j} (xi , xj ) the second
order interaction between xi and xj . Higher order interactions are defined analogously using other
index sets I ⊆ {1, . . . , k}.

On the basis of Theorem 1, we define four sensitivity indices.

Definition 3. Sobol’ Sensitivity Indices.

For a subset I ⊆ {1, . . . , k} the value

i) DI := Var fI (XI ) is called unscaled Sobol’ sensitivity index.

ii) SI := DI
D is called scaled Sobol’ sensitivity index.

iii) DIT := is called unscaled total sensitivity index.

P
J∩I6=∅ DJ

6
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

DIT
iv) SIT := D is called scaled total sensitivity index.

D{i}
D{i} reflects the first order effect of the i-th input variable. Hence, S{i} = D is called first order
Sobol’ sensitivity index. We omit the term scaled since only scaled indices are usually considered
in application. If the index set I in DI contains more than one element, DI measures the “pure
interaction influence of the variables indexed in I” (Fruth, 2015, p. 10). The total sensitivity index
DIT measures the “influence of the variables [indexed in I] including all interactions of any order
which contain at least one of [those variables]” (Fruth, 2015, p. 11), i.e. it reflects the individual
T
D{i}
influence of all factors in I plus every interaction effect. Thus, S{i}
T =
D measures the first order
effect of xi plus all interactions with xi involved. In short, T
S{i} is called total Sobol’ sensitivity
index and S{i} and T
S{i} in combination are called Sobol’ sensitivity indices.

Fruth (2015) in chapter 3.3.2, Saltelli et al. (2010) in chapter 8.3 or Sobol’ (2001) point out that
estimations of all these Sobol’ sensitivity indices can be obtained by Monte Carlo simulation.
Jansen (1999) as well as Martinez (2011) both developed methods for estimating the Sobol’
sensitivity indices. Both approaches, like several other techniques, are based on Monte Carlo
simulation and are implemented in the sensitivity package. This report focuses on the Jansen and
the Martinez methods since they are numerically stable and used in the package ODEsensitivity.
However, for all methods that are based on Monte Carlo estimation, the underlying distributions
of all k input variables need to be known to draw random samples from those distributions.

3 The R Package sensitivity

Both – Morris screening and the variance-based Sobol’ method – are implemented in the R package
sensitivity. The R version used for this report is 3.2.5. The goal of this section is to exemplarily
perform a sensitivity analysis for the so-called Sobol’ g-function. It is scalar-valued with eight
input variables and is one of the most common analytical test functions presented in Saltelli et al.
(2010, chap. 2). In addition, it is an example of a non-monotonic function for which the problem
of “effects of opposite signs” occurs (see subsection 2.1). The definition of the Sobol’ g-function
can be found in Saltelli et al. (2010, pp. 39-40):

8
|4xi − 2| + ai
f (x) := gi (xi ) with gi (xi ) := (4)
Y

i=1
1 + ai

and the coefficients a1 = 0, a2 = 1, a3 = 4.5, a4 = 9, a5 = a6 = a7 = a8 = 99. For the Sobol’

g-function, all input variables are uniformly distributed on [0, 1]. Thus, both presented methods
for sensitivity analysis – Morris screening and the Sobol’ variance-based method – can be applied
here. The Sobol’ g-function is defined in sensitivity as follows:

7
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

library("sensitivity")
[Link]

## function (X)
## {
## a <- c(0, 1, 4.5, 9, 99, 99, 99, 99)
## y <- 1
## for (j in 1:8) {
## y <- y * (abs(4 * X[, j] - 2) + a[j])/(1 + a[j])
## }
## y
## }
## <bytecode: 0x0000000009d93050>
## <environment: namespace:sensitivity>

[Link]() accepts a matrix X as input and returns a vector of function values at the eight-
dimensional points contained in the n rows of X. When running the sensitivity analysis, X contains
the n sampled combinations of the 8 input variables.

An optimal performance using sensitivity is achieved if the model function f : Rk → R is

implemented in R as a function accepting a matrix as input and returning a numeric vector.

When analyzing the sensitivity of an m-dimensional ODE model at q distinct timepoints, the
runtime can be decreased substantially when returning the function values of all m objective
functions at all q timepoints simultaneously. For each row of X, a vector of length q (for m = 1)
or a matrix of size q × m (for m > 1) is returned. The model function in R then returns a
matrix of size n × q (for m = 1) or a three-dimensional array of size n × q × m (for m > 1).
Prior to version 1.12.1, sensitivity could not deal with model functions returning a matrix or
a three-dimensional array. As of version 1.12.1, the functions morris(), soboljansen() and
sobolmartinez() support those model functions.

In order to demonstrate this new feature, we extend the result of [Link]() to a matrix with
two columns by appending its original result times 2:

sobol.fun_matrix <- function(X){

res_vector <- [Link](X)
cbind(res_vector, 2 * res_vector)
}

8
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

Analogously, a model function returning a three-dimensional array is created by doubling the results
from sobol.fun_matrix() once again and combining those two matrices to a three-dimensional
array of dimension lengths (n, 2, 2):

sobol.fun_array <- function(X){

res_vector <- [Link](X)
res_matrix <- cbind(res_vector, 2 * res_vector)
array(data = c(res_matrix, 2 * res_matrix),
dim = c(length(res_vector), 2, 2))
}

For Morris screening, the elementary effects and the sensitivity indices double when analyzing
the doubled results. In contrast, the Sobol’ sensitivity indices don’t change at all since they are
scaled.

3.1 Morris Screening

In sensitivity, Morris screening is called by using the morris() function. A seed is set in order to get
the same pseudo-random numbers when analyzing sobol.fun_array() and sobol.fun_matrix():

[Link](2849)
mor_matrix <- morris(model = sobol.fun_matrix, factors = 8, r = 50,
design = list(type = "oat", levels = 10, [Link] = 1),
binf = 0, bsup = 1, scale = FALSE)
[Link](2849)
mor_array <- morris(model = sobol.fun_array, factors = 8, r = 50,
design = list(type = "oat", levels = 10, [Link] = 1),
binf = 0, bsup = 1, scale = FALSE)

Argument model contains the model function to be analyzed (or alternatively a predict() method,
see the morris() help page of sensitivity). Argument factors contains the number of input
variables k (or a character string with the names of input variables) and r sets the number of
elementary effects to compute per input variable. The sampling design is specified using argument
design (which needs to be supplied as a list):

9
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

• If type = "oat" (as used here), Morris’s OAT design is used. In this case, parameter p
from subsection 2.1 has to be specified using argument levels in the list design. Argument
[Link] sets the number of levels that are vaulted in each step (in subsection 2.1,
[Link] equals 1).

• If type = "simplex", the simplex-based design by Pujol (2009) is used. In this case, the list
supplied for design contains a numeric argument [Link] which sets an expansion
factor for the simplexes (for [Link] = 1, all edges of the simplexes have length one).

For the space-filling optimization by Campolongo et al. (2007), which can be applied to both
sampling design types, a vector of length two has to be supplied for argument r: The first element
indicating the number of elementary effects, the second one the number r∗ of repetitions from
which the trajectories with maximum dispersion are chosen (see subsection 2.1).

Finally, three arguments (binf, bsup and scale) apply to both sampling designs. Arguments
binf and bsup specify the lower and upper boundaries for intervals on which input factors are
assumed to be uniformly distributed. The logical argument scale rescales the trajectory points
to [0, 1] after evaluating the model and before calculating the elementary effects. The meaning of
“impact of an input variable on the output variable” decides if scale = TRUE or scale = FALSE
should be used: Consider two input variables, one varying in a small range, the other one varying
in a large range. Let both cause the same absolute change of f across their ranges. Then, if scale
= TRUE, both input variables are considered to have the same impact on f . In contrast, if scale
= FALSE, the input variable varying in the small range is considered to have a larger impact on f
than the one varying in the large range.

The resulting objects mor_matrix and mor_array are both lists of class "morris" with the
following structures:

str(mor_matrix[c("X", "y", "ee")], [Link] = 1, [Link] = FALSE)

## List of 3
## $ X : num [1:450, 1:8] 0.111 ...
## $ y : num [1:450, 1:2] 0.961 ...
## $ ee: num [1:50, 1:8, 1:2] -2.42 ...

str(mor_array[c("X", "y", "ee")], [Link] = 1, [Link] = FALSE)

## List of 3
## $ X : num [1:450, 1:8] 0.111 ...

10
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

## $ y : num [1:450, 1:2, 1:2] 0.961 ...

## $ ee: num [1:50, 1:8, 1:2, 1:2] -2.42 ...

X is the matrix of the r · (k + 1) sampled points in the k-dimensional parameter space. In this
example, there are r · (k + 1) = 50 · 9 = 450 sampled points. Since we use the same seed,
mor_matrix and mor_array contain the same matrix X. Element y contains the corresponding
450 function values and ee the r = 50 elementary effects for all eight input variables and all
dimensions of y except the first. If we use mor_matrix, ee is a three-dimensional array with the
third dimension corresponding to the columns of y. For mor_array, ee is a four-dimensional
array with the third and fourth dimension corresponding to the second and third dimension of y,
respectively.

The Morris sensitivity indices µ∗i and σi (here for i = 1, . . . , 8) need to be calculated manually
from the elementary effects in element ee (the following code can also be found on the morris()
help page in sensitivity):

# For sobol.fun_matrix():
mu_matrix <- apply(mor_matrix$ee, 3, function(M){
apply(M, 2, mean)
})
mu.star_matrix <- apply(abs(mor_matrix$ee), 3, function(M){
apply(M, 2, mean)
})
sigma_matrix <- apply(mor_matrix$ee, 3, function(M){
apply(M, 2, sd)
})

# For sobol.fun_array():
mu_array <- sapply(1:dim(mor_array$ee)[4], function(i){
apply(mor_array$ee[, , , i, drop = FALSE], 3, function(M){
apply(M, 2, mean)
})
}, simplify = "array")
mu.star_array <- sapply(1:dim(mor_array$ee)[4], function(i){
apply(abs(mor_array$ee)[, , , i, drop = FALSE], 3, function(M){
apply(M, 2, mean)
})
}, simplify = "array")

11
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

sigma_array <- sapply(1:dim(mor_array$ee)[4], function(i){

apply(mor_array$ee[, , , i, drop = FALSE], 3, function(M){
apply(M, 2, sd)
})
}, simplify = "array")

If the model output is doubled, the elementary effects are doubled as well and so are the Morris
sensitivity indices:

# Elementary effects are doubled:

c([Link](2 * mor_matrix$ee[, , 1], mor_matrix$ee[, , 2]),
[Link](2 * mor_matrix$ee, mor_array$ee[, , , 2]))

## [1] TRUE TRUE

# Morris sensitivity indices are doubled:

c([Link](2 * mu_matrix[, 1], mu_matrix[, 2]),
[Link](2 * mu.star_matrix[, 1], mu.star_matrix[, 2]),
[Link](2 * sigma_array[, , 1], sigma_array[, , 2]))

## [1] TRUE TRUE TRUE

sensitivity provides a plot() method for objects of class "morris" to visualise the Morris sensi-
tivity indices. At first, consider a plot of the Morris sensitivity indices for the first column of
mor_matrix$y (the original [Link]()-results). Differences between µ and µ∗ are illustrated
in a plot of µi against σi (for i = 1, . . . , 8) for the same column of mor_matrix$y:

par(mfrow = c(1, 2), mar = c(5, 4, 2, 2) + 0.1, xpd = TRUE)

plot(mor_matrix)
plot(mu_matrix[, 1], sigma_matrix[, 1], xlab = expression(mu),
ylab = expression(sigma), pch = 19)
text(mu_matrix[, 1], sigma_matrix[, 1], labels = paste0("X", 1:8), pos = 4)

12
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

X1 X1
3

3
X2 X2
2

2
σ

σ
1

1
X3 X3

X4 X4

X6
X5
X8
X7 X6
X5
X8
X7
0

0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 −0.4 −0.2 0.0 0.2

µ *
µ

The µ∗ -plot on the left shows the large impact of x1 on f , followed by x2 , x3 and x4 (in this
order). The impact of all other input variables can’t be distinguished from another because they
are all nearly zero. Apart from that, the influence of x1 seems to be nonlinear or dominated by
interactions. The same applies lesser for x2 . An interpretation of µ is more difficult than that of
µ∗ if we study the same situation with a different seed:

[Link](2015)
mor_matrix2 <- morris(model = sobol.fun_matrix, factors = 8, r = 50,
design = list(type = "oat", levels = 10, [Link] = 1),
binf = 0, bsup = 1, scale = FALSE)
mu_matrix2 <- apply(mor_matrix2$ee, 3, function(M){
apply(M, 2, mean)
})
sigma_matrix2 <- apply(mor_matrix2$ee, 3, function(M){
apply(M, 2, sd)
})
par(mfrow = c(1, 2), mar = c(5, 4, 2, 2) + 0.1, xpd = TRUE)
plot(mor_matrix2)
plot(mu_matrix2[, 1], sigma_matrix2[, 1], xlab = expression(mu),
ylab = expression(sigma), pch = 19)
text(mu_matrix2[, 1], sigma_matrix2[, 1], labels = paste0("X", 1:8), pos = 4)

13
Sensitivity Analysis of ODE Models 3 The R Package sensitivity
4

4
X1 X1
3

3
2

2
σ

σ
X2 X2
1

1
X3 X3
X4 X4
X7
X6
X5
X8 X7
X6
X5
X8
0

0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 −0.4 −0.3 −0.2 −0.1 0.0 0.1

µ *
µ

Now, µ1 and µ2 have opposite signs and different absolute values whereas all µ∗i nearly remain
unchanged.

Two arguments were added to the [Link]() method in order to support matrices and
three-dimensional arrays as model outputs: The first one, y_col, sets the column index of y for
which the Morris sensitivity indices are plotted. The second one, y_dim3, does the same for the
third dimension of y (if there is any). The default for both arguments is 1, so the first column and
the first element of the third dimension is used (if existing). To demonstrate the use of y_col and
y_dim3, the sensitivity indices for the second column of the first and second element in the third
dimension of mor_array$y are plotted. The plots show the expected doubled Morris sensitivity
indices if the model output is doubled:

par(mfrow = c(1, 2), mar = c(5, 4, 2, 2) + 0.1, xpd = TRUE)

plot(mor_array, y_col = 2, y_dim3 = 1)
plot(mor_array, y_col = 2, y_dim3 = 2)

14
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

15
X1 X1
6

10
X2 X2
4
σ

5
2

X3 X3

X4 X4

X6
X5
X8
X7 X6
X5
X8
X7
0

0 1 2 3 4 5 6 0 0 2 4 6 8 10 12

µ *
µ *

3.2 Sobol’ Sensitivity Analysis

There are various implementations of the Sobol’ variance-based sensitivity analysis in the sensitiv-
ity package (e.g. sobol(), sobol2002(), sobol2007(), soboljansen() and sobolmartinez()).
The syntax of all these implementations is very similar and only differs in details. However,
only soboljansen() and sobolmartinez() have been adapted for the use with model functions
returning a matrix or a three-dimensional array since they are used in the ODEsensitivity package
for the sensitivity analysis of ODE models. The use of soboljansen() will be exemplarily
presented here.

At first, n random samples have to be drawn twice from the distribution of x = (x1 , . . . , xk )T for
the Monte Carlo estimation of the Sobol’ sensitivity indices. For the Sobol’ g-function, we can
assume a uniform distribution on [0, 1] for all xi ’s (i = 1, . . . , 8). Thus, the random samples can
be generated as follows:

[Link](9204)
n <- 1000
X1 <- [Link](matrix(runif(8 * n), nrow = n))
X2 <- [Link](matrix(runif(8 * n), nrow = n))

15
Sensitivity Analysis of ODE Models 3 The R Package sensitivity

Next, using these random samples X1 and X2, the sensitivity analysis for sobol.fun_matrix()
and sobol.fun_array() is performed by a call to soboljansen():

sob_matrix <- soboljansen(model = sobol.fun_matrix, X1, X2, nboot = 0)

sob_array <- soboljansen(model = sobol.fun_array, X1, X2, nboot = 0)

Note that bootstrapping is not recommended for ODE models because of high computational
costs, so argument nboot is set to 0.

The resulting objects (of class "soboljansen") include both the first order Sobol’ sensitivity
indices (S{i} from subsection 2.2):

str(sob_matrix$S, [Link] = FALSE); str(sob_array$S, [Link] = FALSE)

## num [1:8, 1:2] 0.71 0.1955 0.0616 0.0289 0.0193 ...

## num [1:8, 1:2, 1:2] 0.71 0.1955 0.0616 0.0289 0.0193 ...

and the total Sobol’ sensitivity indices (S{i}

T ):

str(sob_matrix$T, [Link] = FALSE); str(sob_array$T, [Link] = FALSE)

## num [1:8, 1:2] 0.755609 0.233964 0.035171 0.010676 0.000106 ...

## num [1:8, 1:2, 1:2] 0.755609 0.233964 0.035171 0.010676 0.000106 ...

The dimensions of these objects agree with the dimensions of the corresponding object y:
sob_matrix$S and sob_matrix$T are matrices with two columns containing the Sobol’ sen-
sitivity indices for the two columns of sob_matrix$y. sob_array$S and sob_array$T are three-
dimensional arrays containing the Sobol’ sensitivity indices for the second and third dimension of
sob_array$y.

As expected, the (scaled) Sobol’ sensitivity indices (both total and first order) don’t change at all
when analyzing the doubled results from [Link]():

c([Link](sob_matrix$S[, 1], sob_matrix$S[, 2]),

[Link](sob_matrix$T[, 1], sob_matrix$T[, 2]),
[Link](sob_array$S[, , 1], sob_array$S[, , 2]),
[Link](sob_array$T[, , 1], sob_array$T[, , 2]))

## [1] TRUE TRUE TRUE TRUE

16
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

The Sobol’ sensitivity indices can be visualized again by a simple call to plot(). The plot()
methods [Link]() and [Link]() also support the two new arguments
y_col and y_dim3. The use of [Link]() is demonstrated below:

par(mar = c(3, 2, 2, 2) + 0.1, xpd = TRUE)

plot(sob_array, y_col = 2, y_dim3 = 2)
1.0

main effect
total effect
0.8
0.6
0.4
0.2
0.0

X1 X2 X3 X4 X5 X6 X7 X8

The results from Sobol’ sensitivity analysis correspond to those from Morris screening: With
respect to both the first order and the total Sobol’ indices, x1 has the largest impact on f , followed
by x2 , x3 and x4 . The influence of all other input variables can’t be distinguished one from
another, they are all nearly zero. Note that it’s rather the total Sobol’ sensitivity index which
can be compared to µ∗ since they both measure the influence of an input variable and of all the
interactions this variable is involved in.

4 The R Package ODEsensitivity

The sensitivity analysis of ODE models is computationally difficult for two main reasons: Firstly,
a separate sensitivity analysis has to be performed for every timepoint of interest. Secondly, for
a multi-dimensional ODE model, different output variables (state variables in the terminology
of ODE models) have to be analyzed simultaneously. To ease the sensitivity analysis of ODE
models, the R package ODEsensitivity has been created.

Two techniques of sensitivity analysis are implemented in ODEsensitivity: Morris screening

and the Sobol’ sensitivity analysis. ODEmorris() and ODEsobol(), the two main functions

17
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

of ODEsensitivity, correspond to those two techniques. They rely on the functions morris(),
soboljansen() and sobolmartinez() from the sensitivity package. ODEmorris() and ODEsobol()
are generic functions and they both have two methods: one default method for analyzing general
ODE models and one method for analyzing ODE networks that are used for simulating low
frequency oscillations (LFOs). This is why two example cases will be presented in this section:
one for general ODE models (the Lotka-Volterra equations) and one for simulating LFOs.

4.1 Example Case 1: The Lotka-Volterra Equations

The Lotka-Volterra equations describe a predator and its prey’s population development and go
back to Lotka (1925) and Volterra (1926). The prey’s population at time t (in days) will be
denoted with P (t) and the predator’s (or rather consumer’s) population with C(t). P (t) and C(t)
are called state variables. This ODE model is two-dimensional, but it should be noted that ODE
models of arbitrary dimensions (including one-dimensional ODE models) can be analyzed with
ODEsensitivity. The Lotka-Volterra equations as presented in Soetaert et al. (2010) are:

. ∂ P (t)
 
P (t) := P (t) = rG P (t) 1 − − rI P (t) C(t)
∂t K (5)
. ∂
C(t) := C(t) = kAE rI P (t) C(t) − rM C(t).
∂t
In fact, this is a combination of the normal and the competitive Lotka-Volterra model since it
assumes an exponential reduction of the predator’s population in the absence of the prey, but a
logistic growth of the prey’s population in the absence of the predator. The k := 5 parameters
are the prey’s growth rate rG , the carrying capacity K, the consumer’s ingestion rate rI , the
consumer’s assimilation efficiency kAE and the consumer’s mortality rate rM (Soetaert et al.,
2010). rG , rI , rM and kAE are dimensionless and take on values in [0, 1]. The carrying capacity K
is of the same dimension as P (t) (and C(t)) and is assumed to take on values in [1, 20].

In R, the function ode() from the package deSolve (Soetaert et al., 2016) is able to calculate the
values of the state variables at different timepoints for given initial values and given parameter
settings. This is what is meant by “solving” an ODE system. For ode(), the model function has
to be supplied in a specific manner:

LVmod <- function(Time, State, Pars) {

with([Link](c(State, Pars)), {
Ingestion <- rIng * Prey * Predator
GrowthPrey <- rGrow * Prey * (1 - Prey/K)
MortPredator <- rMort * Predator

18
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

dPrey <- GrowthPrey - Ingestion

dPredator <- Ingestion * assEff - MortPredator

return(list(c(dPrey, dPredator)))
})
}

Each of the state variables P (t) and C(t) corresponds to the model function f from section 2.
The k = 5 parameters rG , rI , rM , kAE and K are considered as input variables for the sensitivity
analysis. Hence, we will analyze the sensitivity of the prey’s and the predator’s population with
regard to changes in these 5 parameters.

The parameters names, their lower and upper boundaries, the initial values for the state variables
and the timepoints of interest are saved in separate vectors. Here, we will use initial values of
1 for the prey’s population and 2 for the predator’s population. The time interval of interest is
(0, 50] (days) which is covered in discrete steps of one day.

LVpars <- c("rIng", "rGrow", "rMort", "assEff", "K")

LVbinf <- c(0.05, 0.05, 0.05, 0.05, 1)
LVbsup <- c(1.00, 3.00, 0.95, 0.95, 20)
LVinit <- c(Prey = 1, Predator = 2)
LVtimes <- c(0.01, seq(1, 50, by = 1))

Note that the vector of initial values (here LVinit) has to be a named vector and that the smallest
timepoint has to be strictly positive.

4.2 Example Case 2: Low Frequency Oscillations

For modelling low frequency oscillations in an energy network of n nodes, a system of n connected
harmonic oscillators can be used (Surmann et al., 2014). Each node of the energy network is then
represented by a mass mi (i ∈ {1, . . . , n}) connected to the ground by a damper di and a spring of
length ri . The spring constant of spring i will be denoted by ki . Masses i and j (i, j ∈ {1, . . . , n}
with i 6= j) are interconnected by springs of lengths ri,j and spring constants ki,j . The damper
between mass i and mass j has a damping constant denoted by di,j . The deflection xi (t) of mass i
at time t corresponds to the deflection of the voltage magnitude or angle (by choice) at node i in
the energy network at time t. Figure 1 illustrates the setting and the notation for a network of
n = 4 oscillators.

19
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

x1 (t) x2 (t) x3 (t) x4 (t)

k12 k23 k34

m1 m2 m3 m4

d12 d23 d34

k1 k2 k3 k4

d1 d2 d3 d4

Figure 1: Example of n = 4 interconnected mechanical oscillators, adapted from Surmann et al.

(2014). Note that the spring and the damper connecting mass 1 and mass 4 (with spring
constant k1,4 and damping constant d1,4 , respectively) are not plotted due to clarity.

The oscillation of the masses m1 , . . . , mn is then modelled by the following n-dimensional system
of ordinary differential equations:

.. .
0 = M x(t) + Dx(t) + Kx(t) + b (6)
.. −1 . −1 −1
⇔ 0 = x(t) + M | {z K} x(t) + M
| {z D} x(t) + M | {z b} . (7)
=:A =:B =:g

with

Rn ,
• x(t) := (x1 (t), . . . , xn (t))T ∈

• M := diag{m1 , . . . , mn } ∈ Rn×n ,

−di,j ,
 i 6= j
Rn×n

• D := (d∗i,j ) ∈ with di,j :=
∗ for i, j ∈ {1, . . . , n},
di + i=j
X

 di,j ,

j6=i

−ki,j ,
 i 6= j
Rn×n

• K := (ki,j
∗ )∈ with ki,j :=
∗ for i, j ∈ {1, . . . , n},
+ i=j
X

 ki ki,j ,

j6=i


Rn
ri,j , i<j
∗
• b∈ with bi := −ki ri + and ∗ :=
X
ki,j ri,j ri,j
j6=i
r
j,i = −ri,j , i > j.

The first and second derivatives x(t) ∈

.
Rn and x(t)
..
∈ Rn of x(t) with respect to time describe the
velocity and acceleration of all n masses, respectively. Equation (6) is transformed to equation (7)
since equation (6) is over-parametrized by the masses M (Surmann et al., 2014). Equation (7) (a

20
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

system of n ODEs of second order) is then transformed into a system of 2n ODEs of first order
.
using the substitution f (t) := (xT (t), xT (t))T ∈ R2n:
0n 0
" # " #
. In
f (t) = f (t) + . (8)
−B −A −g
| {z } | {z }
=:C =:h

As Surmann et al. (2014) describe in more detail, this ODE system can be solved analytically.
However, when simulating oscillations that are disturbed by external forces, the solution has to
be determined numerically (Surmann et al., 2014).

Here, we will exemplarily perform a sensitivity analysis for an n := 4-dimensional ODE network
that can be solved analytically. In this example network, the masses will be connected in a
“circle”, which means that mass i is connected to mass i + 1 (for i = 1, 2, 3) and mass 4 is
connected to mass 1. According to Surmann et al. (2014), we choose mi := 2 ∀ i ∈ {1, . . . , 4} and
ki := mi (2π · 0.17)2 ≈ 2.28 ∀ i ∈ {1, . . . , 4} to obtain an oscillation frequency of 0.17 Hz for each
mass. The spring constants for the interconnecting springs are set to k1,2 := k2,3 := k3,4 := k1,4 :=
k1
10 ≈ 0.228 and to zero in every other case (if ki,j = 0, the masses i and j are not connected).
The damping constants are set to di := 0.05 ∀ i ∈ {1, . . . , 4} and to di,j := 0 ∀ i, j ∈ {1, . . . , 4}
with i =
6 j. All lengths of the springs are set to zero (ri := 0, ri,j := 0 ∀ i, j ∈ {1, . . . , n}) to avoid
any ground potential for the springs. Next, an object of class "ODEnetwork" containing all the
information is created in R:

M_mat <- rep(2, 4)

K_mat <- diag(rep(2 * (2*pi*0.17)^2, 4))
K_mat[1, 2] <- K_mat[2, 3] <-
K_mat[3, 4] <- K_mat[1, 4] <- 2 * (2*pi*0.17)^2 / 10
D_mat <- diag(rep(0.05, 4))
library("ODEnetwork")
lfonet <- ODEnetwork(masses = M_mat, dampers = D_mat, springs = K_mat)

The state variables in this ODE model are the deflections x1 (t), . . . , x4 (t) and the velocities
. . .
v1 (t) := x1 (t), . . . , v4 (t) := x4 (t). The starting velocities are all set to zero: vi (0) := xi (0) :=
0 ∀ i ∈ {1, . . . , 4}. The oscillation is only initiated by the starting positions xi (0) := 2 ∀ i ∈
{1, . . . , 4} differing by two units from ri = 0 ∀ i ∈ {1, . . . , 4}. We create a vector of the parameter
names that should be involved in the sensitivity analysis. Those parameters are k1 , . . . , k4 and
d1 , . . . , d4 . The lower and upper boundaries for k1 , . . . , k4 are set to 0.2 and 20, respectively
(together with mi = 2, this gives oscillation frequencies in the (approximate) interval (0.05, 0.5)).

21
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

The boundaries for d1 , . . . , d4 are set to 0.01 and 0.1, respectively. The time interval of interest is
[25, 150] (seconds) and it is covered in steps of 2.5 seconds:

lfonet <- setState(lfonet, state1 = rep(2, 4), state2 = rep(0, 4))

LFOpars <- c("k.1", "k.2", "k.3", "k.4",
"d.1", "d.2", "d.3", "d.4")
LFOtimes <- seq(25, 150, by = 2.5)
LFObinf <- c(rep(0.2, 4), rep(0.01, 4))
LFObsup <- c(rep(20, 4), rep(0.1, 4))

4.3 Morris Screening

4.3.1 General ODE Models

The sensitivity analysis of a general ODE model (here the Lotka-Volterra example introduced
in subsection 4.1) can be performed by using the generic function ODEmorris(). The first six
arguments (mod to bsup) of the default method [Link]() accept the corresponding
objects created in subsection 4.1. Note that argument mod is specified in the way ode() from
the package deSolve expects its model functions. Arguments r, design and scale are exactly
the same as for morris() from the sensitivity package. See subsection 3.1 for more information
on these arguments. Here, we choose Morris’s OAT design with design$levels = 10 and
design$[Link] = 1. For ODE models, we recommend a much higher number r of replications
than the typically used r ∈ [10, 50] for non-ODE models. Here, we use r = 500 replications.
Argument ode_method sets the name of the integrator to use for ode() from the package deSolve.
In this example, we will use the default integrator "lsoda". The last two arguments indicate if
the evaluation of the model should be parallelized and if yes, how many processor cores shall be
used. We decide to parallelize using two processor cores.

library("ODEsensitivity")
[Link](7292)
LVres_morris <- ODEmorris(mod = LVmod,
pars = LVpars,
state_init = LVinit,
times = LVtimes,
binf = LVbinf,
bsup = LVbsup,
r = 500,

22
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

design = list(type = "oat",

levels = 10, [Link] = 1),
scale = TRUE,
ode_method = "lsoda",
parallel_eval = TRUE,
parallel_eval_ncores = parallel::detectCores())

We will now examine the output of [Link]():

str(LVres_morris, [Link] = 3, [Link] = FALSE)

## List of 2
## $ Prey : num [1:16, 1:51] 0.01 -0.019018 0.024223 0.000046 ...
## $ Predator: num [1:16, 1:51] 1.00e-02 9.31e-03 6.22e-05 -1.80e-02 ...

LVres_morris is a list of class "ODEmorris" with one element for each state variable (here,
Prey and Predator). Those elements are matrices of 3 · length(LVpars) + 1 = 16 rows and
length(LVtimes) = 51 columns. The first row contains a copy of all timepoints. The other rows
contain the Morris sensitivity indices µ, µ∗ and σ for all 5 parameters and all 51 timepoints.
ODEsensitivity provides a plot() method for objects of class "ODEmorris":

plot(LVres_morris)

23
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

Morris Screening for State Variable "Prey"

15
6

10
4
µ*

5
2
0

0
0 10 20 30 40 50 0 10 20 30 40 50

Time Time
rIng rGrow rMort assEff K

As can be seen from the plot for µ∗ , the predator’s ingestion rate rI has the largest overall influence
on P (t) for nearly the whole time from day 0 to day 50. rI is closely followed by the predator’s
assimilation efficiency kAE . At the beginning, the prey’s growth rate rG also has a large overall
influence on P (t), but this influence decreases with time and is exceeded by the overall influences
of the predator’s mortality rate rM and the carrying capacity K (at about t = 25 and t = 27,
respectively). Interpretations concerning the nonlinearity or interaction level of the influence of
all parameters can be carried out analogously using the plot for σ.

[Link]() has two important arguments: pars_plot and state_plot. Using pars_plot,
a subset of the parameters included in the sensitivity analysis can be selected for plotting (the
default is to use all parameters). state_plot gives the name of the state variable for which the
sensitivity indices shall be plotted (the default being the first state variable):

plot(LVres_morris, pars_plot = c("rIng", "rMort", "assEff"),

state_plot = "Predator")

24
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

Morris Screening for State Variable "Predator"

15
5
4

10
3
µ*

σ
2

5
1

0
0

0 10 20 30 40 50 0 10 20 30 40 50

Time Time
rIng rMort assEff

4.3.2 Low Frequency Oscillations

For the LFO example introduced in subsection 4.2, the generic ODEmorris() function can be used as
well. If an object of class "ODEnetwork" is supplied for argument mod, ODEmorris() automatically
passes on to the [Link]() method. In this case, argument state_init is not
needed since the initial values of the eight state variables x.1, . . . , x.4 and v.1, . . . , v.4 are
contained in lfonet$state. All other arguments correspond to those of [Link]().
In this example, ode_method is not needed because the ODE system is solved analytically. We
again point out that for Morris screening, all parameters have to be assumed to be uniformly
distributed.

[Link](4628)
LFOres_morris <- ODEmorris(mod = lfonet,
pars = LFOpars,
times = LFOtimes,
binf = LFObinf,
bsup = LFObsup,
r = 500,
design = list(type = "oat",
levels = 10, [Link] = 1),
scale = TRUE,

25
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

parallel_eval = TRUE,
parallel_eval_ncores = parallel::detectCores())

We won’t examine the structure of the output by [Link]() in detail since it

parallels that of [Link](): For each of the eight state variables, LFOres_morris
(an object of class "ODEmorris") contains a matrix with the three Morris sensitivity indices µ,
µ∗ and σ for all of the length(LFOpars) = 8 parameters and for all 51 timepoints. Thus, the
[Link]() method can be used here as well. We visualize the Morris sensitivity indices
µ∗ and σ for the deflection x2 (t) of the second mass:

plot(LFOres_morris, state_plot = "x.2")

Morris Screening for State Variable "x.2"

10

12
10
8

8
6
µ*

6
4

4
2

2
0

20 40 60 80 100 120 140 20 40 60 80 100 120 140

Time Time
k.1 k.3 d.1 d.3
k.2 k.4 d.2 d.4

µ∗ and σ seem to be very unstable when considering their courses over time. Nevertheless, the
spring constant k2 corresponding to the investigated mass i = 2 has the largest overall impact
on x2 (t) for all t ∈ [25, 150], even though this impact decreases over time. k2 is followed by
two other spring constants: k1 and k3 . The overall impact of d2 and k4 is very small and the
damping constants d1 , d3 and d4 have nearly no influence on x2 (t). Almost the same order of the
parameters with respect to µ∗ applies to σ.

26
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

4.4 Sobol’ Sensitivity Analysis

4.4.1 General ODE Models

A Sobol’ sensitivity analysis of ODE models can be performed using ODEsobol() from ODEsen-
sitivity. Just like for ODEmorris(), the model function (in the manner as supplied for ode()),
the names of the parameters, the initial values for the state variables and the timepoints of
interest have to be supplied as arguments. However, the arguments following then are specific to
ODEsobol().

Firstly, n gives the sample size for the Monte Carlo estimation. Here, we use n = 500 since the
default (n = 1000) requires a much longer runtime. The precision of the estimators might suffer
heavily, though, from reducing n.

Secondly, via rfuncs and rargs, the user specifies the distribution of the parameters (by the
name of the R function used to generate random numbers from that distribution) and the
corresponding distributional parameters (as arguments for the corresponding R function named in
rfuncs). If different distributions are needed, a vector containing the names of the corresponding
R functions has to be supplied for rfuncs. The same holds analogously for rargs if different
distributional parameters are needed. Here, we assume a uniform distribution for all parameters,
but with differing lower and upper boundaries. Thus, we set rfuncs = "runif" and for rargs,
we construct a vector of length k = 5 from LVbinf and LVbsup.

Thirdly, argument sobol_method indicates if the Sobol’-Jansen or the Sobol’-Martinez method

shall be used for estimating the Sobol’ sensitivity indices (see subsection 2.2). Results from these
two methods shouldn’t differ substantially, so it’s mainly a question of runtime which method
to choose (in tests, the Sobol’-Martinez method performed a little quicker, but not persistently).
In this example, we use the Sobol’-Martinez method. The last three arguments ode_method,
parallel_eval and parallel_eval_ncores are the same as for [Link]().

[Link](59281)
LVres_sobol <- ODEsobol(mod = LVmod,
pars = LVpars,
state_init = LVinit,
times = LVtimes,
n = 500,
rfuncs = "runif",
rargs = paste0("min = ", LVbinf,
", max = ", LVbsup),

27
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

sobol_method = "Martinez",
ode_method = "lsoda",
parallel_eval = TRUE,
parallel_eval_ncores = parallel::detectCores())
str(LVres_sobol, [Link] = 3, [Link] = FALSE)

## List of 2
## $ Prey :List of 2
## ..$ S: num [1:6, 1:51] 0.01 0.242 0.557 0 ...
## ..$ T: num [1:6, 1:51] 1.00e-02 3.10e-01 5.81e-01 2.30e-06 ...
## $ Predator:List of 2
## ..$ S: num [1:6, 1:51] 0.01 0.1852 0.0166 0.636 ...
## ..$ T: num [1:6, 1:51] 1.00e-02 2.07e-01 1.35e-05 6.31e-01 ...

The resulting object LVres_sobol (a list of class "ODEsobol") contains one element for each state
variable and in each of these elements two matrices: S for the first order indices and T for the total
indices (for all 5 parameters and all 51 timepoints). Just like for Morris screening, ODEsensitivity
provides a plot() method for objects of class "ODEsobol":

plot(LVres_sobol)

Sobol' sensitivity indices for "Prey" (Sobol'−Martinez method)

0.6

0.6
0.5

0.5
First order Sobol' indices

Total Sobol' indices

0.4

0.4
0.3

0.3
0.2

0.2
0.1

0.1
0.0

0.0

0 10 20 30 40 50 0 10 20 30 40 50

Time Time
rIng rGrow rMort assEff K

28
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

For the interpretation, we will focus on the total Sobol’ indices since they are more similar to µ∗
from Morris screening. For the prey’s population P (t), a clear order of the 5 parameters can be
established (at least for t ≥ 10): rI has the largest total effect, followed by kAE , rM , K and rG (in
this order). A similar order had resulted from Morris screening, but we can observe quite evident
differences between the plots of µ∗ and the total Sobol’ indices. However, a comparison of Sobol’
and Morris sensitivity indices is rather difficult since they measure different types of “influence”
of the input variables on the output variable (see section 5 for a more detailed explanation).

[Link]() also supports the arguments pars_plot and state_plot:

plot(LVres_sobol, pars_plot = c("rIng", "rMort", "assEff"),

state_plot = "Predator")

Sobol' sensitivity indices for "Predator" (Sobol'−Martinez method)

0.6

0.6
First order Sobol' indices

0.5

Total Sobol' indices

0.5
0.4

0.4
0.3

0.3
0.2
0.1

0.2
0.0

0 10 20 30 40 50 0 10 20 30 40 50

Time Time
rIng rMort assEff

4.4.2 Low Frequency Oscillations

[Link]() supports almost the same arguments as [Link](). Since

we assume a uniform distribution for all parameters, the Sobol’ sensitivity analysis of the LFO
example from subsection 4.2 can be performed as follows:

[Link](1739)
LFOres_sobol <- ODEsobol(mod = lfonet,

29
Sensitivity Analysis of ODE Models 4 The R Package ODEsensitivity

pars = LFOpars,
times = LFOtimes,
n = 500,
rfuncs = "runif",
rargs = paste0("min = ", LFObinf,
", max = ", LFObsup),
sobol_method = "Martinez",
parallel_eval = TRUE,
parallel_eval_ncores = parallel::detectCores())

The resulting object LFOres_sobol has a structure corresponding to that of LVres_sobol: It is

a list of class "ODEsobol" and contains the matrices S and T for all state variables x.1, . . . , x.4,
v.1, . . . , v.4. The warnings that are thrown here refer to a problem that typically occurs for
small values of n: In this case, the Sobol’ sensitivity indices estimated by Monte Carlo simulation
might be out of [0, 1]. The plots below visualize this issue more clearly.

plot(LFOres_sobol, state_plot = "x.2")

Sobol' sensitivity indices for "x.2" (Sobol'−Martinez method)

1.2
0.8

1.0
First order Sobol' indices

Total Sobol' indices

0.6

0.8
0.4

0.6
0.4
0.2

0.2
0.0

0.0

20 40 60 80 100 120 140 20 40 60 80 100 120 140

Time Time
k.1 k.3 d.1 d.3
k.2 k.4 d.2 d.4

We will focus again on the interpretation of the total Sobol’ indices: k2 has the biggest overall
impact on x2 (t) (for all t ∈ [25, 150]). This could also be observed for Morris screening, but with
decreasing values of µ∗ over time (the behaviour of µ∗ corresponds rather to that of the first order

30
Sensitivity Analysis of ODE Models 5 Conclusion

Sobol’ index). For d2 , the overall influence seems to increase linearly over time and exceeds the
influence of k1 and k3 at about t = 60. For k1 and k3 , the overall impacts also seem to increase
linearly, but with a smaller slope than for d2 . The parameters k4 , d1 , d3 and d4 don’t seem to have
any influence on x2 (t) at all. Unfortunately, just like for Morris screening, the Sobol’ sensitivity
indices are very unstable over time, too. This might indicate that this problem is related rather
to this ODE system than to the method of sensitivity analysis itself.

5 Conclusion

In this report, two methods – Morris screening and Sobol’ sensitivity analysis – have been presented
to analyze the sensitivity of a scalar-valued model function with regard to changes in its input
variables. When applied to ODE models, the results from those two methods agree only roughly,
though. A reason might be that the Morris and the Sobol’ sensitivity indices measure different
types of “influence” of the input variables on the output: The Morris sensitivity index µi measures
the “mean slope” of the output function with regard to the i-th input variable, µ∗i measures the
“mean absolute slope” and σi the variability of the (non-absolute) slope. In contrast, the first
order and total Sobol’ sensitivity indices (S{i} and S{i}
T , respectively) measure the variability of

the output variable with regard to the i-th input variable (plus interactions, for S{i}
T ). Apart

from that, it is always important to consider that all methods applied here rely on randomness
(for generating the parameter combinations where the model function is evaluated). Thus, we
can’t expect results to coincide exactly anyway. However, in consideration of the fairly distinct
deviations between the two methods, this doesn’t seem to be the major cause.

We conclude that the sensitivity analysis of ODE models using Morris screening and the Sobol’
method is rather explorative and results should be interpreted carefully.

5.1 Outlook

Further investigation might focus on the instability of the Morris and Sobol’ sensitivity indices
when analyzing LFOs. Especially, results from Morris screening using the simplex-based sampling
design could be of interest. With regard to LFOs, the sensitivity of other network structures
(not only circles) and networks of different sizes could be analyzed (e.g. n = 1, . . . , 5 and n = 30
for the so-called “New England Test System” from Surmann et al., 2014). Additionally, such an
analysis could also include other parameters, e.g. ki,j and di,j for i, j ∈ {1, . . . , n} with i 6= j.

31
Sensitivity Analysis of ODE Models References

References
Campolongo, F., Cariboni, J., and Saltelli, A. (2007). “An effective screening design for sensitivity
analysis of large models”. In: Environmental modelling & software 22.10, pages 1509–1518.

Confalonieri, R., Bellocchi, G., Bregaglio, S., Donatelli, M., and Acutis, M. (2010). “Comparison
of sensitivity analysis techniques: a case study with the rice model WARM”. In: Ecological
Modelling 221.16, pages 1897–1906.

Fruth, J. (2015). “New methods for the sensitivity analysis of black-box functions with an
application to sheet metal forming”. Dissertation. TU Dortmund University.

Jansen, M. J. W. (1999). “Analysis of variance designs for model output”. In: Computer Physics
Communications 117.1, pages 35–43.

Lotka, A. J. (1925). Elements of Physical Biology. 1st edition. Baltimore: Williams & Wilkins Co.

Martinez, J.-M. (2011). Analyse de sensibilité globale par décomposition de la variance. Presentation
in the meeting of GdR Ondes and GdR MASCOT-NUM, January, 13th, 2011, Institut Henri
Poincare, Paris, France.

Morris, M. D. (1991). “Factorial sampling plans for preliminary computational experiments”. In:
Technometrics 33.2, pages 161–174.

Pujol, G. (2009). “Simplex-based screening designs for estimating metamodels”. In: Reliability
Engineering and System Safety 94.7, pages 1156–1160.

Pujol, G., Iooss, B., Janon, A., Boumhaout, K., Da Veiga, S., Fruth, J., Gilquin, L., Guillaume, J.,
Le Gratiet, L., Lemaitre, P., Ramos, B., Touati, T., and Weber, F. (2016). sensitivity: Global
Sensitivity Analysis of Model Outputs. Online: [Link]
sensitivity, R package version 1.12.1.

R Core Team (2016). R: A Language and Environment for Statistical Computing. Online: http:
//[Link]/. Vienna, Austria.

Saltelli, A., Chan, K., and Scott, E. M. (2010). Sensitivity Analysis. 1st edition. New York: Wiley.

Sobol’, I. M. (1990). “On sensitivity estimation for nonlinear mathematical models”. In: Matem-
aticheskoe Modelirovanie 2.1, pages 112–118.

Sobol’, I. M. (2001). “Global sensitivity indices for nonlinear mathematical models and their
Monte Carlo estimates”. In: Mathematics and computers in simulation 55.1, pages 271–280.

32
Sensitivity Analysis of ODE Models References

Soetaert, K., Petzoldt, T., and Setzer, R. W. (2010). “Solving Differential Equations in R: Package
deSolve”. In: Journal of Statistical Software 33.9, pages 1–25.

Soetaert, K., Petzoldt, T., and Setzer, R. W. (2016). deSolve: Solvers for Initial Value Problems
of Differential Equations (ODE, DAE, DDE). Online: http : / / desolve . r - forge . r -
[Link]/, R package version 1.13.

Surmann, D. (2015). ODEnetwork: Network of Differential Equations. Online: [Link]

[Link]/package=ODEnetwork, R package version 1.2.

Surmann, D., Ligges, U., and Weihs, C. (2014). Modelling Low Frequency Oscillations in an
Electrical System. Energycon 2014 Conference. Dubrovnik: Faculty of Statistics, TU Dortmund
University.

Theers, S., Weber, F., and Surmann, D. (2016). ODEsensitivity: Sensitivity Analysis of Ordinary
Differential Equations. Online: [Link]
R package version 1.0.1.

Volterra, V. (1926). “Fluctuations in the abundance of a species considered mathematically”. In:

Nature 118, pages 558–560.

33