Journal of Computational Physics 493 (2023) 112455

Contents lists available at ScienceDirect

Journal of Computational Physics

journal homepage: [Link]/locate/jcp

DRIPS: A framework for dimension reduction and

interpolation in parameter space
Hannah Lu, Daniel M. Tartakovsky ∗
Department of Energy Science and Engineering, Stanford University, Stanford, CA 94305, USA

a r t i c l e i n f o a b s t r a c t

Article history: Reduced-order models are often used to describe the behavior of complex systems, whose
Received 24 July 2022 simulation with a full model is too expensive, or to extract salient features from the
Received in revised form 26 April 2023 full model’s output. We introduce a new model-reduction framework DRIPS (dimension
Accepted 21 August 2023
reduction and interpolation in parameter space) that combines the offline local model
Available online 29 August 2023
reduction with the online parameter interpolation of reduced-order bases (ROBs). The
Keywords: offline step of this framework relies on dynamic mode decomposition (DMD) to build
Reduced-order modeling a low-rank linear surrogate model, equipped with a local ROB, for quantities of interest
Data-driven learning derived from the training data generated by repeatedly solving the (nonlinear) high-
Parametric systems fidelity model for multiple parameter points. The online step consists of the construction
Manifold interpolations of a parametric reduced-order model for each target/test point in the parameter space,
with the interpolation of ROBs done on a Grassman manifold and the interpolation of
reduced-order operators done on a matrix manifold. The DMD component enables DRIPS
to model (typically low-dimensional) quantities of interest directly, without having to
access the (typically high-dimensional and possibly nonlinear) operators in a high-fidelity
model that governs the dynamics of the underlying high-dimensional state variables, as
required in projection-based reduced-order modeling. A series of numerical experiments
suggests that DRIPS yields a model reduction, which is computationally more efficient
than the commonly used projection-based proper orthogonal decomposition; it does so
without requiring a prior knowledge of the governing equation for quantities of interest.
Moreover, for the nonlinear systems considered, DRIPS is more accurate than Gaussian-
process interpolation (Kriging).
© 2023 Elsevier Inc. All rights reserved.

1. Introduction

Process-based models, typically based on appropriate conservation laws, play a central role in science and engineering.
Rapid advances in software and hardware development, including high-performance computing, have led to the proliferation
of more complex models that account for more (multiscale) phenomena and are solved on finer numerical meshes. Conse-
quently, high-fidelity simulations of many complex systems remain a demanding and often elusive task. That is especially
so in ensemble-based computations (e.g., global sensitivity analysis, uncertainty quantification, inverse modeling, and opti-
mization) that require many solves of a model for different combinations of the input parameters. The high computational
cost also precludes the use of such models to control a system’s dynamic behavior in real time.

* Corresponding author.
E-mail addresses: hannahlu@[Link] (H. Lu), tartakovsky@[Link] (D.M. Tartakovsky).

[Link]
0021-9991/© 2023 Elsevier Inc. All rights reserved.
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Model reduction techniques are designed to alleviate the (prohibitively) high computational cost of physics-based high-
fidelity models (HFMs), while capturing the essential features of the underlying dynamics. Such techniques for the construc-
tion of reduced-order models (ROMs) can be grouped in two general categories, projection-based methods and data-driven
methods. The former category relies on a Galerkin projection from a HFM of the system states onto a representative sur-
rogate model on a low-dimensional “trial” subspace. Examples of these techniques are proper orthogonal decomposition
(POD) [1–3] and discrete empirical interpolation (DEIM) [4] designed for linear and nonlinear systems, respectively. Both
rely on the singular value decomposition (SVD) analysis in the time domain to obtain linear trial subspaces; related tech-
niques in the frequency domain include balanced truncation [5,6] and rational interpolation [7,8]. Nonlinear trial manifolds
can be obtained using deep convolutional autoencoders [9]. The computational saving of such methods stems from replac-
ing the high-dimensional full system with its lower-dimensional counterpart that is used to make predictions. The resulting
ROMs are physics-based in the sense that they inherit the dynamic operator from the projection.
Data-driven ROMs reduce the complexity of a HFM by learning the dynamics of the state variables or the derived quan-
tities of interest (QoIs) directly from the HFM’s output and/or observational data. Techniques from this category include
Gaussian process regression (also known as Kriging or response surface estimation) [10,11], random forest (RF) [12,13], dy-
namic mode decomposition (DMD) [14,15], operator inference [16,17] and neural networks (NN) [18,19]. The flexibility and
robustness of this kind of data-driven learning of ROMs can be enhanced by the infusion of physics ingredients, especially
when data accessibility is limited [20–27].
Many complex systems are designed and analyzed in terms of their dependency on the system/model parameters repre-
senting, e.g., variations in shape, material properties, loading, and boundary and initial conditions. Regardless of the strategy
used for its construction, a ROM designed for one combination of the input parameters is not guaranteed to be accurate for
another combination. Parametric ROMs or PROMs strive to enhance ROM’s robustness and generalizability with respect to
the choice of parameters. The limited generalizability of projection-based ROMs is well understood [28–31] and ameliorated
by PROMs that interpolate, on suitable manifolds, either a reduced-order basis (ROB) [32] or the underlying ROM [33];
subsequent improvements in interpolation include [34–36]. The projection-based PROMs are “physics hungry”, because the
projections can only be designed for the full state variables with full knowledge of the dynamic operators. This complicates
their application to multi-physics nonlinear problems with a large number of state variables that give rise to extremely high
dimension after spatial discretization.
On the other hand, data-driven ROMs might fail to generalize to other parameter values when the time evolution of
QoIs, expressed as function of the parameters and simulated predictors, is highly nonlinear or/and exhibits bifurcations.
Conventional approximators like Gaussian process regression are known to struggle in capturing the dynamics precisely [33],
an observation verified in our numerical example in Section 4.3. Nonlinear machine learning methods, such as RF and NN,
act as accurate PROMs but only when enough high-fidelity data for a large range of input parameters are available for
training [37–39]. Other strategies for the data-driven construction of PROMs include operator inference [17], which needs
assumptions about polynomial structures of the underlying dynamics.
Given the aforementioned challenges, the development of efficient strategies to construct data-driven PROMs remains an
active area of research. Our study contributes to this field by introducing a holistic PROM framework called DRIPS (dimension
reduction and interpolation in parameter space), which combines the advantages of both categories of ROMs in two steps.
The offline step builds a low-rank DMD-based (linear) ROM for observables of the QoIs at each training sample parameter
point; the physics-guided selection of observables is based on the Koopman operator theory and not only increases the
accuracy of the local ROMs but also provides a bridge between the understanding of data and physics. The online step
constructs a new PROM with a corresponding ROB for each target/test point in the parameter space by interpolating the
training PROMs and ROBs; the interpolation is done on a suitable manifold in the same fashion as in projection-based
PROMs. Our framework obviates the need for both repeated access to expensive HFMs (the scourge of projection-based
PROMs) and large amounts of data (the shortcoming of data-driven PROMs).
The remainder of this paper is organized as follows. Section 2 contains the problem formulation. In Section 3, we describe
our general methodology and present algorithms for its implementation. Section 4 presents several numerical examples that
demonstrate the accuracy and robustness of the DRIPS framework, with comparison to other methods. Key results, their
implication for applications, and challenges and future work are summarized in Section 5.

2. Problem formulation

We consider a multi-physics system described by N sv  state variables s(x, t ; p) = {s , . . . , s  }, which vary in space x ∈ D
1 N sv
and time t ∈ [0, T ] throughout the simulation domain D during the simulation time interval [0, T ]. For a given set of N par
parameters p = { p 1 , . . . , p N par } ∈  from the parameter range , the spatiotemporal evolution of these state variables is
described by a system of coupled partial-differential equations (PDEs)
∂ si 
= φi (s; p), (x, t ) ∈ D × (0, T ]; i = 1, . . . , N sv , (2.1)
∂t
where φi are (linear/nonlinear) differential operators that contain spatial derivatives. When solved numerically, the spatial
domain D is discretized into N el elements/nodes, leading to a discretized state variable S(t ; p) of high dimension N sv =
 × N , satisfying a system of coupled ordinary differential equations (ODEs)
N sv el

2
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

dS
(t ; p) = (S(t ; p); p), (2.2)
dt
where  is a vector of linear/nonlinear functions.
More often than not, this model output has to be post-processed to compute N Q QoIs Q(t ; p)=[ Q 1 (t ; p), . . . , Q N Q (t ; p)] ,
such that

Q i (t ; p) = Mi (S(t ; p)), for i = 1, . . . , N Q . (2.3)

The maps Mi represent, e.g., numerical approximations of the integrals over D or a streamline. The QoIs are usually much
easier to visualize and comprehend than the raw output S(t ; p). In a typical application, N Q  N sv and, hence, it is more
advantageous to solve the HFM for the QoIs Q(t ; p),
dQ
(t ; p) = F(Q(t ; p); p), (2.4)
dt
rather than the HFM (2.2) for the state variables S(t ; p). The vector function F has a complicated, and potentially unknown
form. The corresponding discrete-time dynamical system is described by
tk+t

Q(tk+1 ; p) = Ft (Q(tk ; p); p) := Q(tk ; p) + F(Q(τ ; p))dτ , (2.5)

tk

for the uniform time discretization tk = kt ∈ [0, T ] with k = 0, . . . , N T .

The HFM model (2.2) and (2.3) is solved N MC times for different combinations of the parameter values, i.e., points p(i )
(i = 1, . . . , N MC ) in the parameter space . Each solution is recorded as N snap temporal snapshots (at times t 0 , . . . , t N snap ) of
the QoIs Q(t ; p(i ) ), forming a dataset

{Q(t 0 ; p(1) ), . . . , Q(t N snap ; p(1) ), . . . , Q(t 0 ; p( N MC ) ), . . . , Q(t N snap ; p( N MC ) )}, N snap ≤ N T . (2.6)
Our goal is to construct from this dataset a reduced-dimension surrogate of the HFM (2.5). This will allow us to predict the
trajectory of the QoIs Q(t ; p∗ ) at an unsampled parameter point p∗ ∈
/ {p(1) , . . . , p(N MC ) }, i.e., {Q(t 0 ; p∗ ), . . . , Q(t N T ; p∗ )}, at a
low cost without computing the complex HFM (2.2) and (2.3).

3. Methodology

Our DRIPS framework consists of the offline step and the online step that are detailed in Sections 3.1 and 3.2, respectively.

3.1. Offline step: DMD-based surrogates

For complex HFMs like (2.2) and (2.3), the dynamics of QoIs Q, i.e., the functional form of F in (2.4) or Ft in (2.5) is
generally unknown. In the first step of our algorithm, we replace the unknown discrete HFM (2.5) with its linear reduced-
dimension surrogate constructed from the dataset (2.6). The latter task is facilitated by the Koopman operator theory, which
allows one to handle the potential nonlinearity in the unknown dynamic F and Ft :

Definition 3.1 (Koopman operator [15]). For nonlinear dynamic system (2.4), the Koopman operator Kp is an infinite-
dimensional linear operator that acts on all observable functions g : M → C so that

Kp g (Q(t ; p)) = g (F(Q(t ; p); p)). (3.1)

p
For discrete dynamic system (2.5), the discrete-time Koopman operator Kt is
p
Kt g (Q(tk+1 ; p)) = g (Ft (Q(tk ; p); p)) = g (Q(tk+1 ; p)). (3.2)

The Koopman operator transforms the finite-dimensional nonlinear problem (2.4) in the state space into the infinite-
p
dimensional linear problem (3.1) in the observable space. Since Kt is an infinite-dimensional linear operator, it is equipped
∞ ∞
with infinite eigenvalues {λk (p)}k=1 and eigenfunctions {φk (p)}k=1 . In practice, one has to make do with a finite number of
the eigenvalues and eigenfunctions. The following assumption is essential to both a finite-dimensional approximation and a
choice of observables.

Assumption 3.1. Let y(tk ; p) denote an N × 1 vector of observables,

⎡ ⎤
g 1 (Q(tk ; p))
⎢ .. ⎥
y(tk ; p) = g(Q(tk ; p)) = ⎣ . ⎦, (3.3)
g N (Q(tk ; p))

3
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

where g j : M → C is an observable function with j = 1, . . . , N. If the chosen observables g are restricted to an invariant
p
subspace spanned by eigenfunctions of the Koopman operator Kt , then they induce a linear operator K(p) that is finite-
dimensional and advances these eigen-observable functions on this subspace [40].

Assumption 3.1 enables one to deploy the DMD Algorithm 1 to approximate the N-dimensional linear operator K(p) and
its low-dimensional approximation Kr (p) of rank r. At each parameter point p(i ) with i = 1, . . . , N MC , the discrete HFM (2.5)
in the state space is approximated by an N-dimensional linear surrogate model

y(tk+1 ; p(i ) ) = K(p(i ) )y(tk ; p(i ) ) (3.4)

in the observable space. The two spaces are connected by the observable function g and its inverse g−1 . Algorithm 1 induces
the ROM for (3.4),

q(tk+1 ; p(i ) ) = Kr (p(i ) )q(tk ; p(i ) ). (3.5)

Here q(tk ; p(i ) ) is the reduced-order observable vector of dimension r. In terms of a ROB V(p(i ) ), these are expressed as

y(tk ; p(i ) ) = V(p(i ) )q(tk ; p(i ) ) and Kr (p(i ) ) = V(p(i ) ) K(p(i ) )V(p(i ) ). (3.6)

Algorithm 1: DMD algorithm on observable space [15] for parameter point p(i ) , i = 1, . . . , N MC .
Input: {Q(t 0 ; p(i ) ), . . . , Q(t N snap ; p(i ) )}, observable function g

1. Create data matrices of the observables

⎡ ⎤ ⎡ ⎤
| | | |
Y1 (p ) = ⎣ y(t 0 ; p(i ) ) . . .
(i )
y(t N snap −1 ; p(i ) ) ⎦ , Y2 (p ) = ⎣ y(t 1 ; p(i ) ) . . .
(i )
y(t N snap ; p(i ) ) ⎦ , (3.7)
| | | |

where each column is given by y(tk ; p(i ) ) = g(Q(tk ; p(i ) )).

2. Apply SVD Y1 (p(i ) ) ≈ V(p(i ) )(p(i ) )Z(p(i ) ) , where V(p(i ) ) ∈ C N ×r , (p(i ) ) ∈ C r ×r , Z(p(i ) ) ∈ C r × N snap is the conjugate transpose of Z(p(i ) ), and r
is the truncation rank chosen by certain criteria and kept the same for all i = 1, . . . , N MC .
3. Compute Kr (p(i ) ) = V(p(i ) ) Y2 (p(i ) )Z(p(i ) )(p(i ) )−1 as an r × r low-rank approximation of K(p(i ) ).
4. Compute P(i , j ) = V(p(i ) ) V(p( j ) ) for i , j = 1, . . . , N MC .

Output: V(p(i ) ), Kr (p(i ) ) and P(i , j ) .

Remark 3.1. The construction of DMD surrogates can be done offline, which allows one to precompute P(i , j ) for the later
online step. Although this offline step takes a majority of the computational time in the whole framework, mostly due to
the computation of the high-fidelity training data (2.6), its output can be precomputed and stored efficiently; the output
storage is ( N · r + r · r + r · r · ( N MC + 1)/2) · N MC .

Remark 3.2. A theorem in [41] establishes connections between the DMD theory and the Koopman spectral analysis under
specific conditions on the observables and collected data. This theorem indicates that judicious selection of the observables
is critical to the success of the Koopman method. There is no principled way to select observables without expert knowledge
of a dynamical system. Machine learning techniques can be deployed to identify relevant terms in the dynamics from
data, which then guides the selection of the observables [42,43]. In our numerical examples, we rely on knowledge of the
underlying physics to select the observables, as was done, e.g., in [21–24,44].

Remark 3.3. In general, the rank r is chosen to satisfy the energy criteria
σk
r = min{ < ε },
k m σm

where σk are the diagonal elements of  in SVD, and ε is a user-supplied small number. Since r has to be kept the same
for all training parameter points, we choose r in each problem heuristically by observing how the singular value decays in
the training points.

3.2. Online step: interpolation of ROBs and PROMs

For an unsampled parameter point p∗ ∈

/ {p(1) , . . . , p(N MC ) }, the goal is to compute {Q(t 1 ; p∗ ), . . . , Q(t N T ; p∗ )} via the PROM

q(tk+1 ; p∗ ) = Kr (p∗ )q(tk ; p∗ ), (3.8)

4
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

without evaluating the HFM (2.2) and (2.3). Subsequently, the observables y and the QoIs Q are estimated as

y(tk ; p∗ ) = V(p∗ )q(tk ; p∗ ), Q(tk ; p∗ ) = g−1 (y(tk ; p∗ )). (3.9)

Therefore, the online task comprises the computation of three quantities, V(p∗ ), Kr (p∗ ), and q(tk ; p∗ ).

3.2.1. ROB interpolation

We rely on interpolation on the Grassman manifold to compute the ROB V(p∗ ) from the dataset {V(p(1) ), . . . , V(p( N MC ) )}.
We briefly review this interpolation approach [32] below.

Definition 3.2. The following manifolds are of interest:

• Grassmann manifold G (r , N ) is a set of all subspaces in R N of dimension r;

• Stiefel manifold ST (r , N ) is a set of orthogonal ROB matrices in R N ×r .

The ROB V(p(i ) ) ∈ R N ×r , with i = 1, . . . , N MC and r ≤ N, is a full-rank column matrix, whose columns provide a basis of
subspace Si of dimension r in R N . While an associated ROM is not uniquely defined by the ROB, it is uniquely defined by
the subspace Si . Therefore, the correct entity to interpolate is the subspace Si , rather than the ROB V(p(i ) ). Hence, the goal
is to compute S∗ = range(V(p∗ )) by interpolating between {Si }i =MC ∗
N
1 with access to the ROB V(p ).
The subspaces S1 , . . . , S N MC belong to the Grassmann manifold G (r , N ) [45–49]. Each r-dimensional subspace S̃ of R N
is regarded as a point of G (r , N ) and is nonuniquely represented by a matrix Ṽ ∈ R N ×r , whose columns span the subspace
S̃ . The matrix Ṽ is chosen among those whose columns form a set of orthonormal vectors in R N and belong to the
orthogonal Stiefel manifold ST (r , N ) [45,49]. There exists a projection map [45] from ST (r , N ) onto G (r , N ), as each matrix
in ST (r , N ) spans an r-dimensional subspace of R N and different matrices can span the same subspace. At each point S˜
of the manifold G (r , N ), there exists a tangent space TS̃ [45,49] of the same dimension [49]. Each point in this space is
represented by a matrix ˜ ∈ R N ×r . Since T is a vector space, usual interpolation is allowed for the matrices representing
S̃
its points. Let  = mV (V(p(i ) )), where mV denotes a map from the matrix manifolds G (r , N ) onto the tangent space TS̃ .
i

This suggests a strategy of computing ∗ via usual interpolation between {i }i =MC and then evaluating V(p∗ ) through the
N
1
inverse map m− 1 ∗
V ( ).
The map mV is chosen to be a logarithmic mapping, which maps the Grassmann manifold onto its tangent space, and
m− 1
V is chosen to be an exponential mapping, which maps the tangent space onto the Grassmann manifold itself. This choice
borrows concepts of geodesic path on a Grassmann manifold from differential geometry [45,46,50]. This strategy, discussed
in detail in [32], is implemented in Algorithm 2.

Algorithm 2: Interpolation of ROBs [32,51,48].

and target parameter point p∗
N N N
Input: {V(p(i ) )}i =MC
1
, {P(i , j ) = V(p(i ) ) V(p(i ) )}i , MC
j =1
, {p(i ) }i =MC
1

1. Denote Si = range(V(p(i ) )), for i = 1, . . . N MC . A point Si 0 with i 0 ∈ {1, . . . N MC } of the manifold is chosen as a reference and origin point for
interpolation.
2. Select points Si with i ∈ Ii 0 ⊂ {1, . . . , N MC }, which lie in a sufficiently small neighborhood of Si 0 , and use the logarithm map lnSi to map
0
{Si }i ∈Ii0 onto matrices {i }i ∈Ii0 representing the corresponding points of TSi0 . This is computed as

(I − V(p(i 0 ) )V(p(i 0 ) ) )V(p(i ) )(P(i 0 ,i ) )−1 = Ui i Wi , (thin SVD)

(3.10)
i −1
 = Ui tan 
( i )W i .

3. Compute ∗ by interpolating {i }i ∈Ii entry by entry:

0

∗
ij = P (p∗ ; { i (i )
i j , p }i ∈Ii 0 ), i = 1, . . . , N , j = 1, . . . , r . (3.11)

4. Use the exponential map expSi to map the matrix ∗ , representing a point of TSi , onto the desired subspace S∗ on G (r , N ) spanned by the
0 0
ROB V(p∗ ). This is computed as

∗ = U∗ ∗ W∗ , (thin SVD)

(3.12)
V(p∗ ) = V(p(i 0 ) )W∗ cos(∗ )W∗ + U∗ sin(∗ )W∗ .

Output: V(p∗ )

5
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Remark 3.4. A choice of the interpolation method P depends on the dimension of the parameter space, N par . If N par = 1,
a univariate (typically, a Lagrange type) interpolation method is chosen. Otherwise, a multivariate interpolation scheme
(e.g., [52,53]) is used.

Remark 3.5. Since the logarithmic map lnSi is defined in a neighborhood of Si 0 ∈ G (r , N ), the method is expected to be
0
insensitive to a choice of the reference point Si 0 in step 1 of Algorithm 2. This is confirmed in numerical experiments [32].

3.2.2. PROM interpolation

The reduced-order operator Kr (p∗ ) in (3.8) is computed via interpolation on the matrix manifold between the ROMs
{Kr (p(1) ), . . . , Kr (p(N MC ) )}. This is done in two steps [33]:

• Step A). Since any ROM can be endowed with multiple alternative ROBs, the resulting ROMs may have been computed
in different generalized coordinates systems. The validity of an interpolation may crucially depend on a choice of the
representative element within each equivalent class. Given the precomputed ROMs {Kr (p(1) ), . . . , Kr (p( N MC ) )}, a set of
congruence transformations is determined so that a representative element of the equivalent ROBs for each precom-
puted ROM is chosen to assign the precomputed ROMs into consistent sets of generalized coordinates. The consistency
is enforced by solving the orthogonal Procrustes problems [54],

min V(p(i ) ) Qi − V(p(i 0 ) ) F, ∀i = 1, . . . , N MC , (3.13)

Qi ,Qi Qi =Ir

where i 0 ∈ {1, . . . N MC } is chosen as a reference configuration. The representative element is identified by solving the
above problem with a direct procedure. This procedure is summarized in Algorithm 3.

Algorithm 3: Step A of the PROM interpolation [32].

N
Input: {Kr (p(1) ), . . . , Kr (p( N MC ) )}, {P(i , j ) = V(p(i ) ) V(p(i ) )}i , MC
j =1
, reference configuration choice i 0
For i ∈ {1, . . . N MC } \ {i 0 }

– Compute P(i ,i 0 ) = Ui ,i 0 i ,i 0 Zi ,i (SVD),

0
– Compute Qi = Ui ,i 0 Zi ,i ,
0
– Transform K̃r (p(i ) ) = Qi Kr (p(i ) )Qi

End
Output: {K̃r (p(1) ), . . . , K̃r (p( N MC ) )}

Remark 3.6. An optimal choice of the reference configuration i 0 , if it exists, remains an open problem.

Remark 3.7. Algorithm 3 is related to mode-tracking procedures based on the modal assurance criterion (MAC) [55]. This
connection is elucidated in [33].

• Step B). The transformed ROMs {K̃r (p(1) ), . . . , K̃r (p(N MC ) )} are interpolated to compute a ROM Kr (p∗ ). Similar to Sec-
tion 3.2.1, this interpolation must be performed on a specific manifold containing both {K̃r (p(1) ), . . . , K̃r (p( N MC ) )} and
Kr (p∗ ), so that the distinctive properties (e.g., orthogonality, nonsingularity) are preserved. The main idea again is to
first map all the precomputed matrices onto the tangent space to the matrix manifold of interest at a chosen ref-
erence point using the logarithm mapping, then interpolate the mapped data in this linear vector space, and finally
map the interpolated result back onto the manifold of interest using the associated exponential map. This is done in
Algorithm 4.

Algorithm 4: Step B of the PROM interpolation [32].

Input: {K̃r (p(1) ), . . . , K̃r (p( N MC ) )}, reference configuration choice i 0

1. For i ∈ {1, . . . N MC } \ {i 0 }
– Compute i = lnK̃ (p(i0 ) ) (K̃r (p(i ) ))
r
End
∗
2. Compute  by interpolating {i }i ∈Ii entry by entry, as in (3.11)
0
3. Compute Kr (p∗ ) = expK̃ (p(i0 ) ) (∗ )
r

Output: Kr (p∗ )

Remark 3.8. The ln and exp in Algorithm 4 denote the matrix logarithm and exponential respectively. The specific expres-
sions of different matrix manifolds of interest are listed in Table 4.1 of [33]. In general, the set {Kr (p(1) ), . . . , Kr (p( N MC ) )}

6
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

obtained from DMD (Algorithm 1) does not have a special structure (e.g., symmetric positive definite), so we choose the
interpolation on a matrix manifold for general linear group, i.e., lnX (Y) = Y − X and expX () = X + .

Remark 3.9. We choose the same reference point i 0 for Algorithms 2–4, so that V(p∗ ) is interpolated from the reference
coordinate of V(p(i 0 ) ) and corresponds to Kr (p∗ ) in the same consistent generalized coordinates. If the reference point i 0
is chosen differently for Algorithm 2 and Algorithms 3 or 4, then Kr (p∗ ) needs to be adjusted in a way similar to Step A
(Algorithm 3).

3.2.3. Iteration-free computation of the solution

With Kr (p∗ ) and V(p∗ ) computed, we use Algorithm 5 to obtain the solution using the iteration-free feature of DMD
framework.

Algorithm 5: DMD reconstruction.

Input: Kr (p∗ ), V(p∗ ), y(t = 0; p∗ ), g

1. Compute the eigen-decomposition of Kr (p∗ ):

Kr (p∗ )(p∗ ) = (p∗ )(p∗ ), (3.14)

where columns of (p∗ ) are eigenvectors and (p∗ ) is a diagonal matrix containing the corresponding eigenvalues λi with i = 1, . . . , r.
2. Compute the DMD modes

(p∗ ) = V(p∗ )(p∗ ). (3.15)

3. Reconstruct the observables:

yDMD (tk ; p∗ ) = (p∗ )(p∗ )k (p∗ )† y(0; p∗ ) , k = 1, . . . , N T , (3.16)

where † is Moore–Penrose pseudoinverse.

4. Map the observables back to state space:

QDRIPS (tk ; p∗ ) = g−1 (yDMD (tk ; p∗ )) (3.17)

Output: QDRIPS (tk ; p∗ )

3.3. Algorithm summary

The proposed DRIPS framework is implemented in Algorithm 6.

Algorithm 6: DRIPS framework.

Offline Step:
For i = 1, . . . , N MC , compute the high fidelity training data (2.6),
Algorithm 1
Input: {Q(t 1 ; p(i ) ), . . . , Q(t N snap ; p(i ) )} and g −−−−−−→ Output: V(p(i ) ), Kr (p(i ) ) and P(i , j )
End
Online Step:
Algorithm 2
, p∗ −−−−−−→ Output: V(p∗ )
N N N
• Interpolation of ROBs: Input: {V(p(i ) )}i =MC
1
, {P(i , j ) }i , MC
j =1
, {p(i ) }i =MC
1
Algorithms 3 & 4
, reference choice i 0 −−−−−−−−−→ Output: Kr (p∗ )
N N
• Interpolation of PROMs: Input: {Kr (p(i ) )}i =MC
1
, {P(i , j ) }i , MC
j =1
Algorithm 5
• DMD reconstruction: Input: Kr (p∗ ), V(p∗ ), y(t = 0; p∗ ), g −−−−−−→ Output: QDRIPS (tk ; p∗ )

Remark 3.10. The sampling strategy for {p(1) , . . . , p( N MC ) } in the parameter space plays a key role in the accuracy of the
subspace approximation. The so-called “curse of dimensionality”, which refers to the exponential growth of the number
of training samples N MC with the number of parameters, N par , is a well-known challenge. In general, uniform sampling
is used for N par ≤ 5 and moderately computationally intensive HFMs, latin hypercube sampling is used for N par > 5 and
moderately computationally intensive HFMs, and adaptive, goal-oriented, greedy sampling is used for N par > 5 and highly
computationally intensive HFMs. We limit our numerical experiments to N par = 3 for simplicity, leaving the challenge posed
by the curse of dimensionality for future studies.

4. Numerical experiments

We use a series of numerical experiments to demonstrate the performance of DRIPS. In Section 4.1, we consider a linear
advection-diffusion problem, in which the dynamical operators in the HFM are accessible for the construction of POD-PROM.
Our DRIPS algorithm, i.e., DMD-PROM, is compared with the conventional projection-based POD-PROM to illustrate the

7
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

benefits of knowing the physics in constructing accurate ROMs. Section 4.2 deals with a problem of linear heat conduction
in a complex domain, which is solved by the Matlab PDE toolbox that provides a linear HFM without an explicit form. Unlike
POD-PROM, DRIPS requires no large differential matrices and embeddings of complex boundary conditions; consequently it
exhibits advantageous performance in constructing accurate ROMs for different QoIs. In Section 4.3, a nonlinear problem
is used to demonstrate that our physics-aware framework DRIPS outperforms the pure data-driven approach Kriging and
serves as a viable reduced-order modeling tool.
Kriging, aka Gaussian Process (GP) surrogates, provides a popular alternative to the DRIPS framework. Kriging interpolates
the QoI solution map pixel by pixel based on prior covariances, which depend on the distance between the target point and
the sampled points in the parameter space. It is a purely data-driven approach in the sense that it does not account for
the underlying governing equations. Although Kriging is widely used in many applications (mostly for linear problems),
it may fail to detect bifurcations in a complex dynamic system [33]. We will verify this observation by comparing the
performance of Kriging and DRIPS in a series of numerical experiments. A GP surrogate is constructed at each time instance,
with N par = 3 parameter values as input and the QoI at each time step as (N Q -dimensional) output. We use a radial basis
function kernel with default length scale 1 and default bound (10−5 , 105 ).
In the following numerical tests, the performance of DRIPS, POD-PROM, and GP is quantified in terms of the relative L 2
errors,

QPOD (t ; p∗ ) − Q(t ; p∗ )
εPOD (t ; p∗ ) =
2
,
Q(t ; p∗ ) 2
QGP (t ; p ) − Q(t ; p∗ )
∗
εGP (t ; p∗ ) =
2
, (4.1)
Q(t ; p∗ ) 2
QDRIPS (t ; p∗ ) − Q(t ; p∗ )
εDRIPS (t ; p∗ ) =
2
,
Q(t ; p∗ ) 2

and total relative L 2 errors

NT
k =1
QPOD (tk ; p∗ ) − Q(tk ; p∗ ) 2
2
∗
EPOD (p ) = ,
NT
k =1
Q(tk ; p∗ ) 2
2

NT
k =1
QGP (tk ; p∗ ) − Q(tk ; p∗ ) 2
2
EGP (p∗ ) = , (4.2)
NT
k =1
Q(tk ; p∗ ) 2
2

NT
k =1
QDRIPS (tk ; p∗ ) − Q(tk ; p∗ ) 2
2
∗
EDRIPS (p ) = .
NT
k =1
Q(tk ; p∗ ) 2
2

The codes of the numerical experiments are available on Github: [Link]

4.1. Advection-diffusion transport

Consider a state variable s(x, t ), whose dynamics is governed by the two-dimensional advection-diffusion equation

∂s ∂s
+ u − κ ∇ 2 s = 0, x = (x, y ) ∈ D = (0, 1) × (0, 1), t ∈ (0, 1), (4.3a)
∂t ∂x
with the initial and boundary conditions

s(x, t = 0) = 0,
s(x, t ) = s D ( y , t ), for x ∈ D = {0} × [0, 1], (4.3b)
∇ s(x, t ) · n(x) = 0, for x ∈ N = {{1} × [0, 1]} ∪ {[0, 1] × {1}} ∪ {[0, 1] × {0}},
where
⎧ 1
⎪
⎪ y ∈ [0, ]
⎪0
⎪
for
3
⎪
⎨
1 2
s D ( y , t ) = 300 + 325(sin(3π | y − ȳ |) + 1) for y∈[ , ] (4.3c)
⎪
⎪ 3 3
⎪
⎪
⎪
⎩0 2
for y ∈ [ , 1],
3
8
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 1. Temporal snapshots, at times t = 0, 0.5 and 1.0, of the training data generated via the HFM (4.4) for the parameter values p(1) = {1000, 100, 0.36}
(top row) and p(2) = {1000, 300, 0.36} (middle row) and of the test data generated via the HFM (4.4) for the randomly selected parameter value p∗ =
{2934.0597, 209.4464, 0.4184} (bottom row). (For interpretation of the colors in the figure(s), the reader is referred to the web version of this article.)

with p = {u , κ , ȳ } ∈  = [1000, 3000] × [100, 300] × [0.36, 0.42]. The HFM represents a numerical solution of (4.3) obtained
via finite differences, upwind scheme for the advection term and implicit center difference scheme for the diffusion term,
using 75 points in each spatial direction. The resulting discretized state variable S(t ; p ) forms a vector of dimension N sv =
752 and is propagated in time with time step t = 0.01.
The QoIs in the first example are Q(tn ; p ) = S(tn ; p ); they follow the linear HFM

Q(tn+1 ; p ) = A( p )Q(tn ; p ) + b( p ), (4.4)

where A is a tridiagonal matrix representing the advection and diffusion operators and b is a non-zero vector representing
the inhomogeneous boundary conditions.
The training data are generated from the HFM (4.4) evaluated on the Cartesian grid of the (N par = 3)-dimensional
parameter space (a cube) with 3 points (two endpoints and one middle point) in each dimension, i.e., by solving (4.4)
N MC = 27 times. Fig. 1 exhibits temporal snapshots of two representative HFM solutions, Q(tk ; p(1) ) and Q(tk ; p(2) ), where
p(1) = {1000, 100, 0.36} and p(2) = {1000, 300, 0.36}. Our goal is to obtain a solution, Q(tk ; p∗ ), for a randomly selected pa-
rameter value p∗ = {2934.0597, 209.4464, 0.4184}. ∈ , without computing (4.4). This solution is then compared with the
reference solution Q(t ; p∗ ) of the HFM (4.4), which is also shown in Fig. 1.
We compare the performance of DRIPS and POD-PROM, which differ in their computation of the ROM {Ar (p(i ) ), br (p(i ) )}
and Kr (p(i ) ) in the offline step. The first relies on DMD, as described in Algorithm 6. The second replaces DMD with POD
(Algorithm 7) to obtain {Ar (p(i ) ), br (p(i ) )} from the Galerkin projection (A.1) (or alternative projections, e.g., Petrov-Galerkin,

9
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 2. Left: temporal evolution of the relative L 2 error of POD-PROM and DRIPS, εPOD and εDRIPS in (4.1), trained on different numbers of snapshots N snap .
Right: dependence of the total relative L 2 error of POD-PROM and DRIPS, EPOD and EDRIPS in (4.2), on the truncation rank r.

DEIM, etc.). This requires the full knowledge of A(p(i ) ) and b(p(i ) ), which is not needed in the DMD version that learns
Kr (p(i ) ) from the training dataset (2.6). Since the HFM (4.4) is linear, we set the observable function g as

y = g(Q) = (1, Q ) , (4.5)

so that (3.4) reduces to

 
1 0
y(tn+1 ; p ) = K( p )y(tn ; p ), K( p ) = . (4.6)
b( p ) A( p )

Alternatively, one can use the extended DMD, xDMD [24], for the offline step.
The performance of DRIPS and POD-PROM is quantified in terms of the relative L 2 errors defined in (4.1). They compare
the predictions of the POD-PROM and DRIPS, QPOD (t ; p∗ ) and QDRIPS (t ; p∗ ), to the reference HFM prediction Q(t ; p∗ ) at the
test parameter point p∗ . Fig. 2 displays these errors for POD-PROM and DRIPS of rank r = 10, computed from different
numbers of snapshots, N snap . For both methods, the error decreases with N snap . That is to be expected since the low-
dimensional subspace, represented by the ROBs determined from the training data, becomes more optimal as the amount
of informative data increases. When the data are relatively scarce (N snap = 25 and 50), εDRIPS (t ; p ∗ ) ≥ εPOD (t ; p ∗ ) because of
the loss of accuracy in learning Kr ( p (i ) ) from data in the DMD framework. Given enough data (N snap = 100), the discrepancy
between POD-PROM and DRIPS diminishes as the learned Kr ( p (i ) ) becomes sufficiently accurate.
We use another metric, the total relative L 2 errors defined in (4.2) to elucidate the impact of the rank selection on
the performance of POD-PROM and DRIPS algorithm. These errors are shown in Fig. 2 for the same number of snapshots,
N snap = N T = 100. Both EPOD and EDRIPS decrease with the rank r, saturating after r = 8, which verifies the low-rank nature
of this problem. DRIPS is slightly less accurate than POD-PROM for small r because of the loss in accuracy of learning
Kr ( p (i ) ). The accuracy of the DMD-based ROMs is affected by the number of snapshots and the rank truncation, as studied
in, e.g., [22].
The accuracy of the interpolated PROMs also depends on the location of the target points in the parameter space. To
better visualize the distance in the parameter space, we fix κ = 250 and ȳ = 0.4, thus reducing the dimension of the
parameter space to N par = 1. Fig. 3 shows the accuracy of DRIPS and POD-PROM at different target points in the one-
dimensional parameter space, trained from N MC = 2, 3, 4 training parameter points, respectively. The errors of both methods
reach minimum at the training parameter points and increase with the distance from these points. The prediction accuracy
(i.e., accuracy at unsampled points in the whole parameter space) of the PROMs constructed with both methods increases
with the number of training points, N MC .
Next, we repeatedly draw N test = 100 random samples from the N par = 3 parameter space as test data and compare the
computational cost and accuracy of DRIPS (the last column in Table 1), POD-PROM (the second column in Table 1) and GP
(the third column in Table 1). POD-PROM and DRIPS provide a comparable speedup, but the POD-PROM serves as a more
accurate surrogate than DRIPS. More than 95% of the computational time is spent on generating high-fidelity training data
(N MC = 27 solved of the HFM) in the offline step of both methods. The offline step of DRIPS is more expensive than that
of POD due to the construction of the Kr from data (rather than directly from Galerkin projection of A(p(i ) ) and b(p(i ) ) in
POD). The offline training of GP is more expensive in this example because the interpolation is done pixel by pixel on the
data of size N Q ( N Q  1). Once the POD-ROM surrogate is trained, it provides more accurate predictions at the test points

10
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 3. The total relative L 2 error of DRIPS (left) and POD-PROM (right) in the target parameter space using different number of training parameter points.
N MC = 2 is the case of using the two end points 1000 and 3000; N MC = 3 is the case of using three uniform points 1000, 2000 and 3000; N MC = 4 is the
case of using four uniform end points 1000, 5000/3, 7000/3 and 3000.

Table 1
Simulation time and accuracy of the alternative calculations of the QoIs for test data consisting of N test = 100
randomly drawn realization of (4.3).

HFM Q=S Q = s(xs )

POD GP DRIPS POD GP DRIPS

Offline (sec) 149.10 150.57 156.18 155.04 152.99 151.03 151.22

Online (sec) 552.20 7.06 0.77 6.92 7.13 0.04 0.28
Online speedup – 78.25 714.27 79.80 77.48 14569.97 1977.01
Total speedup – 4.45 4.47 4.33 4.38 4.64 4.63
Average E – 1.5 · 10−3 2.0 · 10−3 2.5 · 10−3 1.7 · 10−4 8.0 · 10−4 7.9 · 10−4

than either DRIPS or GP does. The online speedup of GP is superior to that of the other two methods. However, GP cannot
be generalized to future time predictions.
The QoI in the second example is the time series Q (tn ; p) = s(xs , tn ; p), where xs = (0.25, 0.5) is the location of a
sensor. To allow enough time for the flux to reach the sensor location, we are only interested in tn > 0.5. The QoI follows
the linear HFM
S(tn+1 ; p) = A( p )S(tn ; p) + b(p),
(4.7)
Q (tn+1 ; p) = CS(tn+1 ; p).
Here C is a 1 × N sv matrix with entries

1 i = is,
C1,i = (4.8)
0 otherwise,

where i s is an index in the discretized vector S corresponding to xs in s(x). In this example, POD follows the same strategy
as before, which computes the ROM {Ar (p(i ) ), br (p(i ) ), Cr (p(i ) )} from the Galerkin projection (A.1) with access to S(p(i ) ) and
required full knowledge of A(p(i ) ), b(p(i ) ) and C. On the contrary, DMD constructs a surrogate for Q without any knowledge
about the HFM (4.7). The observable function g is chosen as Hermite polynomials H of Q (t n ; p) up to order m = 4,

y = g( Q ) = [H0 ( Q ), . . . , Hm ( Q )] . (4.9)

Fig. 4 shows that POD-PROM and DRIPS algorithm agree well with the reference HFM for both training data (left) and
test data (right). Table 1 compares the computational costs and accuracy of DRIPS, POD-PROM and GP for a randomly drawn
N test = 100 test dataset. Since the dominant computational costs in online step stem from the entry-by-entry interpolation
of ROBs (3.11), the online part of DRIPS is significantly faster than that of POD-PROM because the size of the ROBs in

11
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 4. Left: the reference HFM, POD-PROM and DRIPS approximations of Q (t ; p(1) ) = s(xs , t ; p(1) ) for t ∈ [0.5, 1] at the training data point p(1) =
{1000, 100, 0.36}. Right: the reference HFM, POD-PROM and DRIPS approximations of Q (t ; p∗ ) = s(xs , t ; p∗ ) for t ∈ [0.5, 1] at the test data point
p∗ = {2934.0597, 209.4464, 0.4184}.

DRIPS, (m + 1) · r, is much smaller than that in POD-PROM, N sv · r with N sv  m. In this experiment, GP is more efficient
than the other two methods because the interpolation is now done in the low-dimensional parameter space, N Q = 1.
However, without any knowledge of the underlying physics, the accuracy of GP cannot compete with the physics-aware
POD-PROM.
These computational examples illustrate the benefits of the knowledge of the physics, i.e., of the specific values of A(p(i ) )
and b(p(i ) ), for the construction of accurate ROMs (as conventionally done in POD). It also demonstrates that one can
construct adequate ROMs without full knowledge of the physics, as long as enough data (temporal snapshots) are available.
By deploying DMD for data-driven learning, DRIPS compensates for the incomplete knowledge of the physics. The data-
driven learning component of DRIPS can be improved by a proper selection of the observable function g [22,21,23,24,20,56].
The following numerical examples show how to incorporate physics ingredients into the DRIPS framework.

4.2. Heat conduction in a complex domain

Consider heat conduction in a two-dimensional multi-connected domain D , an 800 × 800 square with an S-shaped
cavity in the middle (Fig. 5). The spatiotemporal distribution of temperature s(x, t ) is governed by a diffusion equation with
inhomogeneous boundary conditions,
⎧ ∂s
⎪
⎪ ρ c − k ∇ 2 s = 0, x = (x, y ) ∈ D, t ∈ (0, 5000]
⎪
⎪
⎨ ∂ t
s(x, 0) = 0; (4.10)
⎪
⎪
⎪
⎪ ∂s ∂s
⎩s(0, y , t ) = s L , s(800, y , t ) = s R , (x, 0, t ) = (x, 800, t ) = 0, s(x, t ) = 3 on ∂ S ,
∂y ∂y
where ∂ D L (green) and ∂ D R (blue) are the Dirichlet boundary segments representing the left and right sides of the com-
putational domain, respectively; and ∂ S is the bounding surface of the cavity in Fig. 5. The material properties are set to
ρ = 1, k = 1 and p = {c , s L , s R } ∈ [1, 2] × [1, 3] × [1, 3]; all defined in some consistent units. The training data and reference
solutions are obtained via the Matlab PDE toolbox on the finite-element mesh with 4287 elements (Fig. 5).
In the first set of experiments, our QoIs are the spatiotemporal distribution of temperature, Q(t n ; p) = S(t n ; p), and the
observable function g is given by (4.5). The training data are the repeated solutions of (4.10) with the three parameters p
sampled on the Cartesian grid of the cube [1, 2] × [1, 3] × [1, 3]. Each of these parameters is sampled at two endpoints and
one middle point, which yields the training data set consisting of N MC = 27 realizations of the HFM. The reference solution
of (4.10) corresponds to a randomly selected parameter value p∗ = {1.9670, 2.0945, 2.9454}. Representative temporal snap-
shots of these data are plotted in Fig. 6 together with the log absolute error map of the DRIPS estimate of Q(t n ; p ∗ ). The
HFM has the form of (4.4), as implemented in the Matlab PDE toolbox. Even in the absence of an explicit HFM formulation,
DRIPS provides an accurate ROM with the (small) rank of r = 10.
If the POD-PROM were to be used instead, one would have to know the exact values of A(p(i ) ) and b(p(i ) ) at the
Cartesian grid of the training data set, which requires large numerical differential matrices and embeddings of the complex
boundary conditions. Fig. 7a shows the accuracy of the r = 10 DRIPS trained on the Cartesian grid (shown in diamonds) for

12
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 5. Multi-connected simulation domain D (left) and the mesh used in the finite-element solution of (4.10) (right).

different values of p∗ ∈  (shown in circles). The total relative error EDRIPS (p∗ ), averaged over 100 randomly selected values
of p∗ , is 0.0041. The error EDRIPS (p∗ ) increases with the distance from the training points. It reaches its minimum at some
training data points, where the error reflects the discrepancy between the HFMs and the ROMs constructed via local ROBs
for p∗ = p(i ) .
The second set of experiments treats the spatial distribution of the heat flux as QoIs, i.e., Q1 (tn ; p) = −k∂x S(tn ; p) and
Q2 (tn ; p) = −k∂ y S(tn ; p). The heat flux, −k∇ s, for p(1) , p(2) , and p∗ , is plotted in Fig. 6 using arrows. The observable function
g is chosen as

y = g(Q1 , Q2 ) = (1, Q  
1 , Q2 ) . (4.11)

Figs. 7c and 7d exhibit the total relative error EDRIPS (p∗ ) of the DRIPS estimators (with rank r = 20) of Q1 and Q2 for
different p∗ ∈ . The total relative error EDRIPS (p∗ ), averaged over 100 randomly drawn values of p∗ , are 0.1126 and 0.0994
for Q1 and Q2 , respectively. The error’s behavior is identical to the previous case, but its magnitude is about an order of
magnitude higher.
In the final set of experiments, our QoI is the heat loss through the segment ∂ S 0 of the cavity’s surface, Q (t ; p) =
n

− ∂ S 0 k∇ s · nd A, where n is the unit normal vector to ∂ S 0 (Fig. 5). The Hermite polynomials H[ Q (t n ; p)] of order up to
m = 8, defined in (4.9), serve as the observable function g. The total relative error EDRIPS (p∗ ) of the DRIPS estimators of
Q (with rank r = 8) is shown in Fig. 7b, for different values of p∗ ∈ ; its behavior is identical to that observed in the
previous experiment, with the average (over the test data for 100 randomly drawn realizations of p∗ ) error of 0.0067. Fig. 8
demonstrates the importance of choosing the observable g by exhibiting the accuracy and computational cost of DRIPS as
functions of the polynomial order m. The left graph shows the log total relative error, averaged over the 27 training data
points and 100 randomly drawn test data points, respectively. Incorporating higher-order polynomials into the observable
results in higher-order of accuracy in both training and test datasets. The latter saturates after m = 5 as the interpolation
error becomes dominant. The right graph shows the computational costs of the offline step and online step, for different m.
The costs of both the offline and online steps increase with the polynomial order in the observable. However, the cost of
the online step is a negligible fraction of the cost of the offline step, wherein the generation of the training data dominates
the computational time.
Table 2 contains the computational cost and accuracy of DRIPS and GP, averaged over a randomly drawn N test = 100
test points. For the three experiments reported above, DRIPS is more than four times faster than the HFM computation,
while maintaining satisfactory accuracy. As discussed earlier, the generation of high-fidelity training data during the offline
step dominates the total computational cost of DRIPS. The online speedup becomes more significant when N, the size of
the surrogate model for a given QoI, is much smaller than N sv , the size of the state variable, due to the reduced size
of the expensive entry-by-entry interpolation of ROBs. As before, GP provides accurate predictions but is computationally
more expensive in the offline training process of interpolating high-dimensional data, N Q  1. GP differs from DRIPS (and
POD-PROM) in the sense that it only interpolates in the parameter space without any dimension reduction in the state/QoI
space.

4.3. Laminar flow past a cylinder

Consider two-dimensional flow of an incompressible fluid with density ρ and dynamic viscosity ν (these and other
quantities are reported in consistent units) around an impermeable circle of diameter D = 0.1. The flow, which takes place
inside a rectangular domain D = {x = (x, y ) : (x, y ) ∈ [0, 2] × [0, 1]}, is driven by an externally imposed pressure gradient;
the center of the circular inclusion is xcirc = (0.3, 0.5) . Dynamics of the three state variables, flow velocity s(x, t ) = (s1 , s2 )
and fluid pressure P (x, t ), is described by the two-dimensional Navier-Stokes equations,

13
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 6. Temporal snapshots, at times t = 500 (left column), t = 2500 (middle column) and t = 5000 (right column), of the training data generated by
solving (4.10) with parameter values p(1) = {1, 1, 1} (first row) and p(2) = {2, 3, 3} (second row). Also shown is the reference solution of (4.10) with
parameter value p∗ = {1.9670, 2.0945, 2.9454} (third row). The log absolute errors of the DRIPS estimator of this reference solution are shown in the last
row.

⎧ ∂s ∂ s2
⎪
⎪
1
+ = 0;
⎪
⎪
⎪
⎪
∂x ∂y
⎪
⎨  
∂ s1 ∂ s1 ∂ s1 1 ∂P ∂ 2 s1 ∂ 2 s1
+ s1 + s2 =− +ν + , x ∈ D, t > 0; (4.12a)
⎪ ∂t
⎪ ∂x ∂y ρ ∂x ∂ x2 ∂ y2
⎪
⎪  2 
⎪
⎪ ∂ s ∂ s ∂ s 1 ∂P ∂ s2 ∂ 2 s2
⎪
⎩ 2 + s1 2 + s2 2 = − +ν + ;
∂t ∂x ∂y ρ ∂y ∂ x2 ∂ y2
14
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 7. The log total relative

 error, ln(EDRIPS (p∗ )), of the DRIPS estimators of (a) the spatial distribution of temperature, Q = S; (b) the heat rate across the
interface ∂ S 0 , Q = − ∂ S k∇ s · nd A; and (c) and (d) the spatial distributions of the two components of heat flux, Q1 = −k∂x S and Q2 = −k∂ y S. These
0
estimates utilize DMD of rank r = 10, 8, and 20, respectively. The diamonds represent training data and the circles represent 100 test data, which are
randomly drawn from the parameter space on the Cartesian grid.

Fig. 8. (Left) Log average total relative error of the training and testing datasets and (Right) Computational costs, both plotted as function of the Hermite
polynomial order m.

subject to initial conditions s(x, y , 0) = (0, 0) and P (x, y , 0) = 0; and boundary conditions

P (2, y , t ) = 0, ∂n P |∂ D\{x=2} = 0, s(0, y , t ) = (s01 , 0) , ∂n s(2, y , t ) = 0, s(x, 0, t ) = s(x, 1, t ) = 0. (4.12b)

Here ∂n = n · ∇ , with n denoting the unit normal vector to the bounding surface. The model’s predictions are affected by
the uncertain parameter set p = {s01 , ρ , μ} ∈ [0.8, 1] × [0.9, 1.1] × [1/700, 1/500].
Our QoI is the magnitude of the flow velocity, Q (x, t ; p) = s21 + s22 , or its discretized and vectorized form Q(t n ; p) =

S1 (t n ; p)2 + S2 (t n ; p)2 . The training data (Fig. 9) and reference solution (Fig. 10) are obtained with the Matlab code [57],

15
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Table 2
The computational cost and accuracy of DRIPS estimates of several QoIs in the HFM (4.10), averaged over 100
randomly drawn test data.

HFM Q=S Q = −k∇ s Q =− k∇ s · nd A
∂ S0

GP DRIPS GP DRIPS GP DRIPS

Offline (sec) 333.51 340.84 337.16 348.24 340.23 334.75 364.55

Online (sec) 1235.24 12.47 23.76 17.48 4.50 0.07 0.16
Online speedup – 99.03 51.98 70.68 275.05 18820.65 7700.99
Total speedup – 4.44 4.35 4.29 4.55 4.69 4.30
Average E – 0.0009 0.0004 (0.28, 0.11) (0.11, 0.10) 0.0005 0.0007

Fig. 9. Training data consisting of temporal snapshots, at times t = 4.125 (left), t = 4.3125 (middle) and t = 4.5 (right), of the QoI Q computed with (4.12)
for p(1) = {0.8, 0.9, 1/500} (top) and p(2) = {1, 1.1, 1/700} (bottom).

which implements a finite-difference scheme on the staggered grid with x =  y = 0.02 and t = 0.0015. In training, we
collect N snap = 250 snapshots of Q from t = 4.125 to t = 4.5 defined on the Cartesian grid spanning the three-dimensional
cube in the parameter space. Each dimension includes two endpoints and one middle point, giving a training data set
comprising N MC = 27 HFM outputs. The training data exhibit various dynamic patterns (two examples are shown in Fig. 9),
from which DMD learns the nonlinear dynamics from a 10-rank surrogate using the same observable as in (4.5). This
construction is equivalent to the xDMD; the details of this numerical example can be found in section 4.3 of [24]. The
reference solution of a test parameter point p∗ = {0.9934, 1.0094, 0.0014}, together with the absolute error map of DRIPS,
are presented in Fig. 10.
Fig. 11 shows the accuracy of the r = 10 DRIPS and Kriging surrogates trained on the Cartesian grid (diamonds) for
different values of p∗ ∈  (circles). The total relative errors EDRIPS (p∗ ) and EGP (p∗ ), both averaged over 100 randomly drawn
test data, are reported in Table 3, together with the computational costs of the two methods. As in the previous examples,
the majority of the computational costs stems from generating the training data. The online step of DRIPS is slightly more
expensive than that of Kriging due to the entry-by-entry interpolation of ROBs, i.e., (3.11), which scales with N. Although
Kriging provides a larger online speedup, DRIPS offers the same total speedup as Kriging and has much better accuracy in
all test parameter points (Fig. 11 and Table 3).

16
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 10. Temporal snapshots, at times t = 4.125 (left column), t = 4.3125 (middle column) and t = 4.5 (right column), of the reference solution of (4.12)
with p∗ = {0.9934, 1.0094, 0.0014} (top row), and the absolute errors in its estimation via DRIPS (middle row) and Kriging (bottom row) in logarithmic
scale.

Table 3
Computational cost and accuracy of Kriging and DRIPS, averaged over
100 randomly drawn test data.

HFM Kriging DRIPS

Offline (sec) 1618.45 1643.77 1634.17

Online (sec) 5994.26 15.67 54.49
Online speedup – 382.52 110.01
Total speedup – 4.59 4.51
Average total relative error – 0.14 0.04

5. Conclusion

We proposed a new physics-aware data-driven framework, DRIPS, to construct reduced-order models of parametric
complex systems. DRIPS combines the advantages of the data-driven modeling tool DMD and ROBs used for manifold inter-
polations in the parameter space. Attractive features of DRIPS include

17
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Fig. 11. (Left) Log total relative error of the DRIPS estimator, ln(EDRIPS (p∗ )); (Middle) log total relative error of the Kriging estimators, ln(EGP (p∗ )); and
(Right) histogram of the log total relative error of the two estimators, for several test values of the parameter set p and the 27 training values of the
parameter set on the Cartesian grid.

• the ability to handle QoIs directly, without having to access the HFMs for the underlying high-dimensional state vari-
ables, as required in projection-based PROM methods;
• the improved accuracy relative to that of the conventional pure-data driven techniques such as Kriging (Gaussian pro-
cesses); and
• the low training data requirements relative to data-intensive nonlinear machine-learning methods such as deep neural
networks, which require O (106 ) training data points.

Our numerical experiments suggest that DRIPS is a robust and flexible framework, which bridges the understanding of data
and physics. The general framework can be applied to many applications and be extended to higher parameter dimensions
with adaptive sampling strategy. Compared to expensive high-fidelity simulations, the significant speedup provided by DRIPS
makes it is suitable for real-time processing.
In the follow-up studies we will develop the current framework in several directions. First, to further improve the
online speedup, we will explore more efficient manifold interpolation methods, e.g., [36]. Second, for problems with high-
dimensional parameter spaces, we will design adaptive sampling strategies and combine DRIPS with parameter space re-
duction techniques [58]. Third, further studies on error estimation DRIPS are needed in order to construct reliable surrogate
models, which may further enable outer-loop applications such as design, inverse problems, optimization and uncertainty
quantification.

CRediT authorship contribution statement

Hannah Lu: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visu-
alization, Writing – original draft. Daniel M. Tartakovsky: Conceptualization, Funding acquisition, Methodology, Project
administration, Supervision, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Data availability

No data was used for the research described in the article.

Acknowledgements

The research was partially supported in part by the Air Force Office of Scientific Research under grant FA9550-21-1-0381,
by the National Science Foundation under award 2100927, by the Office of Advanced Scientific Computing Research (ASCR)
within the Department of Energy Office of Science under award number DE-SC0023163, and by the Strategic Environmental
Research and Development Program (SERDP) of the Department of Defense under award RC22-3278.

18
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

Appendix A. Implementation of the POD-PROM framework [33]

Algorithm 7: POD-PROM framework for linear system (4.4) and (4.7).

Offline Step:
For i = 1, . . . , N MC ,

1. Compute the HFM (4.4)/(4.7), collect {S(t 0 ; p(i ) ), . . . , S(t N snap ; p(i ) )}, A(p(i ) ), b(p(i ) ) and C,
2. Apply SVD [S(t 0 ; p(i ) ) , . . . , S(t N snap ; p(i ) ) ] ≈ V(p(i ) )(p(i ) )Z(p(i ) ) with V(p(i ) ) ∈ C N ×r , (p(i ) ) ∈ C r ×r , Z(p(i ) ) ∈ C r × N snap , where r is the trun-
cated rank chosen by certain criteria and should be the same for all i = 1, . . . , N MC .
3. Use Galerkin-projection to compute local ROMs:

Ar (p(i ) ) = V(p(i ) ) A(p(i ) )V(p(i ) ), br (p(i ) ) = V(p(i ) ) b(p(i ) ),

(A.1)
Cr (p(i ) ) = CV(p(i ) ),

4. Compute P(i , j ) = V(p(i ) ) V(p( j ) ) for i , j = 1, . . . , N MC .

Output: V(p(i ) ), P(i , j ) , Ar (p(i ) ), br (p(i ) ) and Cr (p(i ) ) .

End
Online Step:

• Interpolation of ROBs:
N N N Algorithm 2
(i , j ) } MC , {p(i ) } MC , p∗ −−−−−−→ Output: V(p∗ )
Input: {V(p(i ) )}i =MC
1 , {P i , j =1 i =1
• Interpolation of PROMs:
i , j ) N MC
Input: {Ar (p ), br (p ), Cr (p ), P
( i) ( i) ( i) ( }i , j =1 , reference choice i 0 ,
– Compute P(i ,i 0 ) = Ui ,i 0 i ,i 0 Zi ,i (SVD), Qi = Ui ,i 0 Zi ,i ,
0 0

– Transform Ãr (p(i ) ) = Qi Ar (p(i ) )Qi , b̃r (p(i ) ) = Qi br (p(i ) ) and C̃r (p(i ) ) = Cr (p(i ) )Qi .
N Algorithm 4 ∗ ∗ ∗
– Input: {Ãr (p(i ) ), b̃r (p(i ) ), C̃r (p(i ) )}i , MC
j =1 , i 0 −−−−−−→ Output: {Ar (p ), br (p ), Cr (p )}.
• POD reconstruction:
Input: Ar (p∗ ), br (p∗ ), V(p∗ ) and S(t 0 ; p∗ ),
q(t 0 ; p∗ ) = V(p∗ ) S(t 0 ; p∗ )
For k = 1, . . . , N T ,
q(tk ; p∗ ) = Ar (p∗ )q(tk−1 ; p∗ ) + br (p∗ ), QPOD (tk ; p∗ ) = Cr (p∗ )q(tk ; p∗ ),
End
Output: QPOD (tk ; p∗ ).

References

[1] J.L. Lumley, The structure of inhomogeneous turbulent flows, in: A.M. Yaglom, V.I. Tartarsky (Eds.), Atmospheric Turbulence and Radio Wave Propaga-
tion, Nauka, Moscow, USSR, 1967, pp. 166–177.
[2] G. Kerschen, J.-C. Golinval, A.F. Vakakis, L.A. Bergman, The method of proper orthogonal decomposition for dynamical characterization and order
reduction of mechanical systems: an overview, Nonlinear Dyn. 41 (1) (2005) 147–169.
[3] B. Haasdonk, Reduced basis methods for parametrized PDEs—a tutorial introduction for stationary and instationary problems, in: Model Reduction and
Approximation Theory and Algorithms, vol. 15, 2017, p. 65.
[4] S. Chaturantabut, D.C. Sorensen, Nonlinear model reduction via discrete empirical interpolation, SIAM J. Sci. Comput. 32 (5) (2010) 2737–2764.
[5] C.W. Rowley, Model reduction for fluids, using balanced proper orthogonal decomposition, Int. J. Bifurc. Chaos 15 (03) (2005) 997–1013.
[6] B. Moore, Principal component analysis in linear systems: controllability, observability, and model reduction, IEEE Trans. Autom. Control 26 (1) (1981)
17–32.
[7] S. Gugercin, A.C. Antoulas, C. Beattie, H_2 model reduction for large-scale linear dynamical systems, SIAM J. Matrix Anal. Appl. 30 (2) (2008) 609–638.
[8] A.C. Antoulas, C.A. Beattie, S. Güğercin, Interpolatory Methods for Model Reduction, SIAM, 2020.
[9] K. Lee, K.T. Carlberg, Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys. 404 (2020)
108973.
[10] C.E. Rasmussen, Gaussian processes in machine learning, in: Summer School on Machine Learning, Springer, 2003, pp. 63–71.
[11] G.S.H. Pau, Y. Zhang, S. Finsterle, Reduced order models for many-query subsurface flow applications, Comput. Geosci. 17 (4) (2013) 705–721.
[12] D.J. Booker, R.A. Woods, Comparing and combining physically-based and empirically-based approaches for estimating the hydrology of ungauged
catchments, J. Hydrol. 508 (2014) 227–239.
[13] S.A. Naghibi, H.R. Pourghasemi, B. Dixon, GIS-based groundwater potential mapping using boosted regression tree, classification and regression tree,
and random forest machine learning models in Iran, Environ. Monit. Assess. 188 (1) (2016) 1–27.
[14] P.J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech. 656 (2010) 5–28.
[15] J.N. Kutz, S.L. Brunton, B.W. Brunton, J.L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016.
[16] B. Peherstorfer, K. Willcox, Data-driven operator inference for nonintrusive projection-based model reduction, Comput. Methods Appl. Mech. Eng. 306
(2016) 196–215.
[17] S.A. McQuarrie, P. Khodabakhshi, K.E. Willcox, Non-intrusive reduced-order models for parametric partial differential equations via data-driven operator
inference, SIAM J. Sci. Comput. 45 (4) (2023) A1917–A1946, [Link]
[18] J.S. Hesthaven, S. Ubbiali, Non-intrusive reduced order modeling of nonlinear problems using neural networks, J. Comput. Phys. 363 (2018) 55–78.
[19] T. Qin, K. Wu, D. Xiu, Data driven governing equations approximation using deep neural networks, J. Comput. Phys. 395 (2019) 620–635.
[20] J.N. Kutz, J.L. Proctor, S.L. Brunton, Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems,
Complexity (2018) 2018.
[21] H. Lu, D.M. Tartakovsky, Lagrangian dynamic mode decomposition for construction of reduced-order models of advection-dominated phenomena, J.
Comput. Phys. (2020) 109229.

19
H. Lu and D.M. Tartakovsky Journal of Computational Physics 493 (2023) 112455

[22] H. Lu, D.M. Tartakovsky, Prediction accuracy of dynamic mode decomposition, SIAM J. Sci. Comput. 42 (3) (2020) A1639–A1662.
[23] H. Lu, D.M. Tartakovsky, Dynamic mode decomposition for construction of reduced-order models of hyperbolic problems with shocks, J. Mach. Learn.
Model. Comput. 2 (1) (2021).
[24] H. Lu, D.M. Tartakovsky, Extended dynamic mode decomposition for inhomogeneous problems, J. Comput. Phys. 444 (2021) 110550.
[25] G.E. Karniadakis, I.G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, L. Yang, Physics-informed machine learning, Nat. Rev. Phys. 3 (6) (2021) 422–440.
[26] Y. Zhu, N. Zabaras, P.-S. Koutsourelakis, P. Perdikaris, Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty
quantification without labeled data, J. Comput. Phys. 394 (2019) 56–81.
[27] E. Qian, B. Kramer, B. Peherstorfer, K. Willcox, Lift & Learn: physics-informed machine learning for large-scale nonlinear dynamical systems, Phys. D:
Nonlinear Phenom. 406 (2020) 132401.
[28] B.I. Epureanu, A parametric analysis of reduced order models of viscous flows in turbomachinery, J. Fluids Struct. 17 (7) (2003) 971–982.
[29] C. Homescu, L.R. Petzold, R. Serban, Error estimation for reduced-order models of dynamical systems, SIAM J. Numer. Anal. 43 (4) (2005) 1693–1714.
[30] R. Serban, C. Homescu, L.R. Petzold, The effect of problem perturbations on nonlinear dynamical systems and their reduced-order models, SIAM J. Sci.
Comput. 29 (6) (2007) 2621–2643.
[31] T. Lieu, C. Farhat, Adaptation of aeroelastic reduced-order models and application to an F-16 configuration, AIAA J. 45 (6) (2007) 1244–1257.
[32] D. Amsallem, C. Farhat, Interpolation method for adapting reduced-order models and application to aeroelasticity, AIAA J. 46 (7) (2008) 1803–1813.
[33] D. Amsallem, C. Farhat, An online method for interpolating linear parametric reduced-order models, SIAM J. Sci. Comput. 33 (5) (2011) 2169–2198.
[34] N.T. Son, A real time procedure for affinely dependent parametric model order reduction using interpolation on Grassmann manifolds, Int. J. Numer.
Methods Eng. 93 (8) (2013) 818–833.
[35] R. Zimmermann, A locally parametrized reduced-order model for the linear frequency domain approach to time-accurate computational fluid dynamics,
SIAM J. Sci. Comput. 36 (3) (2014) B508–B537.
[36] R. Zhang, S. Mak, D. Dunson, Gaussian process subspace regression for model reduction, SIAM J. Sci. Comput. 44 (3) (2022) A1428–A1449.
[37] H. Lu, D. Ermakova, H.M. Wainwright, L. Zheng, D.M. Tartakovsky, Data-informed emulators for multi-physics simulations, J. Mach. Learn. Model.
Comput. 2 (2) (2021) 33–54.
[38] K. Bhattacharya, B. Hosseini, N.B. Kovachki, A.M. Stuart, Model reduction and neural networks for parametric PDEs, SMAI J. Comput. Math. 7 (2021)
121–157.
[39] P. Sentz, K. Beckwith, E.C. Cyr, L.N. Olson, R. Patel, Reduced basis approximations of parameterized dynamical partial differential equations via neural
networks, arXiv preprint arXiv:2110.10775, 2021.
[40] S.L. Brunton, B.W. Brunton, J.L. Proctor, J.N. Kutz, Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for
control, PLoS ONE 11 (2) (2016) e0150171.
[41] J.H. Tu, Dynamic mode decomposition: Theory and applications, Ph.D. thesis, Princeton University, 2013.
[42] M. Schmidt, H. Lipson, Distilling free-form natural laws from experimental data, Science 324 (5923) (2009) 81–85.
[43] W.-X. Wang, R. Yang, Y.-C. Lai, V. Kovanis, C. Grebogi, Predicting catastrophes in nonlinear dynamical systems by compressive sensing, Phys. Rev. Lett.
106 (15) (2011) 154101.
[44] Q. Li, F. Dietrich, E.M. Bollt, I.G. Kevrekidis, Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decom-
position of the Koopman operator, Chaos, Interdiscip. J. Nonlinear Sci. 27 (10) (2017) 103111.
[45] P.-A. Absil, R. Mahony, R. Sepulchre, Riemannian geometry of Grassmann manifolds with a view on algorithmic computation, Acta Appl. Math. 80 (2)
(2004) 199–220.
[46] W.M. Boothby, W.M. Boothby, An Introduction to Differentiable Manifolds and Riemannian Geometry, Revised, vol. 120, Gulf Professional Publishing,
2003.
[47] S. Helgason, Differential Geometry, Lie Groups, and Symmetric Spaces, Graduate Texts in Mathematics, AMS, 2001.
[48] I.U. Rahman, I. Drori, V.C. Stodden, D.L. Donoho, P. Schröder, Multiscale representations for manifold-valued data, Multiscale Model. Simul. 4 (4) (2005)
1201–1232.
[49] A. Edelman, T.A. Arias, S.T. Smith, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl. 20 (2) (1998) 303–353.
[50] D. Carmo, M. Perdigao, F.J. Flaherty, Riemannian Geometry, vol. 6, Springer, 1992.
[51] R. Zimmermann, Manifold interpolation, in: P. Benner, S. Grivet-Talocia, A. Quarteroni, G.R.W. Schilders, L.M. Silveira (Eds.), Model Order Reduction, vol.
1 System- and Data-Driven Methods and Algorithms, De Gruyter, 2021, pp. 229–274, Ch. 7.
[52] H. Späth, One Dimensional Spline Interpolation Algorithms, AK Peters/CRC Press, 1995.
[53] C. De Boor, A. Ron, Computational aspects of polynomial interpolation in several variables, Math. Comput. 58 (198) (1992) 705–727.
[54] C.F. Van Loan, G. Golub, Matrix Computations, 3rd edition, Johns Hopkins Studies in Mathematical Sciences, Johns Hopkins University Press, 1996.
[55] D.J. Ewins, Modal Testing: Theory, Practice and Application, John Wiley & Sons, 2009.
[56] M.O. Williams, I.G. Kevrekidis, C.W. Rowley, A data–driven approximation of the Koopman operator: extending dynamic mode decomposition, J. Non-
linear Sci. 25 (6) (2015) 1307–1346.
[57] J. Johns, A Matlab code for numerical solution of Navier-Stokes equations for two-dimensional incompressible flow (velocity-pressure formulation)
along with ability for importing custom scenarios for the fluid flow, [Link]
flow-with-custom-scenarios-MATLAB-, 2018.
[58] C. Lieberman, K. Willcox, O. Ghattas, Parameter and state model reduction for large-scale statistical inverse problems, SIAM J. Sci. Comput. 32 (5)
(2010) 2523–2542.

20