Machine Learning In Python For Dynamic Process

Systems Ankur Kumar download

https://ebookbell.com/product/machine-learning-in-python-for-
dynamic-process-systems-ankur-kumar-52689944

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Machine Learning In Python For Process Systems Engineering Ankur Kumar

Jesus Florescerrillo

https://ebookbell.com/product/machine-learning-in-python-for-process-
systems-engineering-ankur-kumar-jesus-florescerrillo-47411856

Machine Learning In Python Essential Techniques For Predictive

Analysis Michael Bowles

https://ebookbell.com/product/machine-learning-in-python-essential-
techniques-for-predictive-analysis-michael-bowles-36155298

Machine Learning In Python And R For Dummies John Paul Mueller

https://ebookbell.com/product/machine-learning-in-python-and-r-for-
dummies-john-paul-mueller-232123742

Natural Language Processing In Python Master Data Science And Machine

Learning For Spam Detection Sentiment Analysis Latent Semantic
Analysis And Article Spinning Machine Learning In Python
Lazyprogrammer
https://ebookbell.com/product/natural-language-processing-in-python-
master-data-science-and-machine-learning-for-spam-detection-sentiment-
analysis-latent-semantic-analysis-and-article-spinning-machine-
learning-in-python-lazyprogrammer-5481600
Statistics Data Mining And Machine Learning In Astronomy A Practical
Python Guide For The Analysis Of Survey Data Course Book Eljko Ivezi
Andrew J Connolly Jacob T Vanderplas Alexander Gray

https://ebookbell.com/product/statistics-data-mining-and-machine-
learning-in-astronomy-a-practical-python-guide-for-the-analysis-of-
survey-data-course-book-eljko-ivezi-andrew-j-connolly-jacob-t-
vanderplas-alexander-gray-51954088

Statistics Data Mining And Machine Learning In Astronomy A Practical

Python Guide For The Analysis Of Survey Data Updated Edition Eljko
Ivezi Andrew J Connolly Jacob T Vanderplas Alexander Gray

https://ebookbell.com/product/statistics-data-mining-and-machine-
learning-in-astronomy-a-practical-python-guide-for-the-analysis-of-
survey-data-updated-edition-eljko-ivezi-andrew-j-connolly-jacob-t-
vanderplas-alexander-gray-51955478

Statistics Data Mining And Machine Learning In Astronomy A Practical

Python Guide For The Analysis Of Survey Data Eljko Ivezi Andrew
Connolly Jake Vanderplas Alexander Gray

https://ebookbell.com/product/statistics-data-mining-and-machine-
learning-in-astronomy-a-practical-python-guide-for-the-analysis-of-
survey-data-eljko-ivezi-andrew-connolly-jake-vanderplas-alexander-
gray-33573214

Machine Learning For Emotion Analysis In Python Allan Ramsay

https://ebookbell.com/product/machine-learning-for-emotion-analysis-
in-python-allan-ramsay-56305226

Machine Learning For Decision Sciences With Case Studies In Python S

Sumathi

https://ebookbell.com/product/machine-learning-for-decision-sciences-
with-case-studies-in-python-s-sumathi-43242592
 ML for Process Industry Series

Machine Learning in Python

for Dynamic Process Systems
A practitioner’s guide for building process modeling,
predictive, and monitoring solutions using dynamic data

First Edition

Ankur Kumar, Jesus Flores-Cerrillo

`

Machine Learning in Python

for Dynamic Process
Systems

A practitioner’s guide for building process modeling,

predictive, and monitoring solutions using dynamic
data

Ankur Kumar
Jesus Flores-Cerrillo
`

Dedicated to our spouses, family, friends, motherland, and all the data-science enthusiasts
`

न चोर हार्यं न च राज हार्यं न भात्रू भाज्यं न च भारकारर

व्यर्यं कृते वर्धत एव ननत्यं नवद्यार्नं सवधर्नप्रर्ानम

No one can steal it, no king can snatch it,

It cannot be divided among the brothers and it’s not heavy to carry,
As you consume or spend, it increases; as you share, it expands,
The wealth of knowledge is the most precious wealth you can have.

- A popular Sanskrit shloka

`

Machine Learning in Python for Dynamic Process Systems

www.MLforPSE.com

Copyright © 2023 Ankur Kumar

All rights reserved. No part of this book may be reproduced or transmitted in any form or in
any manner without the prior written permission of the authors.
.

Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented and obtain permissions for usage of copyrighted materials.
However, the authors make no warranties, expressed or implied, regarding errors or
omissions and assume no legal liability or responsibility for loss or damage resulting from
the use of information contained in this book.

Plant image on cover page obtained from https://pixabay.com/.

.

To request permissions, contact the authors at [email protected]

First published: June 2023

`

About the Authors

Ankur Kumar holds a PhD degree (2016) in Process Systems Engineering

from the University of Texas at Austin and a bachelor’s degree (2012) in
Chemical Engineering from the Indian Institute of Technology Bombay. He
currently works at Linde in the Advanced Digital Technologies & Systems
Group in Linde’s Center of Excellence, where he has developed several in-
house machine learning-based monitoring and process control solutions for
Linde’s hydrogen and air-separation plants. Ankur’s tools have won several
awards both within and outside Linde. Most recently, one of his tools,
PlantWatch (a plantwide fault detection and diagnosis tool), received the 2021 Industry 4.0
Award by the Confederation of Industry of the Czech Republic. Ankur has authored or co-
authored more than 10 peer-reviewed journal papers (in the areas of data-driven process
modeling and optimization), is a frequent reviewer for many top-ranked research journals,
and has served as Session Chair at several international conferences. Ankur also served as
an Associate Editor of the Journal of Process Control from 2019 to 2021.

Jesus Flores-Cerrillo is currently an Associate Director - R&D at Linde and

manages the Advanced Digital Technologies & Systems Group in Linde’s
Center of Excellence. He has over 20 years of experience in the development
and implementation of monitoring technologies and advanced process
control & optimization solutions. Jesus holds a PhD degree in Chemical
Engineering from McMaster University and has authored or co-authored
more than 40 peer-reviewed journal papers in the areas of multivariate
statistics and advanced process control among others. His team develops and implements
novel plant monitoring, machine learning, IIOT solutions to improve the efficiency and
reliability of Linde’s processes. Jesus’s team received the Artificial Intelligence and Advanced
Analytics Leadership 2020 award from the National Association of Manufacturers’
Manufacturing Leadership Council.
`

Note to the readers

Jupyter notebooks and Spyder scripts with complete code implementations are available for
download at https://github.com/ML-PSE/Machine_Learning_for_DPS. Code updates when
necessary, will be made and updated on the GitHub repository. Updates to the book’s text
material will be available on Leanpub (www.leanpub.com) and Google Play
(https://play.google.com/store/books). We would greatly appreciate any information about
any corrections and/or typos in the book.
`

Series Introduction
In the 21st century, data science has become an integral part of the work culture at every
manufacturing industry and process industry is no exception to this modern phenomenon.
From predictive maintenance to process monitoring, fault diagnosis to advanced process
control, machine learning-based solutions are being used to achieve higher process reliability
and efficiency. However, few books are available that adequately cater to the needs of
budding process data scientists. The scant available resources include: 1) generic data
science books that fail to account for the specific characteristics and needs of process plants
2) process domain-specific books with rigorous and verbose treatment of underlying
mathematical details that become too theoretical for industrial practitioners. Understandably,
this leaves a lot to be desired. Books are sought that have process systems in the backdrop,
stress application aspects, and provide a guided tour of ML techniques that have proven
useful in process industry. This series ‘Machine Learning for Process Industry’ addresses
this gap to reduce the barrier-to-entry for those new to process data science.

The first book of the series ‘Machine Learning in Python for Process Systems
Engineering’ covers the basic foundations of machine learning and provides an overview of
broad spectrum of ML methods primarily suited for static systems. Step-by-step guidance on
building ML solutions for process monitoring, soft sensing, predictive maintenance, etc. are
provided using real process datasets. Aspects relevant to process systems such as modeling
correlated variables via PCA/PLS, handling outliers in noisy multidimensional dataset,
controlling processes using reinforcement learning, etc. are covered. This second book of
the series is focused on dynamic systems and provides a guided tour along the wide range
of available dynamic modeling choices. Emphasis is paid to both the classical methods (ARX,
CVA, ARMAX, OE, etc.) and modern neural network methods. Applications on time series
analysis, noise modeling, system identification, and process fault detection are illustrated
with examples. Future books of the series will continue to focus on other aspects and needs
of process industry. It is hoped that these books can help process data scientists find
innovative ML solutions to the real-world problems faced by the process industry.

Books of the series will be useful to practicing process engineers looking to ‘pick up’ machine
learning as well as data scientists looking to understand the needs and characteristics of
process systems. With the focus on practical guidelines and real industrial case studies, we
hope that these books lead to wider spread of data science in the process industry.
`

Previous book(s) from the series

(https://MLforPSE.com/books/)
`

Preface
Model predictive control (MPC) and real-time optimization (RTO) are among the most critical
technologies that drive the process industry. Any experienced process engineer would vouch
that the success of MPCs and RTOs depend heavily on the accuracy of the underlying
process models. Similarly, the key requirement for other important process technologies like
dynamic data reconciliation, process monitoring, etc. is availability of accurate process
models. Modeling efforts during commissioning of these tools can easily consume up to 90%
of the project cost and time. Modern data revolution, whereby abundant amount of process
data are easily available, and practical difficulties in building high-fidelity first-principles
dynamic models for complex industrial processes have popularized the usage of empirical
data-driven/machine learning (ML) models. Building dynamic models using process data is
called system identification (SysID) and it’s a very mature field with extensive literature.
However, it is also very easy for a process data scientist (PDS) new to this field to get
overwhelmed with the SysID mathematics and ‘drowned’ in the sea of SysID terminology.
Several noteworthy ML books have been written on time-series analysis; however, in process
industry, input-output models are of greater import. Unfortunately, there aren’t many books
that cater to the needs of modern PDSs interested in dynamic process modeling (DPM)
without weighing them down with too much mathematical details and therein lies our
motivation for authoring this book: specifically, a reader-friendly and easy to understand book
that provides a comprehensive coverage of ML techniques that have proven useful for
building dynamic process models with focus on practical implementations.

It would be clear to you by now that this book is designed to teach working process engineers
and budding PDSs about DPM. While doing so, this book attempts to avoid a pitfall that
several generic ML books fall into: overemphasis on ‘modern’ and complex ML techniques
such as artificial neural networks (ANNs) and undertreatment of classical DPM methods.
Classical techniques like FIR and ARX still dominate the dynamic modeling solutions offered
by commercial vendors of industrial solutions and are no less ‘machine-learning’ than the
ANNs. These two along with other classical techniques (such as OE, ARIMAX, CVA) predate
the ANN-craze era and have stood the test of time in providing equal (if not superior)
performance compared to ANNs. Correspondingly, along with modern ML techniques like
RNNs, considerable portion of the book is devoted to classical dynamic models with the
emphasis on understanding the implications of modeling decisions such as the impact of
implicit noise model assumption with ARX model, the implication of differencing data, etc.
`

Guided by our own experience from building process models for varied industrial applications
over the past several years, this book covers a curated set of ML techniques that have proven
useful for DPM. The broad objectives of the book can be summarized as follows:
• reduce barrier-to-entry for those new to the field of SysID
• provide working-level knowledge of SysID techniques to the readers
• enable readers to make judicious selection of a SysID technique appropriate for their
problems through intuitive understanding of the advantages and drawbacks of
different methods
• provide step-by-step guidance for developing tools for soft sensing, process
monitoring, predictive maintenance, etc. using SysID methods

This book adopts a tutorial-style approach. The focus is on guidelines and practical
illustrations with a delicate balance between theory and conceptual insights. Hands-on-
learning is emphasized and therefore detailed code examples with industrial-scale datasets
are provided to concretize the implementation details. A deliberate attempt is made to not
weigh readers down with mathematical details, but rather use it as a vehicle for better
conceptual understanding. Complete code implementations have been provided in the
GitHub repository. Although most of the existing literature on SysID use MATLAB as the
programming environment, we have adopted Python in this book due to its immense
popularity among the broad ML community. Several Python libraries are now available which
makes DPM using Python convenient.

We are quite confident that this text will enable its readers to build dynamic models for
challenging problems with confidence. We wish them the best of luck in their career.

Who should read this book

The application-oriented approach in this book is meant to give a quick and comprehensive
coverage of dynamic modeling methodologies in a coherent, reader-friendly, and easy-to-
understand manner. The following categories of readers will find the book useful:

1) Data scientists new to the field of system identification

2) Regular users of commercial process modeling software looking to obtain a deeper
understanding of the underlying concepts
3) Practicing process data scientists looking for guidance for developing process modeling
and monitoring solutions for dynamic systems
4) Process engineers or process engineering students making their entry into the world of
data science
`

Pre-requisites
No prior experience with machine learning or Python is needed. Undergraduate-level
knowledge of basic linear algebra and calculus is assumed.

Book organization
Under the broad theme of ML for process systems engineering, this book is an extension of
the first book of the series (which dealt with fundamentals of ML and its varied applications
in process industry); however, it can also be used as a standalone text. To give due treatment
to various aspects of SysID, the book has been divided into three parts. Part 1 of the book
provides a perspective on the importance of ML for dynamic process modeling and lays down
the basic foundations of ML-DPM (machine learning for dynamic process modeling). Part 2
provides in-detail presentation of classical ML techniques and has been written keeping in
mind the different modeling requirements and process characteristics that determine a
model’s suitability for a problem at hand. These include, amongst others, presence of
multiple correlated outputs, process nonlinearity, need for low model bias, need to model
disturbance signal accurately, etc. Part 3 is focused on artificial neural networks and deep
learning. While deep learning is the current buzzword in ML community, we would like to
caution the reader against the temptation to deploy a deep learning model for every problem
at hand. For example, the models covered in Part 2 still dominate the portfolio of models
used in industrial controllers and can often provide comparable (or even superior)
performance compared to ANNs with relatively less hassle.

Symbol notation
The following notation has been adopted in the book for representing different types of
variables:
- lower-case letters refer to vectors (𝑥 ∈ ℝ𝑚×1 ) and upper-case letters denote matrices
(𝑋 ∈ ℝ𝑛×𝑚 )
- individual element of a vector and a matrix are denoted as 𝑥𝑗 and 𝑥𝑖𝑗 , respectively.
- any ith vector in a dataset gets represented as subscripted lower-case letter (e.g., 𝑥𝑖 ∈
ℝ𝑚×1 ). Its distinction from an individual element 𝑥𝑗 would be clear from the
corresponding context.
`

Installing Required Packages

In this book, SIPPY package (https://github.com/CPCLAB-UNIPI/SIPPY) is used for classical
dynamic modeling of input-output systems. Below are a few remarks regarding the
installation of the package
▪ The package code can be downloaded from its GitHub repository. You may put the
‘sippy’ folder provided in the package’s distribution in your working directory to use the
provided models
▪ To run the code provided in this book, Slycot package is not required
▪ Control package’s version should be < 0.9 (https://github.com/CPCLAB-
UNIPI/SIPPY/issues/48)
`

Table of Contents

• Part 1 Introduction and Fundamentals

• Chapter 1: Machine Learning and Dynamic Process Modeling 1

1.1. Process Systems Engineering, Dynamic Process Modeling, and Machine Learning
-- components of a dynamic process model
1.2. ML-DPM Workflow
1.3. Taxonomy of ML-based Dynamic Models
1.4. Applications of DPM in Process Industry

• Chapter 2: The Scripting Environment 15

2.1. Introduction to Python

2.2. Introduction to Spyder and Jupyter
2.3. Python Language: Basics
2.4. Scientific Computing Packages: Basics
-- Numpy
-- Pandas
2.5. SysID-relevant Python Libraries
2.6. Typical SysID Script

• Chapter 3: Exploratory Analysis and Visualization of Dynamic Dataset:

Graphical Tools 35

3.1. Visual Plots: Simple Yet Powerful Tools

3.2. Autocorrelation Function (ACF)
-- Examples on inferences drawn using ACF
3.3. Partial Autocorrelation Function (PACF)
3.4. Cross-correlation Function (CCF)
-- Examples on inferences drawn using CCF
3.5. Power Spectral Density (PSD) and Periodogram

• Chapter 4: Machine Learning-Based Dynamic Modeling:

Workflow and Best Practices 48

4.1. System Identification Workflow

4.2. Identification Test/Input Signal Design
-- Pseudo Random Binary Sequence (PRBS)
-- Generalized Binary Noise (GBN)
`

4.3. Data Pre-processing

-- (Measurement) Noise removal
-- Centering and scaling
-- Trend and drift removal
4.4. Model Structure Selection
4.5. Model ID and Performance Assessment
-- Model order selection
4.6. Model Quality Check and Diagnostics
-- Residual analysis
-- Transient response checks
-- Simulation response checks
-- Parameter error checks

• Part 2 Classical Machine Learning Methods for Dynamic Modeling

• Chapter 5: Time Series Analysis: Concepts and Applications 76

5.1. Time Series Analysis: An Introduction

5.2. Autoregressive (AR) Models: An Introduction
-- Model order selection
5.3. Moving Average (MA) Models: An Introduction
-- Model order selection
5.4. Autoregressive Moving Average (ARMA) Models: An Introduction
-- Model order selection
5.5. Monitoring Controlled Variables in a CSTR using ARMA Models
5.6. Autoregressive Integrated Moving Average (ARIMA) Models: An Introduction
-- Model order selection
5.7. Forecasting Signals using ARIMA

• Chapter 6: Input-Output Modeling – Part 1:

Simple Yet Popular Classical Linear Models 100

6.1. FIR Models: An Introduction

6.2. FIR Modeling of Industrial Furnaces
6.3. ARX Models: An Introduction
6.4. FIR Modeling of Industrial Furnaces
6.5. FIR and ARX MIMO Models

• Chapter 7: Input-Output Modeling – Part 2:

Handling Process Noise the Right Way 120

7.1. PEM Models

7.2. ARMAX Models: An Introduction
7.3. ARMAX Modeling of Distillation Columns
`

7.4. OE Models: An Introduction

7.5. Box-Jenkins Models: An Introduction
7.6. ARIMAX Models: An Introduction

• Chapter 8: State-Space Models: Efficient Modeling of MIMO Systems 138

8.1. State-Space Models: An Introduction

8.2. State-Space Modeling via CVA
-- Mathematical background
-- Hyperparameter selection
8.3. Modeling Glass Furnaces via CVA
8.4. Monitoring Industrial Chemical Plants using CVA
-- Process monitoring/fault detection indices

• Chapter 9: Nonlinear System Identification: Going Beyond Linear Models 158

9.1. Nonlinear System Identification: An Introduction

9.2. NARX Models: An Introduction
9.3. Nonlinear Identification of a Heat Exchanger Process Using NARX
9.4. Introduction to Block-Structured Nonlinear Models

• Addendum A1: Closed-Loop Identification: Modeling Processes Under Feedback 170

• Part 3 Artificial Neural Networks & Deep Learning

• Chapter 10: Artificial Neural Networks:

Handling Complex Nonlinear Systems 174

10.1. ANN: An Introduction

10.2. Modeling Heat Exchangers using FFNN-NARX
10.3. RNN: An Introduction
-- RNN outputs
-- RNN-based NARX topology
-- LSTM Networks
10.4. Modeling Heat Exchangers using LSTM
`

Part 1
Introduction & Fundamentals
 Chapter 1
Machine Learning and Dynamic Process
Modeling: An Introduction

P
rocess industry operations are dynamic in nature. In complex process plants such as
oil refineries, 1000s of process measurements may be recorded every second to
capture crucial process trends. Plant engineers often employ dynamic process models
to predict future values of process variables in applications such as process control, process
monitoring, etc. Machine learning (ML) provides a convenient mechanism to bring together
the dynamic process modeling (DPM) needs and the large data resources available in modern
process plants.

This chapter provides an overview of what ML has to offer for DPM. This chapter also
addresses a dichotomy between ML community and DPM community. While the former
generally tends to claim that ML has been reduced to mere execution of ‘model-fitting’ actions,
the later vouches for the need of ‘experts’ to build ‘successful’ models. We will attempt to
reconcile these two differing viewpoints.

Overall, this chapter provides a whirlwind tour of how the power of machine learning is
harnessed for dynamic process modeling. Specifically, the following topics are covered
• Introduction to dynamic process modeling and the need for machine learning
• Typical workflow in a ML-based DPM (ML-DPM) project
• Taxonomy of popular ML-DPM methods and models
• Applications of DPM in process industry

Let’s now tighten our seat-belts as we embark upon this exciting journey of de-mystifying
machine learning for dynamic process modeling.

1
`

1.1 Process Systems Engineering, Dynamic

Process Modeling, and Machine Learning

Process industry is a parent term used to refer to industries like petrochemical, chemical,
power, paper, cement, pharmaceutical, etc. These industries use processing plants to
manufacture intermediate or final consumer products. As emphasized in Figure 1.1, the prime
concern of the management of these plants is optimal design and operations, and high
reliability through proactive process monitoring, quality control, data reconciliation, etc. All
these tasks fall under the ambit of process systems engineering (PSE).

Figure 1.1: Overview of industries that constitute process industry and the tasks process
systems engineers perform

The common theme among the PSE tasks is that they all rely on reliable dynamic
mathematical process models that can accurately predict the future state of the process using
current and past process measurements. Therefore, DPM is one of the defining skills of
process systems engineers. Process industry has historically utilized both first
principles/phenomenological and empirical/data-based models for DPM. While the former
models provide higher fidelity, the later models are easier to build for complex systems. The
immense rise in popularity of machine learning/data science in recent years and exponential
increase in sensor measurements collected at plants have led to renewed interest in ML-
based DPM. Several classical and ‘modern’ ML methods for DPM are at a process data
scientist’s disposal. The rest of the book will take you on a whirlwind tour of these methods.
Let’s now jump straight into the nitty-gritty of ML-DPM.
2
 `

Components of a dynamic process model

In the context of DPM, there are three types of variables/signals in a process system as shown
in Figure 1.2. The measured signals that we desire to predict are termed outputs while the
measured signals that can be manipulated to influence the outputs are called inputs. In
industrial processes, these outputs are seldom solely influenced or completely explained by
inputs. This unexplained component is attributed to the noise signals which can manifest as
measurement noise or process noise (unmeasured disturbances or unmeasured signals that
impact the system are often the cause of process noise). In the pH neutralization process
example shown below, process noise corresponds to unmeasured fluctuations in the
wastewater acidity and/or flow which causes disturbances in the process output.

Figure 1.2: (a) Process system (and its dynamic model) with input, output, and noise signals (b) A pH
neutralization process broken down into its dynamic components

The impact of measurement and process noise can be clubbed together and explained via a
stochastic model as shown in the figure above. The resulting disturbance signal summarizes
all the uncertain characteristics of the process. The stochastic and deterministic models can
be estimated simultaneously as well as separately.

In the engineering community, the task of building mathematical models of a

dynamic system using measured data is called system identification (SysID).
Additionally, the task of modeling dynamic systems without input variables is
termed time-series analysis.

3
`

Dynamic modeling notation

In this book, we will focus on time-invariant discrete-time models wherein the signals will be
assumed to be recorded at regular sampling interval of T time units. The output y at time kT
(k is any integer) will be denoted as y(k). System identification therefore entails finding the
relationship between the three signals y(k), u(k), and v(k). For the pH example, a simple
discrete-time linear model could look like the following

𝑦̂𝑝𝐻 (𝑘) = 𝛼𝑦̂𝑝𝐻 (𝑘 − 1) + 𝛽𝑢𝑎𝑙𝑘𝑎𝑙𝑖𝑛𝑒 (𝑘 − 1)

𝑦𝑝𝐻 (𝑘) = 𝑦̂𝑝𝐻 (𝑘) + 𝑣(𝑘) eq. 1

where α and β are estimable model parameters and the disturbance variable, v(k),
summarizes all the uncertainties. Here, the value of the output at the kth time instant is
predicated upon the past values of both output and input, and the disturbance. In the later
chapters, we will study how to characterize v(k) using stochastic models.

We cannot emphasize enough the importance of proper handling of process

disturbance signals. When in hurry, you may be tempted to ignore the stochastic
component. However, doing so would only be inviting disappointment as you
may end up with biased deterministic models with unsatisfactory performance.
This interlink between stochastic and deterministic models would probably not
be obvious to beginner PDSs. By the time you finish this book, this connection
will become obvious.

Static vs dynamic model

Before we proceed further, let’s take a quick look at how a dynamic model differs from a static
model. Consider a SISO process in Figure 1.3 where a step change in input is induced and
the corresponding change in output is observed.

⋯ 𝑦𝑠𝑡𝑒𝑎𝑑𝑦 ⋯
transition
dynamics

Figure 1.3: Representative dynamic changes in a SISO process output upon a step change
in input

4
`

➢ In static model, we are only concerned with being able to predict ysteady
as the impact of the step change in u.
➢ On the other hand, in dynamic model, we care about the transition
dynamics as well.
➢ It is obvious that a dynamic model estimation is a more demanding
problem and the complexity only increases for multivariable systems
with stochastic components.

Validity of discrete-time models

It won’t be wrong to say that continuous-time description of process systems (which

are inherently continuous in nature) is more natural than discrete-time description.
However, in computer controlled industrial plants, signals from the underlying
continuous process are sampled and made available at discrete time-points.
Therefore, there is sound rationale for focusing on discrete-time models.

When using first principles approach, discrete-time model can be obtained as an

approximation as well as an exact description of the underlying process. For
example, consider the following differential equation

𝑑𝑦
= 𝑓(𝑦(𝑡), 𝑢(𝑡))
𝑑𝑡

Approximating the derivative via forward difference gives,

𝑦(𝑡 + 𝑇) − 𝑦(𝑡)
= 𝑓(𝑦(𝑡), 𝑢(𝑡)); 𝑇 𝑖𝑠 𝑠𝑎𝑚𝑝𝑙𝑖𝑛𝑔 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙
𝑇

⇒ 𝑦(𝑡 + 𝑇) = 𝑦(𝑡) + 𝑇𝑓൫𝑦(𝑡), 𝑢(𝑡)൯

𝑜𝑟
𝑡
𝑦(𝑘 + 1) = 𝑦(𝑘) + 𝑇 𝑓(𝑦(𝑘), 𝑢(𝑘)); 𝑘 =
𝑇
Discrete-time model

In an alternate scenario, if the input variables are constant between samples, then
an exact analytical discrete-time representation can be derived for linear processes.

5
`

1.2 ML-DPM Workflow

Figure 1.4 shows the typical steps involved in system identification. As you can see, SysID is
more than just curve fitting. Overall, there are five broad tasks: data collection, exploratory
data analysis, data pre-treatment, model identification, and model validation. Although we will
study each step in detail in Chapter 4, let’s take a quick overview:

• Data collection and exploratory data analysis: Availability of ‘proper’ data is

absolutely critical and the requirement of ‘richness’ of training data is indispensable. In
the previous section we saw that dynamic modeling is more demanding than static
modeling and consequently, there are more stringent requirements on training data for
dynamic modeling. Training data may be taken from historical database or fresh
experiments may be performed. The training inputs should be such that the output data
contains the dynamic variations of interest.

Exploratory data analysis (EDA) involves preliminary (and usually manual)

investigation of data to get a ‘feel’ of the system’s characteristics. The activities may
include generating some graphical plots to check the presence of trends, seasonality,
and non-stationarity in data. Inferences made during EDA help make the right choices
in the subsequent steps of SysID.

• Data pre-treatment: This step consists of several activities that are designed to
remove the portions of training data that are unimportant (or even detrimental) to model
identification. It may entail removal of outliers and noise, removal of trends, etc.
Alternatively, training data can also be massaged to manipulate the model’s accuracy
as suited for the end purpose of the model. For example, if the model is to be used for
control purpose, then the training data may be pre-filtered to bolster model’s accuracy
for high frequency signals. Overall, the generic guideline is that you have better
chances of a successful SysID with a better conditioned dataset.

• Model training: Model training is the most critical step in SysID and entails a few sub-
steps. First, a choice must be made on the type of disturbance model and deterministic
model. The end use of the model, the available a priori system knowledge, and the
desired degree of modeling complexity dictate these selections. For example, if the
model is to be used for simulation purposes, then OE1 structure may be preferred over
ARX; if process disturbance has different dynamics compared to process input, then

1
We will cover these model structures in the upcoming chapters
6
`

ARMAX will be preferable; if computational convenience is sought, then ARX is usually

the first choice and is preferred over other complex structures such as recurrent neural
networks. Model structure selection is followed by model parameter estimation. This
sub-step is mostly ‘hands-off’ due to the availability of several specialized libraries that
perform parameter estimation.

Representative illustrations

Time plots

detrending

ARX modeling

𝑦(𝑘) = 0.6𝑦(𝑘 − 1) + 0.7𝑢(𝑘 − 3) + 𝑒(𝑘)

Step response check

Figure 1.4: Steps (with sample sub-steps) involved in a typical ML-based dynamic modeling

7
`

• Model validation: Once a model has been fitted, the next step is to check if the model
truly represents the underlying process. Several techniques are at a modeler’s
disposal. For example, you could check the model’s performance on a dataset that has
not been used during parameter estimation, plot the modeling errors to look for leftover
patterns, or check if the model agrees with the a priori information about the system.
Often, your first model will fail the validation procedure. An expert modeler can use the
validation results to infer the reasons for the failure. Some examples of the cause could
be
➢ Training data was not ‘rich’ enough
➢ Training data was not pre-treated adequately
➢ Choice of model structure was wrong
➢ Estimation algorithm did not converge

Once diagnosed, appropriate corrections are made, and the iterative procedure
continues.

We hope that you get the understanding that SysID is more than just black-box application of
modeling algorithm for estimation of model parameters. There are several practical aspects
that require active participation of the modeler during the SysID process. Without
exaggeration, we can say that SysID is an art, and the rest of the book will help you obtain
the necessary skills to become the SysID artist!

1.3 Taxonomy of ML-based Dynamic Models

The field of system identification is overwhelmingly extensive owing to the decades of

research that has led to the development of several specialized techniques. Figure 1.5 gives
an overview of some of the popular methods and models that we will cover in this book. These
models represent only a subset of all the SysID models out there and this selection is based
on our experience regarding the relevance of these models for process systems modeling.
We largely focus on time-invariant2, discrete-time models and these shown models can help
you handle a large majority of DPM problems you will encounter in process industry.

2
Models where time variable does not appear as an explicit variable.
8
`

Figure 1.5: Some popular SysID methods and models covered in this book

There are several noteworthy points in Figure 1.5. First, the apparent domination of linear
models: you may have expected nonlinear models to be preferable for modeling complex
industrial process systems, however, as it turns out, linear models are often justified for DPM.
The justification stems from the fact that industrial processes often operate around an optimal
point making the linear models pretty good approximation for usage in process control and
process monitoring applications. If the end use of the model is process simulation or design
where the system response over wide range of input variables is of interest, then nonlinear
models can prove to be more suitable. Within linear category, different modeling options exist
to cater to process systems with different characteristics, viz, multivariable outputs, presence
of correlated noise, disturbances sharing dynamics with inputs, presence of measurement
noise only, presence of drifts or non-stationarities, etc.

The classification at the root of linear model sub-tree is based on the methodology used for
model fitting. While PEM methods employ minimization of prediction errors, subspace
methods are based on matrix algebra and do not involve any optimization for parameter
optimization.3 Do not worry if these terms do not make much sense right now; they will soon
become ‘obvious’ to you. If not trained about the nuanced differences between these different
models and methods, you may not have much idea at the outset about which model would be
the best one for your system. This book will help you gain adequate conceptual understanding
to become adept at making the right choice and obtaining your coveted model quickly.

3
Another distinction is that PEM is used to obtain input-output models while SIM is used to obtain state-space models.
9
`

Types of models
In Eq. 1 we saw one way of representing a dynamic process model. The models from Figure
1.5 can be used to generate other representations of your dynamic systems; the figure below
shows the different forms/types of models that we will learn to derive in this book using
machine learning. We will also understand the pros and cons of these different model forms.

Figure 1.6: Different forms of dynamic process model (that we will learn to generate in this
book), their defining characteristics, and corresponding representative examples

10
`

Why careful choice of model matters: A simple illustration

Consider the shown process: 𝑒(k) white gaussian noise

u Process model:
y
𝑥(𝑘 + 1) + 𝑎𝑥(𝑘) = 𝑏𝑢(𝑘)

Assume true a=-0.8

true b=0.5

If not careful, you may just ignore the presence of measurement noise and attempt to fit
the following input-output model to estimate the model parameters a and b.

Fitted Model (ARX form): 𝑦(𝑘 + 1) + 𝑎 𝑦(𝑘) = 𝑏𝑢(𝑘)

Data file ‘simpleProcess.csv’ contains 1000 samples of u and y obtained from the true
process. Below are the parameter estimates we get using this data,

𝑎̂ = −0.6628 (± 0.016) , 𝑏෠ = 0.7174 (±0.035)

values in bracket denote standard errors

Two things are striking here: the estimates are grossly inaccurate, and the parameter
error estimates seem to suggest high confidence in these wrong values! What went
wrong in our approach? The reason is that the SNR (signal-to-noise ratio) value is not
very high (data was generated with SNR ~ 10) and the fitted ARX model is wrong input-
output form of the true process. The correct form would be the following

Correct input-output model

𝑥(𝑘 + 1) + 𝑎𝑥(𝑘) = 𝑏𝑢(𝑘)

𝑦(𝑘) = 𝑥(𝑘) + 𝑒(𝑘) ⇒ 𝑦(𝑘 + 1) + 𝑎 𝑦(𝑘) = 𝑏𝑢(𝑘) + 𝑒(𝑘 + 1) + 𝑎𝑒(𝑘)

Equation error is not white (as assumed in

ARX model) but colored!

Ignoring the presence of correlated equation error leads to biased (inaccurate)

parameter estimates. Hopefully, this simple illustration has convinced you against blind
application of ‘convenient’ models. Part II of this book will add several modeling tools
to your ML-DPM arsenal to help avoid such modeling mistakes.

11
`

1.4 Applications of DPM in Process Industry

In Figure 1.1, we saw some of the applications of dynamic models in process industry. To
provide further perspectives into how a typical plant operator or management may use these
varied applications, Figure 1.7 juxtaposes DPM applications alongside the typical decision-
making hierarchy in a process plant. Every step of plant operation is now-a-days heavily
reliant on DPM-based tools and machine learning has proven to be a useful vehicle for quickly
building these tools. You can use the models from Figure 1.5 for building these tools or if you
use commercial vendor solutions, you can find them employing these models in their products.
For example, in the process control field, FIR models have been the bedrock of industrial MPC
controllers. In the last few years, commercial vendors have inducted CVA models in their
offering due to the advantages provided by subspace models. The latest offering by Aspen,
DMC34, incorporates neural networks for MPC and inferential modeling. ARX, BJ models are
also used for industrial MPC5.

Figure 1.7: DPM applications in different layers of operational hierarchy

4
https://www.aspentech.com/en/products/msc/aspen-dmc3
5
Qin and Badgwell, A survey of industrial model predictive control technology. Control Engineering Practice, 2003
12
`

➢ The regulatory control layer primarily comprises of (PID) control valves.

Potential DPM applications may involve usage of soft-sensor for
estimation of difficult-to-measure controlled variables.

➢ The multivariable control layer usually consists of MPC and RTO

modules. Here, dynamic models representing multivariable relationships
between plant variables are used to ensure optimal operations of the
plant. Another interesting application is development of operator training
simulators (OTSs) for training plant operators.

➢ The process diagnostics layer ensures timely fault detection and

diagnostics. Here, process measurements can be compared against
predictions from dynamic models to check for presence of process
abnormalities.

➢ The production scheduling layer has dynamic models to determine short-

term optimal production schedules using forecasts of product demand
and/or raw-material cost.

This concludes our quick attempt to establish the connection between process industry,
dynamic process modeling, and machine learning. It must now be obvious to you that modern
process industry relies heavily on dynamic process modeling to achieve its objectives of
reducing maintenance costs, and increasing productivity, reliability, safety, and product
quality. ML-based DPM helps build tools quickly to facilitate achieving these objectives.

This introductory chapter has also cautioned against blind application of ‘convenient’ ML
models for dynamic modeling. Your process data will throw several questions at you at each
stage of system identification. No straightforward answers exist to these questions and only
some time-tested guiding principles are available. While the rest of the book will familiarize
you with these principles, the onus still lies on you to use your process insights and SysID

13
`

understanding to make the right modeling choices. And yes, remember the age-old advice6,
“All models are wrong, but some are useful.”

Summary

This chapter impressed upon you the importance of DPM in process industry and the role
machine learning plays in it. We familiarized ourselves with the typical SysID workflow,
explored its different tasks, and looked at different ML models available at our disposal for
model identification. We also explored the application areas in process industry where ML has
proved useful. In the next chapter we will take the first step and learn about the environment
we will use to execute our Python scripts containing SysID code.

6
Attributed to the famous statistician George E. P. Box. It basically implies that your (SysID) model will seldom exactly
represent the real process. However, it can be close enough to be useful for practical purposes.
14
`

Chapter 2
The Scripting Environment

I
n the previous chapter we studied the various aspects of system identification and learned
about its different uses in process industry. In this chapter we will quickly familiarize
ourselves with the Python language and the scripting environment that we will use to write
ML codes, execute them, and see results. This chapter won’t make you an expert in Python
but will give you enough understanding of the language to get you started and help understand
the several in-chapter code implementations in the upcoming chapters. If you already know
the basics of Python, have a preferred code editor, and know the general structure of a typical
SysID script, then you can skip to Chapter 3.

If you skim through the system identification literature, you will find almost exclusive usage of
MATLAB software as the computing environment. This is mostly attributed to the System
Identification toolbox, a very powerful MATLAB toolbox developed by Prof. Lennart Jung (a
legend in system identification). Unfortunately, this tool not freely available, and MATLAB is
not yet as popular as Python among the ML community. Luckily, several good souls in the
Python community have developed specialized libraries for all aspects of SysID. Most of the
popular SysID models can now be generated using off-the-shelf Python libraries. Considering
the dominance of Python for deep learning, Python becomes an excellent choice for SysID
scripting.

In the above context, we will cover the following topics to familiarize you to Python
• Introduction to Python language
• Introduction to Spyder and Jupyter, two popular code editors
• Overview of Python data structures and scientific computing libraries
• Python libraries for system identification
• Overview of a typical ML-DPM/SysID script

15
`

2.1 Introduction to Python

Python is a high-level general-purpose computer programming language that can be used for
application development and scientific computing. If you have used other computer languages
like Visual Basic, C#, C++, Java, then you would understand the fact that Python is an
interpreted and dynamic language. If not, then think of Python as just another name in the list
of computer languages. What is more important is that Python offers several features that
sets it apart from the rest of the pack making it the most preferred language for machine
learning. Figure 2.1 lists some of these features. Python provides all tools to conveniently
carry out all steps of an ML-DPM project, namely, data collection, data exploration, data pre-
processing, model ID, visualization, and solution deployment to end-users. In addition, freely
available tools make writing Python code very easy7.

Figure 2.1: Features contributing to Python language’s popularity

Installing Python
One can download official and latest version of Python from the python.com website.
However, the most convenient way to install and use Python is to install Anaconda
(www.anaconda.com) which is an open-source distribution of Python. Along with the core
Python, Anaconda installs a lot of other useful packages. Anaconda comes with a GUI called
Anaconda Navigator (Figure 2.2) from where you can launch several other tools.

7
Most of the content of this chapter is similar to that in Chapter 2 of the book ‘Machine Learning in Python for Process
Systems Engineering’ and have been re-produced with appropriate changes to maintain the standalone nature of this
book.
16
`

Figure 2.2: Anaconda Navigator GUI

Running/Executing Python code

Once Anaconda is installed, you can write your code in any text editor (like Notepad), save it
in .py format, and then run via Anaconda command terminal. However, a convenient
alternative is to use IDEs (integrated development environments). IDEs not just allow creating
.py scripts, but also execute them on the fly, i.e., we can make code edits and see the results
immediately, all within an integrated environment. Among the several available Python IDEs
(PyCharm, VS Code, Spyder, etc.), we found Spyder to be the most convenient and
functionality-rich, and therefore is discussed in the next section.

Jupyter Notebooks are another very popular way of writing and executing Python code. These
notebooks allow combining code, execution results, explanatory text, and multimedia
resources in a single document. As you can imagine, this makes saving and sharing complete
data analysis very easy.

In the next section, we will provide you with enough familiarity on Spyder and Jupyter so that
you can start using them.

17
`

2.2 Introduction to Spyder and Jupyter

Figure 2.3 shows the interface8 (and its different components) that comes up when you launch
Spyder. These are the 3 main components:

• Editor: You can type and save your code here. Clicking button executes the code
in the active editor tab.

• Console: Script execution results are shown here. It can also be used for executing
Python commands and interact with variables in the workspace.

• Variable explorer: All the variables generated by running editor scripts or console are
shown here and can be interactively browsed.

Figure 2.3: Spyder interface

Like any IDE, Spyder offers several convenience features. You can divide your script into cells
and execute only selected cell if you choose to (by pressing Ctrl + Enter buttons). Intellisense
allows you to autocomplete your code by pressing Tab key. Extensive debugging
functionalities make troubleshooting easier. These are only some of the features available in
Spyder. You are encouraged to explore the different options (such as pausing and canceling
script execution, clearing out variable workspace, etc.) on the Spyder GUI.

8
If you have used MATLAB, you will find the interface very familiar
18
`

With Spyder, you have to run your script again to see execution results if you close and reopen
your script. In contrast to this, consider the Jupyter interface in Figure 2.4. Note that the
Jupyter interface opens in a browser. We can save the shown code, the execution outputs,
and explanatory text/figures as a (.ipnb) file and have them remain intact when we reopen the
file in Jupyter notebook.

Figure 2.4: Jupyter interface

You can designate any input cell as a code or markdown (formatted explanatory text). You
can press Ctrl + Enter keys to execute any active cell. All the input cells can be executed via
the Cell menu.

This completes our quick overview of Spyder and Jupyter interfaces. You can choose either
of them for working through the codes in the rest of the book.
19
`

2.3 Python Language: Basics

In the current and next sections, we will see several simple examples of manipulating data
using Python and scientific packages. While these simple operations may seem unremarkable
(and boring) in the absence of any larger context, they form the building blocks of more
complex scripts presented later in the book. Therefore, it will be worthwhile to give these
atleast a quick glance.

Note that you will find ‘#’ used a lot in these examples; these hash marks are used to insert
explanatory comments in code. Python ignores (does not execute) anything written after # on
a line.

Basic data types

In Python, you work with 4 types of data types
# 4 data types: int, float, str, bool
i = 2 # integer; type(i) = int
f = 1.2 # floating-point number; type(f) = float
s = 'two' # string; type(s) = str
b = True # boolean; type(b) = bool

With the variables defined you can perform basic operations

# print function prints/displays the specified message
print(i+2) # displays 4
print(f*2) # displays 2.4

Lists, tuples as ordered sequences

Related data (not necessarily of the same data type) can be arranged together as a sequence
in a list as shown below
# different ways of creating lists
list1 = [2,4,6]
list2 = ['air',3,1,5]
list3 = list(range(4)) # equals [0,1,2,3]; range function returns a sequence of numbers starting
from 0 (default) with increments of 1 (default)
list3.append(8) # returns [0,1,2,3,8]; append function adds new items to existing list
list4 = list1 + list2 # equals [2,4,6,'air',3,1,5]
list5 = [list2, list3] # nested list [['air', 3, 1, 5], [0, 1, 2, 3,8]]
20
`

Tuples are another sequence construct like lists, with a difference that their items and sizes
cannot be changed Since tuples are immutable/unchangeable, they are more memory
efficient.

# creating tuples
tuple1 = (0,1,'two')
tuple2 = (list1, list2) # equals ([2, 4, 6, 8], ['air', 3, 1, 5])

A couple of examples below illustrate list comprehension which is a very useful way of creating
new lists from other sequences

# generate powers of individual items in list3

newList1 = [item**2 for item in list3] # equals [0,1,4,9,64]

# nested list comprehension

newList2 = [item2**2 for item2 in [item**2 for item in list3]] # equals [0,1,16,81,4096]

Indexing and slicing sequences

Individual elements in a list can be accessed and modified as follows

# working with single item using positive or negative indexes

print(list1[0]) # displays 2, the 1st item in list1
list2[1] = 1 # list2 becomes [‘air’,1,1,5]
print(list2[-2]) # displays 1, the 2nd last element in list2

Note that Python indexing starts from zero. Very often, we need to work with multiple items of
the list. This can be accomplished easily as shown below.

# accessing multiple items through slicing

# Syntax: givenList[start,stop,step]; if unspecified, start=0, stop=list length, step=1
print(list4[0:3]) # displays [2,4,6], the 1st, 2nd, 3rd items; note that index 3 item is excluded

print(list4[:3]) # same as above

21
`

print(list4[4:len(list4)]) # displays [3,1,5]; len() function returns the number of items in list
print(list4[4:]) # same as above
print(list4[::3]) # displays [2, 'air', 5]
print(list4[::-1]) # displays list 4 backwards [5, 1, 3, 'air', 6, 4, 2]
list4[2:4] = [0,0,0] # list 4 becomes [2, 4, 0, 0, 0, 3, 1, 5]

Execution control statements

These statements allow you to control the execution sequence of code. You can choose to
execute any specific parts of the script selectively or multiple times. Let’s see how you can
accomplish these

Conditional execution Loop execution

# selectively execute code based on condition # compute sum of squares of numbers in list3
if list1[0] > 0: sum_of_squares = 0
list1[0] = 'positive' for i in range(len(list3)):
else: sum_of_squares += list3[i]**2
list1[0] = 'negative'
print(sum_of_squares) # displays 78
# list1 becomes ['positive', 4, 6]

Custom functions
Previously we used Python’s built-in functions (len(), append()) to carry out operations pre-
defined for these functions. Python allows defining our own custom functions as well. The
advantage of custom functions is that we can define a set of instructions once and then re-
use them multiple times in our script and project.

For illustration, let’s define a function to compute the sum of squares of items in a list

# define function instructions

def sumSquares(givenList):
sum_of_squares = 0
for i in range(len(givenList)):
sum_of_squares += givenList[i]**2

return sum_of_squares

# call/re-use the custom function multiple times

print(sumSquares(list3)) # displays 78

22
`

You might have noticed in our custom function code above that we used
different indentations (number of whitespaces at beginning of code lines) to
separate the ‘for loop’ code from the rest of the function code. This practice is
actually enforced by Python and will result in errors or bugs if not followed.
While other popular languages like C++, C# use braces ({}) to demarcate a
code block (body of a function, loop, if statement, etc.), Python uses
indentation. You can choose the amount of indentation but it must be consistent
within a code block.

This concludes our extremely selective coverage of Python basics. However, this should be
sufficient to enable you to understand the codes in the subsequent chapters. Let’s continue
now to learn about specialized scientific packages.

2.4 Scientific Computing Packages: Basics

While the core Python data-structures are quite handy, they are not very convenient for the
advanced data manipulations we require for machine learning tasks. Fortunately, specialized
packages like NumPy, SciPy, Pandas exist which provide convenient multidimensional
tabular data structures suited for scientific computing. Let’s quickly make ourselves familiar
with these packages.

NumPy

In NumPy, ndarrays are the basic data structures which put data in a grid of values.
Illustrations below show how 1D and 2D arrays can be created and their items accessed

# import numpy package & create a 2D array

import numpy as np
arr2D = np.array([[1,4,6],[2,5,7]])

# getting information about arr2D

print(arr2D.size) # returns 6, the no. of items
print(arr2D.ndim) # returns 2, the no. of dimensions
print(arr2D.shape) # returns tuple(2,3) corresponding
to 2 rows & 3 columns

23
`

# create a 1D array
arr1D = np.array([1,4,6])

# getting information about arr1D

print(arr1D.size) # returns 3, the no. of items
print(arr1D.ndim) # returns 1, the no. of dimensions
print(arr1D.shape) # returns tuple (3,) corresponding
to 3 items

Note that the concept of rows and columns do not apply to a 1D array. Also, you would have
noticed that we imported the NumPy package before using it in our script (‘np’ is just a short
alias). Importing a package makes available all its functions and sub-packages for use in our
script.

Creating NumPy arrays

Previously, we saw how to convert a list to a NumPy array. There are other ways to create
NumPy arrays as well. Some examples are shown below

# creating sequence of numbers

arr1 = np.arange(3, 6) # same as Python range function; results in array([3,4,5])
arr2 = np.arange(3, 9, 2) # the 3rd argument defines the step size; results in array([3,5,7])
arr3 = np.linspace(1,7,3) # creates evenly spaced 3 values from 1 to 7; results in
array([1., 4., 7.])

# creating special arrays

arr4 = np.ones((2,1)) # array of shape (2,1) with all items as 1
arr5 = np.zeros((2,2)) # all items as zero; often used as placeholder array at beginning of script
arr6 = np.eye(2) # diagonal items as 1

# adding axis to existing arrays (e.g., converting 1D array to 2D array)

print(arr1[:, np.newaxis])
>>>[[3]
[4]
[5]]

arr7 = arr1[:, None] # same as above

24
`

# combining / stacking arrays

print(np.hstack((arr1, arr2))) # horizontally stacks passed arrays
>>> [3 4 5 3 5 7]

print(np.vstack((arr1, arr2))) # vertically stacks passed arrays

>>> [[3 4 5]
[3 5 7]]

print(np.hstack((arr5,arr4))) # array 4 added as a column into arr5

>>> [[0. 0. 1.]
[0. 0. 1.]]

print(np.vstack((arr5,arr6))) # rows of array 6 added onto arr5

>>> [[0. 0.]
[0. 0.]
[1. 0.]
[0. 1.]]

Basic Numpy functions

NumPy provides several useful functions like mean, sum, sort, etc., to manipulate and analyze
NumPy arrays. You can specify the dimension (axis) along which data needs to be analyzed.
Consider the sum function for example

# along-the row sum # along the column sum

arr2D.sum(axis=0) # returns 1D array arr2D.sum(axis=1) # returns 1D array
with 3 items with 2 items

Executing arr2D.sum() returns the scalar sum over the whole array, i.e., 25.

Indexing and slicing arrays

Accessing individual items and slicing NumPy arrays work like that for Python lists

25
`

# accessing individual items

print(arr2D[1,2]) # returns 7

# slicing
arr8 = np.arange(10).reshape((2,5)) # rearrange the 1D array into shape (2,5)
print((arr8[0:1,1:3]))
>>> [[1 2]]

print((arr8[0,1:3])) # note that a 1D array is returned here instead of the 2D array above
>>> [1 2]

# accessing entire row or column

print(arr8[1]) # returns 2nd row as array([5,6,7,8,9]); same as arr8[1,:]
print(arr8[:, 4]) # returns items of 5th column as a 1D array
>>> [4 9]

An important thing to note about NumPy array slices is that any change made on sliced view
modifies the original array as well! See the following example

# extract a subarray from arr8 and modify it

arr8_sub = arr8[:, :2] # columns 0 and 1 from all rows
arr8_sub[1,1] = 1000
print(arr8) # arr8 gets modified as well!!
>>> [[ 0 1 2 3 4]
[ 5 1000 7 8 9]]

This feature becomes quite handy when we need to work on only a small part of a large
array/dataset. We can simply work on a leaner view instead of carrying around the large
dataset. However, situation may arise where we need to actually work on a separate copy of
subarray without worrying about modifying the original array. This can be accomplished via
the copy method.

26
`

# use copy method for a separate copy

arr8 = np.arange(10).reshape((2,5))
arr8_sub2 = arr8[:, :2].copy()
arr8_sub2[1, 1] = 100 # arr8 won’t be affected here

Fancy indexing is another way of obtaining a copy instead of a view of the array being indexed.
Fancy indexing simply entails using integer or boolean array/list to access array items.
Examples below clarify this concept

# combination of simple and fancy indexing

arr8_sub3 = arr8[:, [0, 1]] # note how columns are indexed via a list
arr8_sub3[1, 1] = 100 # arr8_sub3 becomes same as arr8_sub2 but arr8 is not modified here

# use boolean mask to select subarray

arr8_sub4 = arr8[arr8 > 5] # returns array([6,7,8,9]), i.e., all values > 5
arr8_sub4[0] = 0 # again, arr8 is not affected

Vectorized operations
Suppose you need to perform element-wise summation of two 1D arrays. One
approach is to access items at each index at a time in a loop and sum them.
Another approach is to sum up items at multiple indexes at once. The later
approach is called vectorized operation and can lead to significant boost in
computational time for large datasets and complex operations.
# vectorized operations
vec1 = np.array([1,2,3,4])
vec2 = np.array([5,6,7,8])
vec_sum = vec1 + vec2 # returns array([6,8,10,12]); no need to loop through index 0 to 3

# slightly more complex operation (computing distance between vectors)

vec_distance = np.sqrt(np.sum((vec1 - vec2)**2)) # vec_distance = 8.0

Broadcasting
Consider the following summation of arr2D and arr1D arrays
# item-wise addition of arr2D and arr1D
arr_sum = arr2D + arr1D

27
`

NumPy allows item-wise operation between arrays of different dimensions, thanks to

‘broadcasting’ where smaller array is implicitly extended to be compatible with the larger array,
as shown in the illustration below. As you can imagine, this is a very convenient feature and
for more details on broadcasting rules along with different example scenarios, you are
encouraged to see the official documentation9.

Pandas

Pandas is another very powerful scientific package. It is built on top of NumPy and offers
several data structures and functionalities which make (tabular) data analysis and pre-
processing very convenient. Some noteworthy features include label-based slicing/indexing,
(SQL-like) data grouping/aggregation, data merging/joining, and time-series functionalities.
Series and dataframe are the 1D and 2D array like structures, respectively, provided by
Pandas

# Series (1D structure) # Dataframe (2D structure)

import pandas as pd data = [[1,10],[1,8],[1,6]]
df = pd.DataFrame(data, columns=['id', 'value'])
data = [10,8,6] print(df)
s = pd.Series(data) >>>
print(s)
>>>

# dataframe from series

s2 = pd.Series([1,1,1])
df = pd.DataFrame({'id':s2, 'value':s}) # same as above

Note that s.values and df.values convert the series and dataframe into corresponding NumPy
arrays.

9
numpy.org/doc/stable/user/basics.broadcasting.html
28
`

Data access
Pandas allows accessing rows and columns of a dataframe using labels as well as integer
locations. You will find this feature pretty convenient.

# column(s) selection
print(df['id']) # returns column 'id' as a series
print(df.id) # same as above
print(df[['id']]) # returns specified columns in the list as a dataframe
>>> id
0 1
1 1
2 1

# row selection
df.index = [100, 101, 102] # changing row indices from [0,1,2] to [100,101,102] for illustration
print(df)
>>> id value
100 1 10
101 1 8
102 1 6

print(df.loc[101]) # returns 2nd row as a series; can provide a list for multiple rows selection
print(df.iloc[1]) # integer location-based selection; same result as above

# individual item selection

print(df.loc[101, 'value']) # returns 8
print(df.iloc[1, 1]) # same as above

Data aggregation
As alluded to earlier, Pandas facilitates quick analysis of data. Check out one quick example
below for group-based mean aggregation

# create another dataframe using df

df2 = df.copy()
df2.id = 2 # make all items in column 'id' as 2
df2.value *= 4 # multiply all items in column 'value' by 4
print(df2)
>>> id value
100 2 40

29
`

101 2 32
102 2 24

# combine df and df2

df3 = df.append(df2) # a new object is retuned unlike Python’s append function
print(df3)
>>> id value
100 1 10
101 1 8
102 1 6
100 2 40
101 2 32
102 2 24

# id-based mean values computation

print(df3.groupby('id').mean()) # returns a dataframe
>>> value
id
1 8.0
2 32.0

File I/O
Conveniently reading data from external sources and files is one of the strong forte of Pandas.
Below are a couple of illustrative examples.
# reading from excel and csv files
dataset1 = pd.read_excel('filename.xlsx') # several parameter options are available to customize
what data is read
dataset2 = pd.read_csv('filename.xlsx')

This completes our very brief look at Python, NumPy, and Pandas. If you are new to Python
(or coding), this may have been overwhelming. Don’t worry. Now that you are atleast aware
of the different data structures and ways of accessing data, you will become more and more
comfortable with Python scripting as you work through the in-chapter code examples.

30
`

2.5 SysID-relevant Python Libraries

In the previous chapter we saw several dynamic modeling options for SysID. We also saw
some of the pre-processing steps involved in data preparation and in the subsequent residual
assessment step. Technically, one can employ the previously described scientific packages
to implement the SysID algorithms, but it can be cumbersome and inconvenient. Fortunately,
the Python user-community has made available several libraries that make implementation of
the SysID algorithms very convenient. Figure 2.5 shows some of these libraries.

Figure 2.5: SysID-relevant Python packages10 and the corresponding available functionalities

You will find us using these packages heavily in the upcoming chapters. We understand that
many of the terms in the above figure may be alien to you right now, but we figured that it
would be good to give you a ‘feel’ of what Python has to offer for SysID.

Note that we did not include the celebrated Sklearn library in Figure 2.5 that offers
several generic ML-related functionalities such as dataset splitting,
standardization, scoring, etc. Our SysID scripts will make extensive use of
Sklearn library as well.

10
SIPPY: https://github.com/CPCLAB-UNIPI/SIPPY
SysIdent: https://sysidentpy.org/; https://github.com/wilsonrljr/sysidentpy
Pyflux: https://pyflux.readthedocs.io/en/latest/

31
`

The concluding message is that Python provides most of the tools for SysID that you may find
in other languages such as R and MATLAB.

2.6 Typical SysID Script

In Figure 1.4 in Chapter 1, we graphically portrayed different stages of a SysID exercise. Let’s
now see what a typical SysID script looks like. You will also understand how Python, NumPy,
and advanced packages are utilized for ML-DPM scripting. We will study several SysID
aspects in much greater detail in the next few chapters, but this simple script is a good start.
The objective of this simple script is to take data for an input and an output variable from a file
and build an ARX model between them. The first few code lines take care of importing the
libraries that the script will employ.

# import packages
from sippy import system_identification SIPPY provides ARX models
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import r2_score
from statsmodels.graphics.tsaplots import plot_acf Used for model diagnostics

We studied NumPy before. SIPPY, Sklearn, and Statsmodels were introduced in previous
section. Another library11 that we see here is matplotlib12 which is used for creating
visualization plots. The next few lines of code fetch raw data and mean-center them.

# fetch data
data = np.loadtxt('InputOutputData.txt')
u = data[:,2:]; y = data[:,1:2] # first column is timestamp

# mean-center (pre-process) model variables

u_scaler = StandardScaler(with_std=False)
u_centered = u_scaler.fit_transform(u)
y_scaler = StandardScaler(with_std=False)
y_centered = y_scaler.fit_transform(y)

11
If any package/library that you need is not installed on your machine, you can get it by running the command pip
install <package-name> on Spyder console
12
Seaborn is another popular library for creating nice-looking plots
32
`

Here NumPy’s loadtxt function is used to read space-separated data in the file
InputOutputData.txt. The data get stored in a 2D NumPy array, data, where the 2nd and 3rd
columns contain data for the output and input variables, respectively. NumPy slicing is used
to separate the u and y data. Thereafter, variables are pre-processed to mean-center them.
Next, an ARX model13 is fitted and used to make predictions.

# fit ARX model and compare training predictions vs observations

Id_ARX = system_identification(y_centered, u_centered, 'ARX', ARX_orders=[3,3,2])
y_centered_pred = np.transpose(Id_ARX.Yid) # Yid gives predictions during identification

fitted model 𝑦(𝑘) − 0.98𝑦(𝑘 − 1) + 0.02𝑦(𝑘 − 2) + 0.12𝑦(𝑘 − 3) = 0.02𝑢(𝑘 − 1) + 0.06𝑢(𝑘 − 2) +

0.07𝑢(𝑘 − 3)

plt.figure()
plt.plot(y_centered, 'o', label='raw data')
plt.plot(y_centered_pred, '--', label='ARX fit')
plt.legend(), plt.xlabel('k'), plt.ylabel('y')

In the last step, model quality assessment is done.

# Assess model accuracy

print('Fit accuracy = ', r2_score(y_centered, y_centered_pred))
>>> Fit accuracy = 0.998

# plot ACF of model residuals

res = y_centered - y_centered_pred
plot_acf(res, lags=50)
plt.xlabel('lag')

The residual plot suggests that further model refinement is warranted. In the later chapters,
we will learn how this inference has been made and what recourses are at our disposal for
model refinement.

13
SIPPY: Model coefficients can be obtained via the G (or NUMAERATOR and DENOMINATOR) attribute(s) of the model
object. See SIPPY’s manual for more details.
33
Random documents with unrelated
content Scribd suggests to you:
 Zeus

Since you obeyed my word.

Alcmena

I was no friend to such obedience

In the dead days that were my life’s design.

Zeus

You tremble. Speak your fear.

Alcmena

Heart’s utterance
Were mockery, if spoken by the tongue.

Zeus

Yet, be assured, nothing is hid from me.

Alcmena

Unmoving figure of Amphitryon

I knew and hated, when you crossed the threshold,
Hope seemed to step beside you.

Zeus

Hope is mine.

Alcmena
Then say, where have you found the keys of life,
That you unlock its portals suddenly?

Zeus

At my command all doors are set ajar.

Alcmena

The miserable forebodings of the night

Have fallen from me like the gossamer
Which spiders weave until a master-hand
Sweeps clean their tracery. Mark you a change
In me, as I in you?

Zeus

I am unchanging,
But, till this moment, me you have not known.

Alcmena

Or known myself save as a falling leaf,

The toy of winds, uncherished and unloved,
Gliding to earth and slow decay in earth
Of what was green and young.

Zeus
 When you were younger
And guarded still the pitiable illusion
That life is good and destiny exalted,
Did you not dream perhaps of sacrifice
In which yourself as immolated victim
Should satisfy delirious desire,
Wedded at last in death with strength,—which marriage
Humanly shaped has never learned to yield?

Alcmena

Your voice has in it the power of new command

To pierce my secret.

Zeus

Naught is hid from me.

Alcmena

My soul is weak with longing for your counsel.

Zeus

When Semele, with lightning-darted flame

Engirdled, woke with knowledge she must die,
Having aspired to touch the majesty
Of the omnipotent, in no wise dismayed
Was she consumed with that unquenchable fire
Which burns all veils that overspread the flesh.

Alcmena
 Whence came the thought of Semele to you?
And why this chain of words now coiled on me
As a predestined victim?

Zeus

I myself
Blaze with the fire of Semele. This hand
Shall rend the veil once more. Myself am hope,
Sole arbiter of germinating life,
The driver of the lusty winds of morning,
The cloud-compeller, dancer of the dance
Wherein the sea is festive and the hills
Nod musical assent, the charioteer
That drags the world behind his flashing wheels,
Bringer of life and change that is called death
And vibrant longing, setter of an end
To fear and doubt, a darting two-edged sword
That heals the wounds created of itself,
The crystal-veined one, in whose blood there flows
The flame of life—in such wise apprehend
Me standing here, and in such wise remark
The honour I have done you.

Alcmena

Open-eyed
At last, I see a spirit stands beside me.
For this cause I grew pale and bent my head
In sweet confusion. Bringer of release,
Even if it should be my worship falls
Before a devil from hell, behold I kneel
To kiss the fragrance of your garment’s hem.
 V. DE S. PINTO
(CHRIST CHURCH)
ART

FATE from an unimaginable throne

Scatters a million roses on the world;
They fall like shooting stars across the sky
Glittering:
Under a dark clump of trees
Man, a gaunt creature, squats upon the ground
Ape-like, and grins to see those brilliant flowers
Raining through the dark foliage:
He tries
Sometimes to clutch at them, but in his hands
They melt like snow.
Then in despair he turns
Back to his wigwam, stirs the embers, pats
His blear-eyed dog, and smokes a pipe, and soon,
Wrapped in his blankets, drowses off to sleep.

But all his dreams are full of flying flowers.

 ALAN PORTER
(QUEEN’S)
LIFE AND LUXURY
I HELD imagination’s candle high
To thread the pitchy cavern, life. A whisper
Dazed all the dark with sweetness oversweet,
A lithe body languished around my neck.
“Do out this unavailing light;” she pleaded.
“Soother is darkness. How may candle strive
With topless, bleak, obdurate blanks of space?
It can but cold the darkness else were warm.
Leave, leave to search so bitter-toilfully
Unthroughgone silence, leave and follow me;
For I will lead where many riches lie,
Where rippling silks and snow-soft cushions, rare
Cool wines, and delicates unearthly sweet,
And all the comfort flesh of man craves more.
We two shall dallying uncurl the long
And fragrant hours.” She reached a slender arm
Slowly along mine to the light. I flung her
Off, down. My candle showed her cheeks raddled,
Her bindweed pressure made me sick and mad;
I flung her back to the gloom. Her further hand
Clanked; hidden gyves fell ringing to the rock.
Peering behind her barely I could discern
Outstretching bodies clamped along the floor,
Unmoving most and silent, some uneasy,
Stirring and moaning. Smothery clutches came
Of slothful scents and fingered at my throat;
But, brushing by them, unaccompanied
I held aloft my rushlight in the cave
And searched for beauty through the cleaner air.
Thus far in parable. Laugh loud, O world,
Laugh loud and hollow. There are those would spurn
Your joys unjoyous and your acid fruits.
They would not tread the corpsy paths of commerce
Nor juggle with men’s bones; they would not chaffer
Their souls for strumpet pleasure Cast them out
Their souls for strumpet pleasure. Cast them out,
Deny what little they would ask of life,
Assail, starve, torture, murder them, and laugh.
Shall it be war between us? Better war
Than faint submission—better death. And yet
I would not, no, nor shall not die. How weaponed
Shall I go passionate against your host?
How, cautelous, elude your calm blockade?

Of older days heart-free the poet roved

Along the furrowed lanes, and watched the robin
Squat in a puddle, whir his stumpy wings,
And tweet amid the tempest he aroused;
A hare would hirple on ahead (keep back,
Let her get out of sight; quick, cross yourself),
Or taper weasel slink past over the road;
And, seeing native blossoms, breathing air
From English hills, what recked the wanderer
That barons threw no penny to his song?
Should he be hungered, he would seek some rill
And, scrambling down the hazel scarp, would walk
Wet-ankled up the stream until he found
A larger pool of cold, colourless water,
Full two-foot deep, scooped out of solid stone
By a chuckling trickle spated after rains.
There he would rest upon the bank, while slowly
His fingers crept along the crannied rock.
Poor starveling belly!—No, that lower fissure,
Straight, lipless grin like an unholy god’s,
Reach out for that. The water stings to his armpit,
He hangs above the pool from head to waist,
His legs push tautly back for body’s poise,
And careful, careful creep the sensitive fingers.

—Sudden touch of cold, wet silk.

Now flesh be one with brain! He lightly strokes
The slippery smoothness upward to the gills
 pp y p g
And throws a twiring trout upon the grass.
Or where the rattle of the water slacks
To low leaf-whisper, there he gropes beneath
Root-knots that hug black, unctuous mould from toppling
To slutch the daylit stream. His wary nerves
Tell blunt teeth biting at his thumb. Stormswift
He snatches a heavy hand over his head.
A floundering eel flops wildly to the floor,
And glides for the water. Quick the hungry poet
Spins round, whips out his knife, and shears the neck
How firm soever gripped, the limber body
Long after wriggles headless out of hand.
But if he roam across foot-tangling heath
And bracken, where no burble glads the root
Of juicy grasses? If along his way
Never a kingcup lifted bowls of light,
Nor burly watermint with bludgeon scent,
Beat down the fair, mild, slumbering meadowsweet?
If no nearby forgetmenot looks up
With frank and modest eye, no yellow flag
Plays Harold crowned and girt by fearless pikes?
No more he fails of ample fare; nor famine
Drains out his blood and piecemeal drags his flesh
From outward-leaping bones, till wrathful death,
Grudging to lose a pebble from his cairn,
Bears off the pitiful orts. For, stepping soft,
He finds a rabbit gazing at the world
With eyes in which not many moons have gleamed;
And, raising a bawl of more expended breath
Than fritter your burghers in a year of gabbling,
He runs and hurls himself headlong on to it.
Stunned at the cry, the rabbit waits and dithers;
His muscles melt beneath him; “Pluck up strength,”
He calls to his legs; “oh, stiffen, stiffen!” and still
He waits and dithers. Now the trembling scale
Of timeless pain crashes suddenly down,
 p y ,
And life’s a puffed-out flame.

Thus the poet

Of bygone England (as an alchemist
After ill magics and long labours wrought
Seals in the flask his magisterium,
Lest volatile it waste among the winds,
And all men breathe a never-ageing youth)
Found way to pend within his body life
And what of pain or interwoven joy
Life brings to poets. Friend, I do not gulp
And weep with maudlin, sentimental tears,
Lacking a late lamented golden age.
The more of life was ever misery’s,
And Socrates won hemlock. Yet before
Was man so constant enemy to man?
Did earth grow bleak at all these purposeless,
Rotting and blotting, roaking, smoking chimneys?
Look, men are dying, women dying, children dying.
They sell their souls for bread, and poison-filths
Whiten their flesh, bow their bodies. Crippled,
Consumption-spotted, feeble-minded, sullen,
They seek, bewildered, out of black despair,
The star of life; so, dying a Christian death,
Lie seven a grave unheedful. “Bad as that?
Put down five hundred on the Lord Mayor’s list.
After the cost of organizing’s paid
There’ll still be something left. Besides, it looks well,
And charity brings the firm new customers.
Not that I hold with all this nonsense really.
When I was young I’d nothing more than they,
But I climbed, and trampled other people down.
Why shouldn’t they?” O murderers, look, look, look.
No man but tramples, tramples on his neighbour,
And these the lowest wrench and writhe and kick
And crush the desperate lives of whom they can.
I will not tread the corpsy path of commerce
Nor juggle with men’s bones. The world shall wend
Those murderous ways. Not I, no, never I.
You shall not gaol me round with city walls;
I will not waste among your houses; roads
That indiscriminate feel a thousand footings
Shall not for mine augment their insolence.
But, as of old the poet, poet now
Shall hold a near communion with earth,
Free from all traffic or truck with worldlihood:
As poet one time lived of natural bounty,
So now shall I. Yet differs even this.
Me no man wronging still the world shall hound
With interdict of food. Gamekeepers, bailiffs,
And all the manlings vail and bob to lords
Shall sturdy stand on decent English Law
And threat my famine with a worser fate,
The seasonless monotonies of walls
That straitlier cabin than the closest town.
So let them threat. War stands between us. I
Take peril comrade, knowing a hazel scarp
That breaks down ragged to a scampering brook;
Knowing a hill whose deep-slit, slanting sides
Brave out the wind and shoulder the rough clouds through.

A FAR COUNTRY
THIS wood is older born than other woods:
The trees are God’s imagining of trees,
Anemones
So pale as these
Have never laughed like children in far solitudes,
Shaking and breaking worldforweary moods
To pure and childish glees.

The dripple from the mossed and plashing beck

Has carven glassy walls of pallid stone,
Where ferns have thrown
Fine silks unsewn,
Faint clouds unskied, that, one enchanted moment, check
And chalice waterdrops. They, silver grown,
With moons the darkness fleck.
 HILDA REID
(SOMERVILLE)
THE MAGNANIMITY OF BEASTS

MAN—you who think you really know

The beast you gaze on in the show,
Nor see with what consummate art
Each animal enacts its part—
How different do they all appear
The moment that you are not there!
Then, fawns with liquid eyes a-flame
Pursue the bear, their nightly game;
Wolves shiver as the rabbit roars
And stretches his terrific claws;
While trembling tigers dare not sleep
For passionate, relentless sheep,
And frantic eagles through the skies
Are chased by angry butterflies.
—But beasts would suffer all confusions
Before they shattered man’s illusions.
 EDGELL RICKWORD
(PEMBROKE)
INTIMACY

SINCE I have seen you do those intimate things

That other men but dream of; lull asleep
The sinister dark forest of your hair,
And tie the bows that stir on your calm breast
Faintly as leaves that shudder in their sleep.
Since I have seen your stocking swallow up,
A swift black wind, the pale flame of your foot,
And deemed your slender limbs so meshed in silk
Sweet mermaid sisters drowned in their dark hair;
I have not troubled overmuch with food,
And wine has seemed like water from a well;
Pavements are built of fire, grass of thin flames.
All other girls grow dull as painted flowers
Or flutter harmlessly like coloured flies
Whose wings are tangled in the net of leaves
Spread by frail trees that grow behind the eyes.

GRAVE JOYS
TO PEGGY
WHEN our sweet bodies moulder under-ground,
Shut off from these bright waters and clear skies,
When we hear nothing but the sullen sound
Of dead flesh dropping slowly from the bone
And muffled fall of tongue and ears and eyes;
Perhaps, as each disintegrates alone,
Frail broken vials once brimmed with curious sense,
Our souls will pitch old Grossness from his throne,
And on the beat of unsubstantial wings
Soar to new ecstasies still more intense.
There the thin voice of horny, black-legged things
Shall thrill me as girls’ laughter thrills me here,
And the cold drops a passing storm-cloud flings
Be my strong wine, and crawling roots and clods
My trees and hills, and slugs swift fallow deer.
There I shall dote upon a sexless flower
By dream-ghosts planted in my dripping brain,
And suck from those cold petals subtler power
Than from your colder, whiter flesh could fall,
Most vile of girls and lovelier than all.
But in your tomb the deathless She will reign
And draw new lovers out of rotting sods
That your lithe body may for ever squirm
Beneath the strange embraces of the worm.

ADVICE TO A GIRL FROM THE WARS

 WEEP for me but one day,
Dry then your eyes;
Think, is a heap of clay
Worth a maid’s sighs?

Sigh nine days if you can

For my waste blood;
Think then, you love a man
Whose face is mud;

Whose flesh and hair thrill not

At your faint touch;
Dear! limbs and brain will rot,
Dream not of such.

YEGOR
“What shall I write?” said Yegor.—Tchekov.
“WHAT shall I write?” said Yegor;
“Of the bright-plumed bird that sings
Hovering on the fringes of the forest,
Where leafy dreams are grown,
And thoughts go with silent flutterings,
Like moths by a dark wind blown?”

“Oh, write of those quiet women,

Beautiful, slim and pale,
Whose bodies glimmer under cool green waters,
Whose hands like lilies float
Tangled in the heavy purple veil
Of hair on their breast and throat.”

“Or write of swans and princes

Carved out of marble clouds,
Of the flowers that wither upon distant mountains,
Grey-pencilled in the brain;
Of fiercely hurrying night-born crowds
By the first swift sun-ray slain.”

“Nay, I will sing,” said Yegor,

“Of stranger things than these,
Of a girl I met in the fresh of morning,
A laughing, slender flame;
Of the slow stream’s song and the chant of bees,
In a land without a name.”

STRANGE ELEMENTS
WHEN my girl swims with me I think
She is a Shark with hungry teeth,
Because her throat that dazzles me
Is white as sharks are underneath.

And when she drags me down with her

Under the wave, she clings so tight,
She seems a deadly Water-snake
Who smothers me in that dim light.

Yet when we lie on the hot sand,

I find she cannot bite or hiss,
But she swears I’m a Tiger fierce
Who kills her slowly with a kiss.
 W. FORCE STEAD
(QUEEN’S)
THE BURDEN OF BABYLON[A]

“It is in the soul that things happen.”

[A] The lyrics from “The Burden of Babylon” appeared in Oxford

Poetry, 1919. The present editors have decided to reprint them with
their context.

Scene: An upper chamber in the

Palace of the King of Babylon.
Dusk on a hot summer’s evening.
The voice of one singing far off
beyond the palace-gardens is
heard vaguely from time to time.
The King is sitting by an open
window.
The King of Babylon

SINCE I am Babylon, I am the world.

The windy heavens and the rainy skies
Attend the earth in humble servitude.
And I am Babylon, I am the world:
The heavens and their powers attend on me.

The Voice of One Crying in the Night

 Babylon, the glory of the Kingdoms,
And the Chaldee’s excellency,
Is become as Sodom and Gomorrah,
Whom God overthrew by the Sea.

The King

Who is that fellow crying by the river?

I think I heard him lift his voice in praise
Of Babylon: some minstrelle seeking hire:
I need him not to tell me who I am,
For I am Baladan of Babylon.
The splendours of my sceptre, throne, and crown,
And all the awe that fills my royal halls,
The pomp that heralds me, the shout that follows,
Are flying shadows and reflections only
From the wide dazzlings of myself, the King.
This I conceive: and yet, we kings have labour
To apprehend ourselves imperially,
And see the blaze and lightnings of our person;
The thought of their own sovereignty amazes
The princelings even, and the lesser kings:
But I am Baladan of Babylon.

The Voice in the Night

Never again inhabited,
Babylon, O Babylon
Even the wandering Arabian
From thy weary waste is gone.
Neither shall the shepherd tend his fold there,
Nor any green herb be grown:
It cometh in the night-time suddenly,
And Babylon is overthrown.

The King
Pale from the east, the stars arise, and climb,
And then grow bright, beholding Babylon;
They would delay, but may not; so they pass,
And fade and fall, bereft of Babylon.
Quick from the Midian line the sun comes up,
For he expects to see my palaces;
And the moon lingers, even on the wane....
Mine ancient dynasty, as yon great river,
Euphrates, with his fountains in far hills,
Arose in the blue morning of the years;
And as yon river flows on into time,
Unalterable in majesty, my line
Survives in domination down the years.
I know, but am concerned not, that some peoples,
At the pale limits of the world, abide
As yet beyond the circle of my sway,
The miserable sons of meagre soil
That needs much tillage ere the yield be good.
I only wait until they ripen more,
And fatten toward my final harvesting:
When I am ready, I will reap them in.
For it is written in the stars, and read
Of all my wise men and astrologers,
That I, and my great line of Babylon,
Shall rule the world, and only find a bound
Where the horizon’s bounds are set, an end
When the world ends; so shall all other lands,
All languages, all peoples, and all tongues,
Become a fable told of olden times,
Deemed of our sons a thing incredulous.

The Voice in the Night

Woeful are thy desolate palaces,
Where doleful creatures lie,
And wild beasts out of the islands
In thy fallen chambers cry.
Where now are the viol and the tabret?—
But owls hoot in moonlight,
And over the ruins of Babylon
The satyrs dance by night.

The King
That voice, that seems to hum my kingdom’s glory
Fails in the vast immensity of night,
As fails all earthly praise of Him who hears
The ceaseless acclamation of the stars.
What needs there more?—the apple of the world,
Grown ripe and juicy, rolls into my lap,
And all the gods of Babylon, well pleased
With blood of bulls and fume of fragrant things,
Even while I take mine ease, attend on me:
The figs do mellow, the olive, and the vine,
And in the plains climb the big sycamores;
My camels and my laden dromedaries
Move in from eastward bearing odorous gums,
And the Zidonians hew me cedar beams,
Even tall cedars out of Lebanon;
Euphrates floats his treasured freightage down,
And all great Babylon is filled with spoil.
Wherefore, upon the summit of the world,
The utmost apex of this thronèd realm,
I stand, as stands the driving charioteer,
And steer my course right onward toward the stars.
Mean-fated men my horses trample under,
And my wine-bins have drained the blood of mothers,
And smoothly my wheels run upon the necks
Of babes and sucklings,—while I hold my way,
Serene, supreme, secure in destiny,
Because the gods perceive mine excellence,
And entertain for mine imperial Person
Peculiar favours.... I am Babylon:
Exceeding precious in the High One’s eyes.

The Voice in the Night

Babylon is fallen, fallen,
And never shall be known again!
Drunken with the blood of my belovèd,
And trampling on the sons of men.
But God is awake and aware of thee,
And sharply shines His sword,
Where over the earth spring suddenly
The hidden hosts of the Lord;
Armies of right and of righteousness,
Huge hosts, unseen, unknown:
And thy pomp, and thy revellings, and glory,
Where the wind goes, they are gone.
 L. A. G. STRONG
(WADHAM)
FROST

UNNATURAL foliage pales the trees,

Frost in compassion of their death
Has kissed them, and his icy breath
Proclaims and silvers their election.
Death, wert thou beautiful as these,
We scarce would pray for resurrection.

VERA VENVSTAS
Corporis

PROUD Eastern Queene,

Borne forth in splendour to thy buriall.
What need of gems
To deck thee? Bear the Tyrian gauds aside.
Thy own dead loveliness outshines the pride
Of diadems.

Animæ
 O splendid hearte,
Scorned and afflicted, still thou needest not
Comfort of me.
What matter though the body be uncouthe
Wherein thou art? Fear not. He seeth truth
Who gave it thee.

[To be chaunted as in a solemn Dumpe by such as fear God.]

A BABY

TWO days with puckered face of pain

The accidental baby cried,
And on the morning of the third
Unclenched her tiny hands, and died.

FROM THE GREEK

BILL Jupp lies ’ere, aged sixty year:

From Tavistock ’e came.
Single ’e bided, and ’e wished
’Is father’d done the same.

A DEVON RHYME
GNARLY and bent and deaf ’s a post
Pore ol’ Ezekiel Purvis
Goeth creepin’ slowly up the ’ill
To the Commoonion Survis.

Tap-tappy-tappy up the haisle

Goeth stick and brassy ferule;
And Parson ’ath to stoopy down
And ’olley in ees yerole.

THE BIRD MAN

TO ERIC DICKINSON
I DREAD the parrots of the summer sun,
The harsh and blazing screams of July noon,
A riot of jays and peacocks and macaws.
There is some presage of big ardours due
Even in the pale flamingoes of the dawn;
While golden pheasants and hoopoes of the West
Burn fierce and proudly still, when he has set.

Better the winter wagtails of pied skies,

Cold ospreys of the north, cormorants of squall,
Brown wrens of rain, white silent owls of snow,
And bitterns of great clouds that in October
Sweep from the west at evening. Lovelier still
The night’s black swans, the daws of starless night
(Daw-like to hide what’s shiny), plovers and gulls
Of winds that cry on autumn afternoons....

These every one I love: but above these

Rarest of all my birds, I dearly love
The blue and silver herons of the moon.

CHRISTOPHER MARLYE
 CHRISTOPHER MARLYE damned his God
In many a blasphemous mighty line,
—Being given to words and wenches and wine.

He wrote his Faustus, and laughed to see

How everyone feared his devils but he.

Christopher Marlye passed the gate,

Eager to stalk on the floor of Heaven,
Outface his God, and affront the Seven:

But Peter genially let him in,

Making no mention of all his sin.

And he got no credit for all he had done,

Though he grabbed a hold on the coat of God,
And bellowed his infamies one by one,
Blasphemy, lechery, thought, and deed ...

But nobody paid him the slightest heed.

And the devils and torments he thought to brave

He left behind, on this side of the grave.

Heigh-ho! for Christopher Marlye.

PRINTED IN GREAT BRITAIN BY

BILLING AND SONS, LTD., GUILDFORD AND
ESHER
*** END OF THE PROJECT GUTENBERG EBOOK OXFORD POETRY,
1920 ***

Updated editions will replace the previous one—the old editions

will be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States
copyright in these works, so the Foundation (and you!) can copy
and distribute it in the United States without permission and
without paying copyright royalties. Special rules, set forth in the
General Terms of Use part of this license, apply to copying and
distributing Project Gutenberg™ electronic works to protect the
PROJECT GUTENBERG™ concept and trademark. Project
Gutenberg is a registered trademark, and may not be used if
you charge for an eBook, except by following the terms of the
trademark license, including paying royalties for use of the
Project Gutenberg trademark. If you do not charge anything for
copies of this eBook, complying with the trademark license is
very easy. You may use this eBook for nearly any purpose such
as creation of derivative works, reports, performances and
research. Project Gutenberg eBooks may be modified and
printed and given away—you may do practically ANYTHING in
the United States with eBooks not protected by U.S. copyright
law. Redistribution is subject to the trademark license, especially
commercial redistribution.

START: FULL LICENSE

Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com