Energy 223 (2021) 120065

Contents lists available at ScienceDirect

Energy
journal homepage: www.elsevier.com/locate/energy

Quantitative evaluation of the building energy performance based on

short-term energy predictions
Jiangyan Liu a, b, *, Qing Zhang a, b, Zhenxiang Dong a, b, Xin Li a, b, Guannan Li c, Yi Xie d,
Kuining Li a, b
a
Key Laboratory of Low-Grade Energy Utilization Technologies and Systems, Chongqing University, Ministry of Education, Chongqing, China
b
School of Energy and Power Engineering, Chongqing University, Chongqing, China
c
School of Urban Construction, Wuhan University of Science and Technology, Wuhan, PR China
d
School of Automotive Engineering, Chongqing University, Chongqing, China

a r t i c l e i n f o a b s t r a c t

Article history: Building energy prediction is a potential tool for benchmarking the future energy uses of individual
Received 21 October 2020 buildings. However, inevitable gaps between predicted and actual energy uses and discrepancies be-
Received in revised form tween buildings make it challenging to quantify energy consumption. Therefore, this paper proposes a
22 December 2020
systematic methodology of quantitative building energy evaluation based on short-term energy pre-
Accepted 3 February 2021
diction. First, the 24-h ahead building energy prediction model is developed based on a recurrent neural
Available online 9 February 2021
network via the Multi-Input Multi-Output strategy. Second, the quantitative energy evaluation strategy is
proposed to quantify prediction gaps based on the 1-D k-means clustering. Third, case studies are
Keywords:
Quantitative energy evaluation
conducted on five real buildings to verify the reliability of the proposed methodology. Results show that
Short-term energy prediction the energy prediction models achieved outstanding accuracies. Besides, it is necessary to analyze the
Deep learning absolute percentage error (APE) variation of each time step to deeply understand the building energy
Clustering analysis performance rather than the overall prediction performance evaluation index, such as CV-RMSE and
MAPE. Further, customized energy quantification systems are established for buildings per their specific,
individual energy performance. The building energy is quantified by labeling APEs into multiple levels.
Moreover, building operation characteristics can be further understood by quantifying energy uses.
© 2021 Elsevier Ltd. All rights reserved.

1. Introduction existing buildings. On the one hand, reasonable energy evaluation

can provide a precise understanding of the current building energy
The building sector occupies the lion’s share of global energy. characteristic and energy-saving potential. It is an essential refer-
Indeed, it takes up approximately 40% of the global primary energy ence for building retrofits to avoid the blind use of the so-called
demand [1]. Understanding energy usage and improving energy “new technology.” Conversely, energy evaluation can objectively
efficiencies of existing residential and commercial buildings are the assess the building’s energy performance before and after retrofit
most direct and effective solutions to the growing global energy so that some truly energy-saving techniques can be further pro-
demand [2]. Various advanced technologies have been proposed moted. Energy benchmarking is a reliable solution in this sense.
and used to enhance the building efficiency, including photovoltaic Building energy benchmarking refers to the energy performance
generation integrations [3], online fault detection and diagnosis [4], comparison between the current building and the baseline or
automatic predictive maintenance [5], and thermal energy storage typical peer [7]. It can be conducted via numerous widespread and
[6]. It is necessary to conduct a comprehensive evaluation of well-known building rating systems; for example, Energy Star [8],
building energy performance before applying these technologies to Energy Smart Office Label [9], Display Energy Certificates (DEC) [10],
Energy and Environmental Design (LEED) [11] and Building Research
Establishment Environmental Assessment Method (BREEAM) [12].
Generally, these tools benchmark the assessed building energy
* Corresponding author. Key Laboratory of Low-Grade Energy Utilization Tech-
nologies and Systems, Chongqing University, Ministry of Education, Chongqing,
usage by comparing it with the typical or best reference peer at a
China. macroscopic level. Additionally, the monthly or annual benchmarks
E-mail address: [email protected] (J. Liu).

https://doi.org/10.1016/j.energy.2021.120065
0360-5442/© 2021 Elsevier Ltd. All rights reserved.
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

are usually used to evaluate buildings’ average energy performance models) [32]. It is challenging to judge whether the energy change
through a month [13,14] or year [15,16]. The strategy is reasonable is caused by the inherent prediction gaps or malfunction, leading to
and feasible when benchmarking buildings at a large scale (e.g., a energy waste [33]. Third, energy quantification should accommo-
climate zone). However, an individual building’s dynamic energy date the discrepancy between buildings. On the one hand, the
performance is hard to be reflected only by a monthly average or an prediction gap magnitude is dependent on occupant behavior,
annual average due to its uncertain climate, occupant behavior, time, climate, and the temporal resolution at which the prediction
plug-in equipment loads, and the like. A shorter time-span gap is investigated [32]. Conversely, the gap between the predicted
assessment (e.g., daily or hourly) should be conducted to pre- and actual energy uses should be quite different between buildings
cisely evaluate individual building performance. Conversely, the due to their various types, locations, industrial attributes, and
daily or hourly energy evaluation is closely related to various system structure. A customized energy quantification system is
building energy systems’ routine operations. It can provide useful needed for each building rather than a general quantification
references for building owners to understand the current energy system.
consumption level and energy-saving potential. Towards this point, According to the presented limitations and challenges, the main
building energy prediction could be a desirable solution. issues that the paper intends to address are summarized as follows.
The building energy prediction refers to forecasting building
energy future uses through models. It has been the fundamental C The existing building rating systems usually employ monthly
basis of many advanced building energy management techniques, or annual benchmarks to evaluate the average energy per-
such as an online energy diagnosis [17], model predictive control formance of buildings over a month or year. It is challenging
[18], energy storage operation [19], automatic predictive mainte- to reflect the dynamic energy performance of a building on a
nance [5], and smart grid power planning [20]. Besides, sensing, smaller time scale. Hence, this paper aims to propose a
automatic control, and data storage technologies have enabled quantitative energy evaluation method for assessing short-
buildings to become information intensive. Indeed, massive term (i.e., hourly) energy uses of individual buildings.
amounts of operation data are recorded by the building manage- C Previous studies of building energy prediction have dealt less
ment systems or building automation systems. It provides a solid with its energy evaluation. There are a few ways to quantify
foundation for data-driven-based energy prediction research, and a dynamic energy uses and deal with energy prediction gaps.
considerable amount of related literature has been published on it Hence, this paper aims to propose a methodology to use
[21,22]. Generally, the building energy prediction can be divided short-term energy prediction for energy evaluation. Also,
into four classes per its application scenario in various time hori- energy quantification systems are established by quantifying
zons (i.e., ultra-short-term energy prediction, short-term energy energy prediction gaps.
prediction, medium-term energy prediction, and long-term energy C The energy prediction gaps are quite different between
prediction) [23]. Ultra-short-time energy prediction is energy buildings. Therefore, the energy quantification system
prediction within 1 h. It is mainly used for safety monitoring [24] should accommodate the discrepancy between buildings. In
and predictive model control [25]. Short-term energy prediction this paper, customized energy quantification systems are
usually refers to daily or weekly energy predictions. It can be used established for each building rather than a general quantifi-
for fault detection [17], control optimization [26], thermal storage cation system.
control [19], demand response [27], and daily or weekly plan
arrangement [28]. Medium-term energy prediction concentrates This paper proposes a systematic quantitative energy evaluation
on monthly to annual energy forecasting. It is mainly employed for methodology based on short-term energy prediction for individual
predictive maintenance [5] and component operation mode buildings with insights on the research issues. The objectives of this
determination [29]. Long-term energy prediction refers to pre- research are highlights as follows.
dicting energy uses over a year or longer, a reference for power grid
planning and reconstruction [30], and distributed energy system C The multi-step ahead short-term building energy prediction
planning [31]. Previous studies have demonstrated that the data- models are developed based on a recurrent neural network
driven-based prediction models achieved outstanding accuracies (RNN) via the Multi-Input Multi-Output (MIMO) strategy.
on energy prediction [21]. It is reasonable to employ the prediction The gaps between predicted and actual energy uses are
models as benchmarks to evaluate the building energy perfor- visualized and analyzed.
mance in the future. In contrast, the short-term building energy C A quantitative energy evaluation strategy is proposed to
predictions have greater potential in individual building energy quantify the gaps based on the 1-D k-means clustering al-
evaluation, given its close relationship to various services systems’ gorithm. Multiple energy quantification systems are
daily operations in the application. customized for different buildings.
Previous studies have proposed a mass of energy prediction C Case studies are conducted on five real buildings to verify the
methods and used them in different advanced building energy proposed quantitative energy evaluation methodology’s
management techniques. Still, few studies have applied them to reliability. The five buildings’ energy uses are quantitatively
building energy evaluation. There are several reasons behind it. assessed, and their energy usage characteristics are analyzed
First, the method of quantifying energy use is lacking when using in-depth.
the predicted energy as a benchmark. The perfect situation is that
the actual energy used is completely consistent with the predicted The rest of the paper is organized as follows. Section 2 presents
ones, indicating that future energy uses align with expectations. the theoretical basis of the algorithms and strategies used in this
However, when a difference is generated between the predicted paper. A detailed description of the proposed methodology is given
and actual energy consumption, how do we quantify the energy in Section 3. Case studies and results are presented in Section 4.
uses? Second, gaps between predicted and actual energy uses are Section 5 contains a discussion, and conclusions are summarized in
inevitable due to the realities of uncertainties (inherent in pre- Section 6.
dictions) and data scattering (inherent in measurements and

2
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

2. Theoretical background

2.1. Recurrent neural networks

The recurrent neural network (RNN) is a class of artificial neural

networks specially designed to recognize patterns in data se-
quences. Compared to the traditional neural work that assumes all
inputs and outputs are independent of each other, RNNs can use
sequential information by a “memory,” with outputs depending on Fig. 2. Schematic diagram of LSTM layer unfolded along time.
previous computations. Fig. 1 shows an example of a basic RNN
layer along the time steps. In each time step, both the cell state C
and input x at the time step t are used to calculate the current step’s
output h. Then, the cell state in the next time step (i.e., tþ1) is set as ~ t ¼ tanhðW , ½h ; xt  þ b Þ
C (4)
C t1 C
the output at time step t. For the first time step, the cell state is
usually processed as an all-zero vector. Theoretically, RNNs can where WC and bC are the weight matrix and bias for the input gate,
make use of information in arbitrarily long sequences. However, its respectively.
practical capacity to capture long-term dependencies is usually The cell state at next time step t þ 1 can be then updated per the
limited due to vanishing gradient or exploding gradient effects. In following equation based on the information screening of forget
other words, the model will become untrainable with the increase gate and input gate:
of recurrent layers.
~t
Ct ¼ ft 1 Ct1 þ it 1C (5)
Researchers investigated the theoretical reasons for the above
effects in the past decades, and several novel RNNs were proposed
to tackle this problem, such as Long short term memory (LSTM) and where ☉ is pointwise multiplication.
Gate recurrent unit (GRU) [34]. LSTM is proposed by Hochreiter and Finally, the cell output at time step t can be expressed by
Schmidhuber [35]. Its core idea is to create an information-carrying
ht ¼ ot 1tanhðCt Þ (6)
orbit across multiple time steps. As shown in Fig. 2, the LSTM
separates a memory cell from the output cell, and a forget gate is
added to determine how much memory can be carried to the next
time step. The forget gate of LSTM can be expressed by 2.2. Multi-step ahead energy predictions

 Multi-step ahead prediction is the task of predicting a sequence

ft ¼ s Wf , ½ht1 ; xt  þ bf (1) of values in a time series. It has been widely used for short-term
building energy prediction, given that building energy use shows
where s is the sigmoid activation function and Wf and bf are the a strong temporal correlation between each time step [36e38].
weight matrix and bias for the forget gate, respectively. There are three major strategies for multi-step ahead prediction:
Simultaneously, two other gate units are designed for LSTM to the recursive strategy, the direct strategy, and the MIMO strategy
guide the information flow (i.e., the input gate and the output gate). [39]. The recursive strategy is the oldest and most intuitive multi-
The input gate is defined as step ahead prediction strategy. It achieves multi-step ahead pre-
diction based on a recursive utilization of a one-step ahead pre-
diction model [40]. In other words, the model predicts the first step
it ¼ sðWi , ½ht1 ; xt  þ bi Þ (2) before the predicted value can be used as part of input variables for
predicting the next step (by the same one-step ahead prediction
where Wi and bi are the weight matrix and bias for the input gate, model), as shown in Fig. 3. This manner will be performed until the
respectively. entire horizon has been predicted.
Similarly, the output gate is given by The recursive strategy is susceptible to error accumulation when
errors in the intermediate predictions propagate forward. Indeed,
the recursive predictions are utilized in the subsequent prediction.
ot ¼ sðWo , ½ht1 ; xt  þ bo Þ (3)
The direct strategy addressed this problem by developing multiple
where Wo and bo are the weight matrix and bias for the input gate, independent models for each time step prediction in the horizon, as
respectively. shown in Fig. 4. In other words, k models are established for pre-
~ of the current time step t is dictions at time t, t þ 1, …, t þ k, respectively. However, predictions
A cell state updating candidate C
defined as

Fig. 1. Schematic diagram of the basic RNN layer unfolded along time. Fig. 3. The recursive strategy for multi-step ahead prediction.

3
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
uP
2 ,
u n Pn
t y0  y
i¼1 yi
i¼1 i i
CV  RMSE ¼ (10)
n n

APE shows the prediction accuracy of each time step. Therefore,

the MAPE is used to reflect the overall accuracy by calculating the
mean APE of the whole-time horizon. The MAPE can be expressed
Fig. 4. The direct strategy for multi-step ahead prediction. by
 
Pn y0i yi 
of each time step are independent of each other. The complex i¼1  yi 
temporal correlations between predictions will be ignored so that MAPE ¼  100 (11)
n
the overall prediction accuracy will be affected. Moreover, it will
give rise to a higher computational load when multiple prediction
models are established.
The MIMO strategy has been motivated by needing to avoid the
error accumulation or stochastic dependency neglect in a single- 2.4. 1-D k-means clustering
output mapping (e.g., the recursive and direct strategies). It de-
velops a multi-output structure by returning a vector of future The k-means clustering is a popular clustering algorithm for
values in a single time step prediction, as shown in Fig. 5. For the data partitioning in data mining. It has been widely used in building
historical time series [y1, …, yN], the next H values can be predicted characteristic energy analysis [42] and energy consumption pattern
by MIMO via recognition [43]. However, the standard k-means algorithm heavily
depends on the initial cluster centers, whose result is neither
½ytþH ; …; ytþ1  ¼ Fðyt ; …; ytdþ1 Þ þ W (7) optimal nor repeatable [44]. The unstable clustering result limits its
usage in some cases, especially for one-dimensional (1-D) clus-
where t2{d, …,NeH}, d is the embedding dimension of the time tering tasks. Therefore, Wang and Song [45] developed the 1-D k-
series, F is the vector-valued function that denoted the dependency means algorithm to address the above problem and guarantee
between future observations, and W is the noise vector, including optimality in the 1-D space. It employs a comparable dynamic
modeling errors, disturbances and/or noise. programming strategy to preserve probability distributions in a 1-D
quantization problem. The detailed theoretical basis is as follows.
Given the input array, x1, x2, …, xn is sorted in a non-descending
2.3. Prediction performance evaluation indexes order. The 1-D k-means clustering groups elements of the input 1-D
array into k clusters by minimizing the within-class sum of squares
The AE is defined as the difference or residual between the in these clusters. The original objective function of the 1-D k-means
predicted value y; and the actual value y, given by clustering can be expressed by
 
AE ¼ y0i  yi  (8)
X
k X
D½n; k ¼ min kx  mi k2 (12)
The AE reflects the deviation degree of the predicted value to the
i¼1 x2Ci
actual value directly. However, it cannot compare the reliability of
different prediction results (e.g., prediction results of different where k is the number of clusters, and mi is the mean of points in
building energy uses). Hence, a non-dimensional index (i.e., APE) is cluster Ci.
employed to gauge how close the predicted value is to the actual Then, a sub-problem is defined to find the minimum within-
value. It shows the ratio of the error to the actual value and can be class sum of squares of clustering x1, x2, …, xi into m clusters
expressed using
 0  X
m X
y  yi  D½i; m ¼ min kx  ml k2 ;
APE ¼  i   100 (9)
yi  l¼1 x2Ci
(13)
1  i  n; 1  m  k
The CV-RMSE is a scale-independent index that indicates the
relative size of the error. It is a commonly used evaluation index
where d (xj, …, xi) refers to the sum of squared distances from xj, …,
and recommended by ASHRAE guideline 14 [41]. CV-RMSE is
xi to their mean mi. When m ¼ 0 or i ¼ 0, D [i, m] ¼ 0, j is defined as
defined as the root mean square error divided by the mean of the
the index of the smallest number in cluster m of the optimal so-
data
lution to D [i, m]. Hence, the D [j-1, m-1] can be the optimal within-
class sum of squares for the first j-1 points in m-1 clusters. Based on
the recurrences, an optimal substructure for dynamic programming
can be established by
  
D½i; m ¼ min D½j  1; m  1 þ d xj ; …; xi ;
mji (14)
1  i  n; 1  m  k
Further, an auxiliary n by k matrix B is designed to find a clus-
tering of data with the minimum within-class sum of squares of D
Fig. 5. The MIMO strategy for multi-step ahead prediction. [n, k]. It is used to record the index of the smallest number in cluster
m:
4
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

   because building energy consumption normally shows weekly and

B½i; m ¼ argmin D½j  1; m  1 þ d xj ; …; xi ; daily patterns [47]. Besides, the input variables of each time step
mji (15)
can be grouped into two parts. The first part is the historical
1  i  n; 1  m  k
measurements, including power consumption, outdoor dry-bulb
The starting and ending indices of all clusters can be back- temperature, and relative humidity. The second part is the time-
tracked from B [n, k]. Accordingly, an optimal solution is generated related information, such as Hour (i.e., 1 to 24), Day (i.e., Monday
for the 1-D k-means problem. More details about the 1-D k-means to Sunday), and Month (i.e., January to December). These three
clustering can be found in Ref. [46]. variables are discrete variables at levels 24, 7, and 12, respectively.
Additionally, the discrete variables are transformed into numerical
3. Research methodology formats by the one-hot encoding method. For a discrete variable
with L levels, a matrix of L columns will be derived when trans-
3.1. Research outline forming it into a numerical manner. The prediction model’s output
variable is the building power consumption, which is 24-h ahead
The structure of the quantitative building energy evaluation with 24 dimensions.
methodology is illustrated in Fig. 6. It consists of three phases:
short-term energy prediction, quantitative energy evaluation, and a 3.2.2. Short-term energy prediction model of LSTM
post-analysis of building energy performance. The first phase (i.e., In this paper, the LSTM algorithm is used to conduct short-term
short-term energy prediction) aims to develop proper multi-step energy predictions. Conventional machine-learning and statistic
ahead building energy prediction models for buildings. The algorithms are usually used for analyzing cross-sectional data, such
MIMO strategy is employed to conduct multi-step ahead pre- as Support Vector Machine (SVM), Classification and Regression
dictions. Besides, the LSTM algorithm is used to establish energy Tree (CART), Random Forest, Multiple Linear Regression (MLR) [38].
prediction models. In the second phase, the 1-D k-means clustering The output of these algorithms is a single element (single-output).
analysis is conducted to group the APEs between the predicted and It is thus suitable for predicting energy uses at the single time step t.
actual energy uses into multiple levels. Energy quantification sys- If the multi-step ahead prediction is required, it should be used in
tems are developed to quantify the energy uses based on the conjunction with recursive or direct strategies, as shown in section
clustering analysis results. The third phase is to conduct a post- 2.2. However, the model may suffer from error accumulation or
analysis on the building energy performance. Case studies are stochastic dependency neglect in the single-output mapping.
implemented to verify the reliability of the proposed quantitative Inversely, the limitation mentioned above can be naturally resolved
energy evaluation method. The energy consumption characteristics by the LSTM algorithm as the model’s output length can be flexibly
of various buildings are analyzed in-depth. changed by adjusting the output layer of the neural network.
Hence, the MIMO strategy can be conducted by the LSTM directly.
Moreover, previous researches have demonstrated that the LSTM
3.2. Short-term energy prediction can provide more accurate results than other popular machine-
learning techniques in the single time step energy prediction
3.2.1. Model inputs and outputs [48,49]. Therefore, the LSTM algorithm and MIMO strategy are used
This paper uses the MIMO strategy to achieve short-term energy for achieving short-term energy predictions in this paper.
predictions. Specifically, the input and output time steps are 168 The sequence-to-sequence (or seq2seq for short) LSTM con-
and 24, respectively. The prediction task is defined to predict 24-h struction is implemented to conduct the MIMO strategy. The
ahead energy use based on the past 168 h (one week) data. This is seq2seq LSTM consists of two LSTM networks that act as an encoder
and a decoder pair. The encoder maps a variable-length source
sequence to a fixed-length vector, while the decoder maps the
vector representation back to a variable-length target sequence
[50]. Such architecture provides flexibility for practical applications
since it enables generating output sequences with different lengths
to input sequences. Besides, the hyper-parameters of LSTM are
tuned to obtain optimal energy prediction models, including
dropout, recurrent dropout, epochs, and batch size. Finally,
different buildings’ energy prediction models are assessed using
various performance indexes (i.e., AE, APE, CV-RMSE, MAPE).

3.3. Quantitative energy evaluation strategy

The established short-term energy prediction models are

referred to as benchmarks for energy evaluation by comparing the
predicted values with the actual values. The perfect situation is that
the actual energy uses are completely consistent with the predicted
ones, indicating that the current energy consumption is reasonable
and consistent with the expectations. However, gaps between the
predicted and actual energy uses are inevitable due to the inherent
model fitting error, uncertain occupant behavior, and changeable
climate conditions in the future time steps. Normally, the gaps
between predicted and actual energy performance should remain
within a certain range. When a disturbance occurs in the building
(e.g., operation faults or errors, irregular occupant behavior,
Fig. 6. Research outline. improper control), the gaps would be beyond the current range.
5
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

each level can be obtained by considering the statistical distance of

each cluster.
The APE is used to represent the gap between prediction and
actual energy uses in this paper. This is because the APE can eval-
uate the prediction accuracy of each time step. It is also a non-
dimensional index that can be used to compare the energy pre-
diction reliabilities of different buildings. As shown in Fig. 7, the
large APE values will be grouped into high levels, indicating the
considerable gaps between predicted and actual energy uses.
Inversely, the small APE values correspond to low levels, implying
that the gaps may distribute normally. Besides, as the clustering
Fig. 7. Schematic diagram of energy quantification system.
analysis for APE is a one-dimension problem, the 1-D k-means
clustering is used to group the APE into multiple levels. The optimal
Table 1 cluster number can be determined by two evaluation indexes:
Characteristics of five buildings. Dunn and Silhouette [51]. In this step, the cluster number is deter-
No. Name Type Location Gross floor area (m2)
mined according to the intrinsic data distribution. Hence, the ob-
tained energy quantification levels would be dynamic for different
1 Cara University dormitory New York 15,728
buildings due to their various operational characteristics. In other
2 Ellie Bank office Los Angeles 46,128
3 Evie School classroom Chicago 18,327 words, customized energy quantification systems are established
4 Garrett Government office London 1059 for different buildings. Further, each cluster’s statistical distance
5 Uriah School classroom Singapore 12,000 can be used to determine the numerical boundary of each level in
the energy quantification system, as shown in Fig. 7.
When the energy quantification system is developed, it can be
The clustering analysis is employed to group the gaps between used to quantitatively evaluate the current building’s energy. On
predicted and actual energy uses into different clusters to evaluate the one hand, building energy uses can be quantified by labeling
these gaps quantitatively. The energy quantification system is then the energy uses into various levels. Conversely, the building oper-
established based on the clustering results by labeling the cluster ation characteristics and abnormal energy consumption patterns
data into various levels, as shown in Fig. 7. Accordingly, the range of can be post-analyzed based on the quantitative energy evaluation

Fig. 8. Power consumption profiles (a) Cara (b) Ellie (c) Evie (d) Garrett (e) Uriah.

6
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

results. The detailed post-analysis of the building energy perfor- threshold would be deemed as outlies. The outliers were replaced
mance is illustrated in Section 4.4. by linear interpolation to maintain the data’s integrity over a 24-h
period for the 24-h ahead energy prediction. Besides, the missing
4. Results values were also made up using the linear interpolation method.
Further, if there are more than five missing values in the data record
4.1. Data description and preparation on the same day, all data on that day would be deleted from the
database.
4.1.1. Data source Second, proper input variables were selected for the prediction
The building data used for this paper’s analysis are acquired model’s development. As described in Section 3.2, six features were
from the Building Data Genome project [52]. The project provides used in this paper: power consumption, outdoor dry-bulb tem-
an open, shareable set of building performance-related data of 507 perature, relative humidity, Hour, Day, and Month. Additionally,
real, non-residential buildings worldwide. Each building includes Hour, Day, and Month’s discrete variables were transformed into
one year of hourly, whole-building electrical meter data. Addi- numerical formats by the one-hot encoding.
tionally, it provides various building characteristic meta-data, Finally, the database was transformed into a subsequence
including the floor area, type, weather, and industry. In this pa- database, out of which 70% and 30% were used for model training
per, five representative buildings were selected from the project, as and testing, respectively.
shown in Table 1. These buildings have various primary use types,
locations, and gross floor area. Hence, the energy performance 4.2. Short-term energy prediction results
affected by the climate, occupancy, and functions would vary
among these buildings. This paper employed the LSTM algorithm to conduct energy
Fig. 8 shows the whole year’s energy uses profiles of the five prediction. The LSTM models were implemented on the Python
buildings. Building Cara is a university dormitory in New York, and platform and the Keras library (based on Tensorflow). As introduced
its energy consumption pattern is highly related to the students’ in Section 3.1, four types of hyper-parameters were optimized via a
lifestyle. For example, as shown in Fig. 8 (a), the energy con- grid search, including the dropout, recurrent dropout, epochs, and
sumption was very low during the Christmas holiday (from batch size. The grid search settings were illustrated in Table A1 of
December to mid-January). Similarly, it had relatively low energy Appendix A. Besides, the program was performed in a sliding
uses during the summer holiday (from June to August) but not as window of 24. In other words, the model would slide forward 24
low as the Christmas holiday since some students might stay at the steps to predict the next 24-h ahead energy uses after each pre-
university. Building Ellie is an office building located in Los Angeles diction. Additionally, the early-stopping training strategy was
that has a Mediterranean climate. Throughout the year, the use of employed to prevent overfitting of the LSTM model. It terminates
the HVAC system gave rise to relatively even yearly energy con- the training process when the validated data’s accuracy stops to
sumption, as shown in Fig. 8 (b). Building Evie is a primary school increase after a certain number of iterations.
classroom, and its energy consumption patterns were strongly Table 2 shows the optimal model hyperparameters and overall
correlated to the school calendar and holidays (e.g., summer holi- prediction accuracies of the optimal models of different buildings.
day from June to August), as shown in Fig. 8 (c). Building Garrett is a The accuracies illustrated were calculated per the 24-step ahead
government office building located in London. The energy uses of predictions of the test dataset. The CV-RMSE of five buildings, Cara,
Garrett was largely affected by the Temperate Marine climate. For Ellie, Evie, Garrett, and Uriah, were 0.1315, 0.2059, 0.1675, 0.1536,
example, as shown in Fig. 8 (d), the energy consumption was very 0.1652, respectively. Previous studies specified that if the resulting
low from June to September since the air-conditioning system was CV-RMSE was below 30% when using hourly data, the model would
less used in the summer. Building Uriah is a school building located be calibrated and sufficiently close to the physical reality for engi-
in the rainy tropical climate of Singapore. As shown in Fig. 8 (e), it neering purposes [53]. In this paper, the CV-RMSE of five prediction
had high energy consumption throughout the year due to air- models were well below the 30% threshold, indicating that the
conditioning systems. Besides, Uriah’s energy performance was prediction performances were reliable for a later energy evaluation.
affected by the school calendar; it had low energy uses during the
holidays and festivals.

4.1.2. Data pre-processing

The whole year’s operation data of the five buildings were
employed for this analysis. First, the outliers and missing values in
the initial databases were processed. The interquartile range rule
identified outliers. It defines an upper threshold (Q3þ1.5  (Q3-Q1))
and a lower threshold (Q1-1.5  (Q3-Q1)) based on the first quartile
(Q1) and third quartile (Q3) of the variable. The higher threshold
samples than the upper threshold or lower than the lower

Table 2
The energy prediction performance of five buildings under optimal model
parameters.

Building Dropout Recurrent dropout Epochs Batch size MAPE CV-RMSE

Cara 0.15 0 200 64 8.20 0.1315

Ellie 0.1 0.5 200 64 13.55 0.2059
Evie 0.2 0 200 64 15.27 0.1675
Garrett 0.1 0 200 64 19.94 0.1536
Uriah 0.1 0.2 200 64 8.42 0.1652
Fig. 9. APE distribution of five buildings.

7
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

4.3. Establishment of the energy quantification system quantification system. Table 4 illustrates the established energy
quantification systems of the five buildings.
4.3.1. Analysis of the prediction gaps Cara, Ellie, Evie, Garrett, and Uriah’s energy quantification
As shown in Table 2, the model performance evaluation indexes systems had 3, 2, 5, 6, and 2 levels, respectively. The large APE
(MAPE and CV-RMSE) gave the overall prediction accuracies, which values were grouped into high levels, indicating that the gaps be-
represented the average prediction performance in the whole-time tween the predicted and actual energy uses were considerable.
horizon. However, the local prediction performance at each time Inversely, small APE values were classified into relatively low levels,
step was not present. Fig. 9 shows the distributions of APE between implying that gaps may be distributed within the normal fluctua-
the predicted and actual energy uses of the five buildings. It can be tion range. On this basis, the actual energy consumption of build-
observed the APEs are distributed in a wide range of the whole-time ings could be quantitatively evaluated by labeling the APE of the
horizon for each building. The means of APE (i.e., MAPE) were 8.20%, current time steps into different levels.
13.55%, 15.27%, 19.94%, and 8.42% for Cara, Ellie, Evie, Garrett, and
Uriah, respectively. However, the highest APE was, respectively,
34.13%, 231.99%, 80.13%, 150.50%, and 95.00% for these buildings, 4.4. Post analysis of building energy performance
which were significantly higher than the MAPEs. Results show that
the overall evaluation indexes (i.e., MAPE) were inadequate to Fig. 10 shows the Cara building’s quantitative energy evaluation
reflect each time step’s detailed prediction performance. The in- results in a typical month (from Nov. 29 to Dec. 26). The APEs be-
formation provided by the overall evaluation index was limited. In tween the predicted and actual energy consumptions were
other words, more attention should be paid to the prediction per- grouped into three levels. It can be observed that the energy uses at
formance of each time step to understand the building energy the peak load time of the day are usually labeled as level 3, indi-
performance in-depth. cating that the peak load was challenging to predict due to its great
stochastic characteristics. Besides, the energy uses at base load
(unoccupied) time were also grouped into level 3 in some cases,
4.3.2. Clustering analysis for APE although the residuals (AEs) between the predicted and actual
In this step, the established short-term prediction models were values were small. This is because the energy uses at base load time
referenced as benchmarks for further building energy evaluation. (e.g., at night and on holidays) was far less than it in the normal
Gaps between the predicted and actual energy uses were inevi- time (e.g., at the working time) due to shutting down the majority
table. The actual building operations were usually uncertain due to of energy systems (e.g., HVAC systems and most lights). The little
indeterminate occupants, changeable climate conditions, com- difference between the predicted and actual energy uses can lead to
plexities of sub-systems, and control settings. Therefore, the clus- drastic changes in the APE. Indeed, it reflects the relative change
tering analysis was conducted to quantitatively evaluate these gaps, ratio between residual and actual value. Moreover, results show
according to the methodology introduced in Section 3.2. The APEs that the energy quantification system can identify irregular energy
between the predicted and actual energy uses were grouped using consumption at night. It might be caused by an unswitched lighting
1-D k-means. The energy quantification system was then estab- system, unclosed plug-in electrical equipment, or abnormal
lished based on the clustering results by labeling the cluster data personnel activities. The building owners should pay attention to it
into various levels. Additionally, the range of each level can be when the energy consumption is located at high levels.
obtained by considering the statistical distance of each cluster. Additionally, Cara is a university dormitory whose energy con-
The 1-D k-means clustering analyses were conducted on the sumption pattern is significantly affected by the campus calendar.
APEs of the five buildings, respectively. Both the Dunn and Silhouette As shown in Fig. 10, the energy use decreased gradually from
indexes were employed to evaluate the clustering results based on December 15 due to the upcoming Christmas holiday. It achieved
a search range of cluster numbers from 2 to 10. Table 3 shows the the lowest point after December 23. However, energy consumption
clustering results of the five buildings. The optimal cluster number on December 24 was labeled as level 3, although the residual be-
of Cara, Ellie, Evie, Garrett, and Uriah were 3, 2, 5, 6, and 2, respec- tween the predicted and actual energy uses seems little. Presum-
tively. These buildings’ clustering results were different, indicating ably, there was a special activity held on Christmas Eve, thus
that each building had specific, individual performance gaps and consuming more energy. Results imply that the APE between pre-
quite different operational characteristics. dicted and actual values was more sensitive to abnormal energy
Further, the energy quantification systems were established uses in base load periods.
based on the clustering results by categorizing the cluster data into Fig. 11 illustrates the quantitative energy evaluation results of
various levels. Besides, each cluster’s statistical distance was used building Ellie from March 12 to April 8. The energy uses of Ellie were
to determine the numerical boundary of each level in the energy grouped into two levels. The majority of energy uses were ranked in

Table 3
Clustering results of five buildings.

Building Evaluation index Number of clusters

2 3 4 5 6 7 8 9 10

Cara Dunn 0.000160 0.000254 0.000232 0.000158 0.000237 0.000209 0.000213 0.000164 0.000196
Silhouette 0.557301 0.601126 0.542705 0.536515 0.533759 0.526129 0.524273 0.524920 0.527768
Ellie Dunn 0.002389 0.000136 0.000126 0.000086 0.000029 0.000043 0.000042 0.000038 0.000066
Silhouette 0.654451 0.560509 0.556090 0.539429 0.535471 0.535846 0.531414 0.530734 0.548456
Evie Dunn 0.000070 0.000031 0.000045 0.000115 0.000035 0.000070 0.000097 0.000103 0.000047
Silhouette 0.518686 0.537334 0.530202 0.562739 0.512447 0.512370 0.518027 0.561761 0.553212
Garrett Dunn 0.000044 0.000036 0.000053 0.000116 0.000184 0.000153 0.000110 0.000155 0.000074
Silhouette 0.558842 0.507415 0.514504 0.512468 0.572535 0.559994 0.551611 0.544041 0.561752
Uriah Dunn 0.000210 0.000069 0.000103 0.000061 0.000137 0.000087 0.000143 0.000177 0.000146
Silhouette 0.582016 0.541975 0.537790 0.521823 0.554136 0.568422 0.562978 0.560680 0.556588

8
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

Table 4
The energy quantification system of five buildings.

Building level 1 level 2 level 3 level 4 level 5 level 6

Cara [0, 8.18) [8.18, 12.91) [12.91, 34.13)

Ellie [0, 41.42) [41.42, 231.99)
Evie [0, 5.66) [5.66, 10.99) [10.99, 16.84) [16.84, 25.74) [25.74, 80.13)
Garrett [0, 6.07) [6.07, 12.61) [12.61, 21.21) [21.21, 39.97) [39.97, 91.90) [91.90, 150.50)
Uriah [0, 13.83) [13.93, 95.00)

Fig. 10. Quantitative energy evaluation results of building Cara in a typical month.

Fig. 11. Quantitative energy evaluation results of building Ellie in a typical month.

Fig. 13. Quantitative energy evaluation results of building Garrett in a typical month.
Fig. 12. Quantitative energy evaluation results of building Evie in a typical month.

9
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

level 1. It can be observed that Ellie has energy consumption level 5, indicating that uncertainty in these times could provoke
characteristics of typical office buildings; regular and higher energy more significant prediction gaps than other periods.
use on working days and irregular and low energy use on off days. Fig. 14 illustrates the quantitative energy evaluation results of
Also, it seems that the office building had a single-day weekend per building Uriah from March 12 to April 8. Uriah is a school building
week due to its relatively high energy consumption on Saturdays. located in Singapore. It had a regular school calendar so that the
Nevertheless, the energy use patterns on Saturdays were incon- energy uses were regular. Accordingly, its energy consumption
sistent with that of weekdays, so that it was usually labeled at a pattern was relatively concise, and only two levels were divided.
high level. As shown in Fig. 11, the energy uses of off-duty hours on The majority of the energy uses were ranked in level 1. Besides,
Saturdays were always grouped into level 2, indicating its uncertain both the energy uses in the peak load and base load can cause a
energy uses that were challenging to accurately predict. Addition- significant mismatch between the predicted and actual values. It
ally, evidence showed that the energy uses with level 2 mainly would be labeled as level 2 in these periods.
occurred during the base load time. Compared to the peak load
prediction, the base load prediction tended to have higher APEs 5. Discussion
between the predicted and actual values.
Fig. 12 illustrates the quantitative energy evaluation results of This paper employed the LSTM-based method to achieve a 24-h
building Evie from January 23 to February 19. The energy quanti- ahead energy uses prediction of various non-residential buildings.
fication system set up five levels. It implies that the energy con- Results show that the prediction models obtained desirable accu-
sumption pattern of Evie is relatively complex. Evie is a primary racies based on the overall performance evaluation of CV-RMSE and
school classroom, and it has regular energy consumption patterns MAPE. However, APEs of numerous time steps were higher than the
on weekdays. It can be observed that the gaps between the pre- MAPE based on an APE distribution visualization of the whole-time
dicted and actual energy uses were relatively large at the peak load horizon. The overall prediction performance evaluation indexes
time of weekdays; they were mainly grouped into level 3 or level 4. (CV-RMSE and MAPE) can only reflect the average accuracy level on
Moreover, the energy consumption of weekends was quite uncer- the overall time horizon but presents the prediction performance in
tain due to additional course activities. Hence, it usually led to each time step. Hence, more attention should be paid to each time
dramatic gaps between predicted and actual energy uses so that a step’s prediction performance to understand the building energy
higher level would be labeled in these periods. performance in depth.
Fig. 13 shows the quantitative energy evaluation results of Generally, both AE and APE can reflect the prediction perfor-
building Garrett from August 17 to September 15. The energy uses mance of each time step. AE is a dimensional index, which directly
in this period seem irregular despite Garrett being an office build- represents the gaps between the predicted and actual values. It is
ing. The reason behind it is that Garrett is located in London, which sensitive to larger gaps, such as prediction errors in peak loads.
has a temperate marine climate. The main energy section of the Minor prediction errors in base loads would be neglected when
building (the air-conditioning system) was less used in the summer using the AE index if they were caused by an unswitched lighting
due to its mild weather. The lighting system’s energy uses, elevator system or unclosed plug-in electrical equipment. Inversely, the APE
system, and plug-in electrical component were irregular, which reflects the relative change ratio between the residual and actual
was mainly affected by the occupancy behavior. However, the en- value. If the gaps between the predicted and actual energy uses are
ergy prediction model still showed a desirable forecasting perfor- large enough relative to the actual energy, the APEs would drasti-
mance, as shown in Fig. 13. Besides, there were six energy cally change in either situation (e.g., the peak or base loads).
quantification levels established due to the complex energy con- Moreover, the APE is a non-dimensional/normalized index, which
sumption pattern of Garrett. The energy uses labeled at high levels can be used for the parallel comparison of various building energy
(levels 5 and 6) intensively occurred on August 19. More precisely, it prediction results. Nevertheless, the APE also ignored some rela-
was between 6 pm on August 19 and 6 am on August 20. Pre- tively large energy prediction gaps under the peak load since these
sumably, major maintenance was implemented on this night. On gaps might not be so large relative to the current benchmarks.
the one hand, the energy consumption from 6 pm to 6 am was Therefore, in the actual situation, which index should be selected
lower than the same period of other days, as shown in Fig. 13. This (or both) must be assessed according to the building’s operating
indicates that the majority of energy systems were completely characteristics. For instance, the university dormitory building
shutting down. Conversely, it occurred in a proper period for Cara’s peak load was challenging to predict due to its great sto-
maintenance (at night) and during the transition season when the chastic characteristics. The AE might be suitable to evaluate its
air-conditioning system was not used. Moreover, Fig. 13 shows that dynamic energy uses. Inversely, the office building Ellie has stable
the energy uses at the base load time were sometimes assessed as energy uses on weekdays. The APE might be proper to assess un-
stable energy uses in the base loads of weekends. Besides, selecting
an evaluation index relates to the energy-saving period (peak load
or base load) that the building owner is concerned about. Further, it
is also key to find a more suitable energy evaluation indicator to
balance AE and APE’s sensitivity difference.
Conversely, the gaps between the predicted and actual energy
uses are inevitable due to the inherent model fitting error, uncer-
tain occupant behavior, and changeable climate conditions in the
future time steps. It is unavoidable to deal with these gaps when
benchmarking building energy uses based on short-term energy
predictions. Normally, the gaps between the predicted and actual
energy performance should remain within a certain range, though
there are uncertain factors. When a disturbance occurs in the
building (e.g., faults or errors, irregular occupant behavior,
improper control), the gaps will go beyond the current range. In this
Fig. 14. Quantitative energy evaluation results of building Uriah in a typical month. paper, the proposed energy quantification system developed by the
10
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

1-D k-means clustering analysis can group the gaps into various Future research should find a more suitable energy evaluation
levels according to the APE variation. In this case, the predicted indicator to balance the sensitive difference between the absolute
energy consumption can be referenced as a benchmark for future error and the relative error. Besides, the usefulness of the quanti-
energy evaluation. Besides, gaps were tolerated and quantified, tative energy evaluation method in online building energy moni-
regardless of the prediction model’s inherent performance (the toring will be investigated. The actual operation will be conducted
higher the accuracy of the prediction mode, the better). Moreover, to verify the reliability of the energy evaluation results.
it is essential to realize that each building had specific, individual
performance gaps. The clustering analysis provided a customized Credit author statement
energy quantification system for each building. The more complex
the building operation characteristic, the more energy quantifica- Jiangyan Liu: Conceptualization, Methodology, Writing; Qing
tion levels would be divided. Zhang: Software, Writing e original draft; Zhenxiang Dong: Data
Finally, the five buildings’ case study results showed that large curation, Software, Resources; Xin Li: Data curation, Reviewing and
gaps between the predicted and actual energy uses usually Editing; Guannan Li: Reviewing and Editing; Yi Xie: Supervision,
occurred at both the peak and base load times. Uncertain climate Reviewing and Editing; Kuining Li: Validation, Funding acquisition
conditions and occupant behavior may cause the former. Inversely,
there could be multiple reasons for prediction gaps in the base load, Declaration of competing interest
including special activities during the holidays, unswitched energy
systems at night, and system maintenance. More attention should The authors declare that they have no known competing
be drawn to the building owners. The gaps in the peak load times financial interests or personal relationships that could have
are usually non-human changeable (e.g., climate and staff activities appeared to influence the work reported in this paper.
in the working time). However, the gaps in the base load time could
be resolved in some cases. For example, they remind the personnel
Acknowledgement
to turn off the lights and electrical equipment after work. Addi-
tionally, the abnormal energy uses caused by faults and errors
This work was supported by the Natural Science Foundation of
should remind the building owners to investigate the causes. It may
Chongqing (cstc2019jcyj-msxmX0537), National Key Research and
provoke continuously higher energy consumption levels during
Development Project (No. 2018YFB0106102 and No.
long-term operations in the future. Afterward, the inferences on the
2018YFB0106104) and the Fundamental Research Funds for the
building operation characteristic should be closely combined with
Central Universities (No. 2019CDXYDL0007).
actual investigations and verified through maintenance results.
From this perspective, the gaps were not confirmed due to absent
building maintenance records in the public datasets, which is a Appendix A. Hyper-parameter tuning for LSTM
limitation of this paper.

6. Conclusion
Table A1
To conclude, this paper presents a systematic methodology of The grid-search settings for LSTM optimization
quantitative building energy evaluation based on short-term en-
Hyper-parameter Grid-search values
ergy predictions. The method addressed the major challenge of
quantifying gaps between the predicted and actual energy uses in Dropout 0, 0.05, 0.1, 0.15, 0.2, 0.25
Recurrent dropout 0, 0.1, 0.2, 0.3, 0.4, 0.5
buildings. Case studies are conducted based on the quantitative
Number of epochs 50, 100, 150, 200
energy evaluation of five representative, non-residential buildings Batch size 32, 64, 128
worldwide (Cara, Ellie, Evie, Garrett, and Uriah). Results show that Activation function Relu
different buildings’ energy performance was well assessed based Optimization method Adam
Loss fuction Mean square error
on the established energy quantification systems.
This study’s contribution can be summarized into three aspects.
First, the gaps between the predicted and actual energy uses were
investigated in-depth. The 24-h ahead energy uses of five buildings References
are predicted by LSTM-based models using the MIMO strategy.
[1] Uerge-Vorsatz D, Cabeza LF, Serrano S, et al. Heating and cooling energy
Results show that the overall performance evaluation indexes (CV- trends and drivers in buildings. Renew Sustain Energy Rev 2015;41:85e98.
RMSE and MAPE) can only represent the average accuracy level on [2] Carpenter J, Woodbury KA, O’Neill Z. Using change-point and Gaussian pro-
the overall time horizon. It is necessary to analyze each time step’s cess models to create baseline energy models in industrial facilities: a com-
parison. Appl Energy 2018;213:415e25.
APE variation to understand the building energy performance [3] Sobri S, Koohi-Kamali S, Rahim NA. Solar photovoltaic generation forecasting
deeply. Second, the energy quantification systems were developed methods: a review. Energy Convers Manag 2018;156:459e97.
based on the 1-D k-means clustering analysis. Results show that [4] Zhao Y, Li T, Zhang X, et al. Artificial intelligence-based fault detection and
diagnosis methods for building energy systems: advantages, challenges and
building energy consumption could be quantified by labeling the
the future. Renew Sustain Energy Rev 2019;109:85e101.
APE into multiple levels. Besides, each building had specific, indi- [5] Cauchi N, Macek K, Abate A. Model-based predictive maintenance in building
vidual performance gaps. The more complex the building’s opera- automation systems with user discomfort. Energy 2017;138:306e15.
tion characteristic, the more energy quantification levels were [6] Ben Romdhane S, Amamou A, Ben Khalifa R, et al. A review on thermal energy
storage using phase change materials in passive building applications. J Build
divided by the clustering analysis. Third, the building operation Eng 2020:101563.
characteristics were understood by quantifying the energy uses. [7] Perez-Lombard L, Ortiz J, Gonza
lez R, et al. A review of benchmarking, rating
Results show that both the gaps between predicted and actual and labelling concepts within the framework of building energy certification
schemes. Energy Build 2009;41:272e8.
energy use at the peak and base load times are significant due to [8] ENERGY STAR score for offices in the United States. Washington, D.C: . Envi-
uncertain occupant behavior, climate conditions, and maintenance. ronment Protection Agency (EPA); 2014.
More attention should be paid to the base load time gaps since it [9] Eang LS, Priyadarsini R. Building energy efficiency labeling programme in
Singapore. Energy Pol 2008;36(10):3982e92.
could be easily bridged in some cases.
11
J. Liu, Q. Zhang, Z. Dong et al. Energy 223 (2021) 120065

[10] User guide to the calculation tool for Display energy Certificates (DEC) for 2018;225:273e89.
public buildings. Dublin: Sustainable Energy Authority Of SEAI; 2013. [32] de Wilde P. The gap between predicted and measured energy performance of
[11] Scofield JH, Doane J. Energy performance of LEED-certified buildings from buildings: a framework for investigation. Autom ConStruct 2014;41:40e9.
2015 Chicago benchmarking data. Energy Build 2018;174:402e13. [33] Liu J, Li K, Liu B, et al. Improvement of the energy evaluation methodology of
[12] Doan DT, Ghaffarianhoseini A, Naismith N, et al. A critical comparison of green individual office building with dynamic energy grading system. Sustain Cities
building rating systems. Build Environ 2017;123:243e60. and Soc 2020;58:102133.
[13] Yan C, Wang S, Xiao F, et al. A multi-level energy performance diagnosis [34] Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with
method for energy information poor buildings. Energy 2015;83:189e203. gradient descent is difficult. IEEE Trans Neural Network 1994;5:157e66.
[14] Yan C, Wang S, Xiao F. A simplified energy performance assessment method [35] Hochreiter S, Schmidhuber J. Long-short term memory. Neural Comput
for existing buildings based on energy bill disaggregation. Energy Build 1997;9:1735e80.
2012;55:563e74. [36] Fan C, Wang J, Gang W, et al. Assessment of deep recurrent neural network-
[15] Exploring the use of Display energy Certificates. London: Department of En- based strategies for short-term building energy predictions. Appl Energy
ergy & Climate Change (DECC); 2013. 2019;236:700e10.
[16] Park HS, Lee M, Kang H, et al. Development of a new energy benchmark for [37] Xu C, Chen H, Wang J, et al. Improving prediction performance for indoor
improving the operational rating system of office buildings using various temperature in public buildings based on a novel deep learning method. Build
data-mining techniques. Appl Energy 2016;173:225e37. Environ 2019;148:128e35.
[17] Liu J, Liu J, Chen H, et al. Energy diagnosis of variable refrigerant flow (VRF) [38] Fan C, Xiao F, Zhao Y. A short-term building cooling load prediction method
systems: data mining technique and statistical quality control approach. En- using deep learning algorithms. Appl Energy 2017;195:222e33.
ergy Build 2018;175:148e62. [39] Ben Taieb S, Bontempi G, Atiya AF, et al. A review and comparison of strategies
[18] Kneiske TM, Niedermeyer F, Boelling C. Testing a model predictive control for multi-step ahead time series forecasting based on the NN5 forecasting
algorithm for a PV-CHP hybrid system on a laboratory test-bench. Appl Energy competition. Expert Syst Appl 2012;39:7067e83.
2019;242:121e37. [40] Tiao GC, Tsay RS. Some advances in non-linear and adaptive modelling in
[19] Cox SJ, Kim D, Cho H, et al. Real time optimal control of district cooling system time-series. J Forecast 1994;13:109e31.
with thermal energy storage using neural networks. Appl Energy 2019;238: [41] Guideline 14-2014, measurement of energy and demand savings. Atlanta,
466e80. Georgia: American Society of Heating, Ventilating, and Air Conditioning En-
[20] Ahmad T, Zhang H, Yan B. A review on renewable energy and electricity gineers; 2014.
requirement forecasting models for smart grid and buildings. Sustain Cities [42] Huang P, Sun Y. A clustering based grouping method of nearly zero energy
and Soc 2020;55:102052. buildings for performance improvements. Appl Energy 2019;235:43e55.
[21] Sun Y, Haghighat F, Fung BCM. A review of the-state-of-the-art in data-driven [43] Li G, Hu Y, Chen H, et al. Data partitioning and association mining for iden-
approaches for building energy prediction. Energy Build 2020;221:110022. tifying VRF energy consumption patterns under various part loads and
[22] Amasyali K, El-Gohary NM. A review of data-driven building energy con- refrigerant charge conditions. Appl Energy 2017;185:846e61.
sumption prediction studies. Renew Sustain Energy Rev 2018;81:1192e205. [44] Grønlund A, Larsen KG, Mathiasen A, et al. Fast exact k-means, k-medians and
[23] Qian F, Gao W, Yang Y, et al. Potential analysis of the transfer learning model bregman divergence clustering in 1D. 2017.
in short and medium-term forecasting of building HVAC energy consumption. [45] Wang H, Song M. Ckmeans.1d.dp: optimal k-means clustering in one
Energy 2020;193:116724. dimension by dynamic programming. R J. 2011;3:29e33.
[24] Svintsov AP, Shchesnyak EL, Galishnikova VV, et al. Monitoring of heating [46] Wang H, Song M. Ckmeans.1d.dp: optimal k-means clustering in one
systems as a factor of energy safety of buildings. J Build Eng 2020;31:101384. dimension by dynamic programming. R J. 2011;3:29e33.
[25] Smarra F, Jain A, de Rubeis T, et al. Data-driven model predictive control using [47] Fan C, Sun Y, Xiao F, et al. Statistical investigations of transfer learning-based
random forests for building energy optimization and climate control. Appl methodology for short-term building energy predictions. Appl Energy
Energy 2018;226:1252e72. 2020;262:114499.
[26] Peng Y, Rysanek A, Nagy Z, et al. Using machine learning techniques for [48] Cai M, Pipattanasomporn M, Rahman S. Day-ahead building-level load fore-
occupancy-prediction-based cooling control in office buildings. Appl Energy casts using deep learning vs. traditional time-series techniques. Appl Energy
2018;211:1343e58. 2019;236:1078e88.
[27] Antonopoulos I, Robu V, Couraud B, et al. Artificial intelligence and machine [49] Wang Z, Hong T, Piette MA. Building thermal load prediction through shallow
learning approaches to energy demand-side response: a systematic review. machine learning and deep learning. Appl Energy 2020:263.
Renew Sustain Energy Rev 2020;130:109899. [50] Cho K, van Merrienboer B, Gulcehre C, et al. Learning phrase representations
[28] Ayo n X, Gruber JK, Hayes BP, et al. An optimal day-ahead load scheduling using RNN encoder-decoder for statistical machine translation. 2014.
approach based on the flexibility of aggregate demands. Appl Energy [51] Brock G, Pihur V, Datta S, et al. clValid: an R package for cluster validation.
2017;198:1e11. J Stat Software 2008;25.
[29] Thangavelu SR, Myat A, Khambadkone A. Energy optimization methodology of [52] Miller C, Meggers F. The Building Data Genome Project: an open, public data
multi-chiller plant in commercial buildings. Energy 2017;123:64e76. set from non-residential building electrical meters. Energy Procedia
[30] Park B, Hur J. Spatial prediction of renewable energy resources for reinforcing 2017;122:439e44.
and expanding power grids. Energy 2018;164:757e72. [53] Reddy TA, Maor I, Panjapornpon C. Calibrating detailed building energy
[31] Husein M, Chung I. Optimal design and financial feasibility of a university simulation programs with measured data-Part I: general methodology (RP-
campus microgrid considering renewable energy incentives. Appl Energy 1051). HVAC R Res 2007;13:221e41.

12