Industrial Engineering Journal

ISSN: 0970-2555
Volume : 52, No. 2, JULY - DEC 2023
Machine Learning using Smart Weather Forecasting
Dr.D.Lakshmaiah1, Dr.I.Satyanarayana2, Dr.MD.Rahman3, G.Anusha4, U.Mamatha5, V.Dheeraj Reddy6,
P.Hemanth7
1,3
Professor, Department of ECE ,Sri Indu Institute of Engineering & Technology, Hyderabad
2
Professor, Department of MECH & PRINCIPAL, Sri Indu Institute of Engineering & Technology, Hyderabad
4
Assistant Professor, Department of ECE, Sri Indu Institute of Engineering & Technology, Hyderabad
5,6,7
IVth Btech Student, Department of ECE, Sri Indu Institute of Engineering & Technology, Hyderabad

ABSTRACT

Traditionally, weather predictions are performed with the help of large complex models of physics, which utilize different
atmospheric conditions over a long period of time. These conditions are often unstable because of perturbations of the weather system,
causing the models to provide inaccurate forecasts. The models are generally run on hundreds of nodes in a large High Performance
Computing (HPC) environment which consumes a large amount of energy. In this paper, we present a weather prediction technique that
utilizes historical data from multiple weather stations to train simple machine learning models, which can provide usable forecasts about
certain weather conditions for the near future within a very short period of time. The models can be run on much less resource intensive
environments. The evaluation results show that the accuracy of the models is good enough to be used alongside the current state-of-the-art
techniques. Furthermore, we show that it is beneficial to leverage the weather station data from multiple neighboring areas over the data of
only the area for which weather forecasting is being performance

1 INTRODUCTION show that our simple model can provide reliable weather forecasts
for this city.
Weather conditions around the world change rapidly and The major contributions of this paper include:
continuously. Correct forecasts are essential in today’s daily life. (1) The utilization of machine learning in prediction of weather
From agriculture to industry, from traveling to daily commuting, conditions in short periods of time, which can run on less
we are dependent on weather forecasts heavily. As the entire world resource-intensive machines.
is suffering from the continuous climate change and its side effects, (2) Implementation of automated systems to collect historical
it is very important to predict the weather without any error to data from a dedicated weather service.
ensure easy and seamless mobility, as well as safe day to day (3) Thorough evaluation of the proposed technique and
operations. comparison of several machine learning models in the
The current weather prediction models heavily depend on prediction of future weather conditions.
complex physical models and need to be run on large computer
The rest of this paper is organized as follows: Section 2 gives an
systems involving hundreds of HPC nodes. The computational
overview of machine learning in weather forecasting, as well as the
power of these large systems is required to solve the models that
related works. In Section 3, we present the methodology of the
describe the atmosphere. Despite using these costly and complex
proposed idea, which includes the methods to pull data from a
devices, there are often inaccurate forecasts because of incorrect
weather station. We illustrate training and test data collection and
initial measurements of the conditions or an incomplete
their preprocessing in Section 4. Section 5 shows the evaluation
understanding of atmospheric processes. Moreover, it generally
results of several machine learning techniques. We conclude the
takes a long time to solve complex models like these.
paper in Section 6.
As weather systems can travel a long way over time in all
directions, the weather of one place depends on that of others
considerably [10]. In this work, we propose a method to utilize 2 BACKGROUND
surrounding city’s historical weather data along with a particular This section briefly presents how machine learning can be
city’s data to predict its weather condition. We combine these data used inweather forecasting and the related works in the literature
and use it to train simple machine learning models, which in turn, on this fast growing research topic.
can predict correct weather conditions for the next few days.
2.1 Machine Learning for Weather Forecasting
These simple models can be run on low cost and less resource-
Machine learning is a data science technique which creates a model
intensive computing systems, yet can provide quick and accurate
from a training dataset. A model is basically a formula which outputs
enough forecasts to be used in our day-to-day life. In this work,
we present a case study on the city of Nashville in Tennessee,
USA, which is known for its fluctuating weather patterns, and

UGC CARE Group-1 64

 Industrial Engineering Journal
ISSN: 0970-2555
Volume : 52, No. 2, JULY - DEC 2023
First, we combine the weather observations at a particular record. That is, each record in the data will contain the temperature,
timestamp from all the cities that we consider to create a single humidity, wind direction, atmospheric pressure, condition, etc. data
Mid-Southeast’18, November 2018, Gatlinburg, Tennessee, USA A. Jakaria et al.
a target value based on individual weights and values for each
training variable. In each record, corresponding weights (sometimes
between 0 and 1) to each variable tells the model how that variable
is related to the target value. There must be sufficient amount of
training data to determine the best possible weights of all the
variables. When the weights are learned as accurately as possible, a
model can predict the correct output or the target value given a test
data record.
Utilizing simple machine learning techniques allow us be relieved
from the complex and resource-hungry weather models of
traditional weather stations. It has immense possibilities in the realm
of weather forecasting [4]. Such a forecasting model can be offered
to the public as web services very easily [2].
surrounding cities. The circled cities can have impact on the weather
2.2 Related Works systems of Nashville.
Machine learning in weather forecasting is a recent trend in
of all the cities. The target variable for this record is assigned as the
the literature. There are several works which discuss this topic. temperature at the same timestamp of the next day. Thus, we
Holmstrom et al. proposed a technique to forecast the maximum predict the temperature of the next day given the weather
and minimum temperature of the next seven days, given the data of observation of the current day.
past two days [6]. They utilized a linear regression model, as well as Fortunately, by the same way, we can also predict any other
a variation of a functional linear regression model. They showed that weather variable such as humidity, rainfall, wind speed and
both the models were outperformed by professional weather direction, visibility, etc. of the next day, as well as the next few days.
forecasting services for the prediction of up to seven days. However, However, we restrict our study for predicting the temperature only
their model performs better in forecasting later days or longer time in this research.
scales. A hybrid model that used neural networks to model the
physics behind weather forecasting was proposed by Krasnopolsky
and Rabinivitz [7]. Support vector machines was utilized for weather 3.2 Machine Learning Techniques
prediction as a classification problem by Radhika et al. [9]. A data In this research, as the predicted outcomes are continuous
mining based predictive model to identify the fluctuating patterns of numeric values, temperature in our case, we use regression
weather conditions was proposed in [11]. The patterns from technique. We find that Random Forest Regression (RFR) is the
historical data is used to approximate the upcoming weather superior regressor, as it ensembles multiple decision trees while
conditions. The proposed data model uses Hidden Markov Model making decision. In addition, we show comparison of several other
for prediction and k-means clustering for extracting weather state-of-the-art ML techniques with the RFR technique. The
condition observations. Grover et al. studied weather prediction via incorporated regression techniques are Ridge Regression (Ridge),
a hybrid approach, which combines discriminatively trained Support Vector (SVR), Multi-layer Perceptron (MLPR), and Extra-
predictive models with deep neural networks that models the joint Tree Regression (ETR).
statistics of a set of weather-related variables [5].
Montori et al. used the concept of crowdsensing, where 4 DATASET
participating users share their smart phone data to environmental Once we collect the data, we split the raw data in training and
phenomenons [8]. They introduced an architecture named test set. However, the target variable is always the next day hourly
SenSquare, which handles data from IoT sources and crowdsensing temperature for Nashville. The Training set contains two months of
platforms, and display the data unifiedly to subscribers. This data is weather data starting from the 1st day of July, 2018. In contrast, the
used in smart city environment monitoring. However, none of these test set contains 7 days of data starting from September 1, 2018 and
works use the idea of combining data from neighboring places. ending on September 7, 2018. Essentially, the trained model predicts
hourly temperature of the 2nd September while inputting 1st
September as test data. Similarly, temperature of September 3rd will
3 TECHNIQUE OVERVIEW
be predicted based on data from September 2nd, and so on.
In this section, we present an overview of our proposed technique.

3.1 Methodology
In this case study, we aim to use ML techniques to predict the
temperature of the next day at any particular hour of the city of
Nashville, Tennessee, based on the weather data of the current day
of this city and a couple of its surrounding cities.
Figure 1: Google map showing the city of Nashville and its

UGC CARE Group-1 65

 Industrial Engineering Journal
ISSN: 0970-2555
Volume : 52, No. 2, JULY - DEC 2023
atasets.
4.1 Dataset from Weather Station
We collected real weather data for the city of Nashville from ← −µ
Furthermore, we perform mean scaling x xσ to all the
wunderground.com [1], as well as nine more cities around Nashville: continuous variables so that the variables possess approximately
Knoxville, Chattanooga, Jackson, Bowling Green, Paducah, zero mean, which in practice, reduces computational cost while
Birmingham, Atlanta, Florence, and Tupelo (See Fig. 1). For a given training the models.
place and date, the wunderground API returns a list of weather
observations data [1]. A historical JSON query result consists of
RESULTS
In this section, we present a thorough evaluation of our
models trained with the weather station data. First set of results
show prediction accuracy while increasing training data by adding
more neighboring cities, and by adding more weeks. Second set of
results mostly emphasize the noticeable performance
improvement of our models when neighboring cities are included
in the training data.
Smart Weather Forecasting Using Machine Learning:
A Case Study in Tennessee Mid-Southeast’18, November 2018, Gatlinburg, Tennessee, USA

Number of Cities Number of Weeks ML Techniques

(a) (b) (c)

Figure 2: (a) RMSE on test set while considering neighboring cities,

(b) RMSE on test set with increasing training size, (c)
RMSEdifferent weather parameters such as temperature, humidity,
dew ontest set for different ML models
point, atmospheric pressure, wind direction, etc. at each hour of a
day for a particular city. We collect this for the required number of
days for different experiments.
We developed a program to convert the JSON data to a text file,
where each record (row) corresponds to a particular timestamp. We
skip any data from the JSON results that do not have data for all the
ten cities for a particular timestamp.

4.2 Data Preprocessing

After having the raw data from ‘wunderground’, we make sure
that each row (record) in the dataset has records for all ten cities for
a particular timestamp. We eliminate any feature with empty or
invalid data while creating the dataset. Also, we convert the
categorical features in the dataset, such as wind direction and
condition, into dummy/indicator variables using a technique called
‘One Hot Encoding’ [3]. We perform this conversion prior to the
separation of training and test data. This is because, in both training
and test data, we need the same number of feature variables. If we
do this conversion after the separation, then there remains no
guarantee that both of them will have all the categorical values for
the corresponding features. If the number of categorical values for
training and test sets is not the same, then the conversion yields to
different number of features for these sets. That is why we need to
perform this conversion before separation of training and test

UGC CARE Group-1 66

 Industrial Engineering Journal
ISSN: 0970-2555
Volume : 52, No. 2, JULY - DEC 2023
5.1 Performance Measure bucket for multiple cities, while the distribution of one city is more
In all our experiments, we use the root mean squared error likely to end up in residual buckets far from central.
(RMSE) to evaluate our models. The calculation of RMSE is Fig. 3(c) presents the RMSE for test data of increasing size in
pretty straight forward shown in Equation 1. each run. We start with 20 test examples, where ten-city case shows
50% less RMSE than the single-city case. Once the number of test
sÍn
t
(yˆt −yt)2 (1) examples reaches 60, the ten-city case shows surprisingly less error.
=
With bigger test sets beyond 60, performance of both cases degrade,
1 because weather data in some test data might be very dissimilar to
RMSE = that of training set. However, for bigger test size, It is apparent that
n the model performs better when it is trained with data from multiple
surrounding cities compared to training with one city.
Where n is the number of test examples. yˆt and yt are
predictedtemperature and actual temperature, respectively.

5.2 Performance Comparison of Models

At the very beginning of our experiment, we incrementally
add more cities to observe the RMSE trend on the test data. Fig.
2(a) exposes our expected trend, where we add nine other cities to
train the RFR model. Once the model is trained with data of
Nashville only, the observed RMSE is noticeably high, which is
approximately 4.5. As we add one more neighboring city, the
model starts to perform well. However, when 3 neighboring cities
are considered, model accuracy degrades slightly. The potential
reason is that Nashville weather might not depend on all the
neighboring cities essentially all the time. Some seasonal change
or bad weather situation of a neighboring region might not affect a
particular region throughout the whole year. However, once we
add more cities, RMSE apparently continues to decrease. At the
ten-citycase, significantly low RMSE is observed, which is nearly
35% less as compared to only one city data.
In Fig. 2(b), we show RMSE on test data for growing number
of training data of all the ten cities. At first, only one week data is
used as training set, that week is the immediate previous week of
the test week. As we said earlier, the test data comprises of seven
days of data starting from September 1, 2018. Thus, first week of
training data represents the weather data starting from the 25th
August and ending on the 31st August, 2018. Two weeks of
training refers to the data of the previous two weeks of the test
week, and so on. Using only one week, RMSE is noticeably high,
nearly 3.3. As we increase the number of weeks, RMSE drops
considerably. At 5-week situation, RMSE is nearly 3.03. After
that, we find a quick increase of RMSE as we add some more
weeks. The possible reason is that in some weeks, weather
condition might change abruptly, which influences the training
model when we incorporate those data. However, when training
set comprises of 8 weeks of data, we find lowest RMSE on the test
set.
Later on, we show comparison of different ML models on the test
data in Fig. 2(c). Each model shows RMSE when training with ten

Mid-Southeast’18, November 2018, Gatlinburg, Tennessee, USA

A. Jakaria et al.

UGC CARE Group-1 67

 Industrial Engineering Journal
ISSN: 0970-2555
Volume : 52, No. 2, JULY - DEC 2023

(a) (b) (c)

Figure 3: (a) Error in prediction of the test set, (b) distribution of errors in prediction of the test set, (c) RMSE for different

UGC CARE Group-1

68
 Industrial Engineering Journal
ISSN: 0970-2555
Volume : 52, No. 2, JULY - DEC 2023
cities, as well as when training with only one city (Nashville). 5 CONCLUSION AND FUTURE DIRECTIONS
number of test data. In this paper, we presented a technology to utilize machine
The first ML technique we use is Ridge regression. In this learning techniques to provide weather forecasts. Machine
particular regression model, both ten and single city situation show learning technology can provide intelligent models, which are
relatively high RMSE above 4.0. After that, we use Support much simpler than traditional physical models. They are less
Vector Regressor (SVR). This technique shows pretty good resource-hungry and can easily be run on almost any computer
accuracy for ten-city case compared to one city. The RMSE including mobile devices. Our evaluation results show that these
difference is near to 1 for the two cases. Next, we use Multi-Layer machine learning models can predict weather features accurately
Perceptron Regressor (MLPR), which is a two layer Neural enough to compete with traditional models. We also utilize the
Network, comprising with 100 units in first hidden layer and 50 historical data from surrounding areas to predict weather of a
units in second hidden layer. Unfortunately, the MLPR technique particular area. We show that it is more effective than
shows high RMSE in both ten and one city cases. Final two considering only the area for which weather forecasting is done.
models are Random Forest Regressor (RFR) and Extra-Tree In future, we have plans to utilize low-cost Internet of Things
Regressor (ETR). Fortunately, in these two models, the ten-city (IoT) devices, such as temperature and humidity sensors, in
case presents nearly similar RMSE close to 3.0, which is the lowest collecting weather data from different parts of a city. The use of
among all the previous three models. On the contrary, in one city different sensors could increase the number of local features in
situation, both of these models show quite high RMSE. This thetraining dataset. This data, along with the weather station
explains the necessity of considering weather data of neighboring data, willfurther improve the performance of our prediction
cities when predicting temperature of a particular city or region. models.
In Fig. 3(a), we show the prediction errors or residuals, i.e., the
difference between the predicted target value and the actual target REFERENCES
value, in the test set. As before, We plot two sets of data points for [1][n. d.]. Weather Underground.
https://www.wunderground.com/weather/api.
the same test set - one with training set from only the city of
[2]2018. Weather forecast using the sensor data from your IoT hub in
Nashville and the other one is including nine more surrounding Azure Machine Learning. https://docs.microsoft.com/en-us/azure/iot-
cities. It can be observed that the data points for only one city are hub/ iot-hub- weather-forecast-machine-learning.
more dispersed than the data points for ten cities. This apparently [3]Dan Becker. 2018. Using Categorical Data with One Hot Encoding.
shows that the possibility of error is much lower for every test https://www. kaggle.com/dansbecker/using-categorical-data-with-one-
pointwhen we consider data from multiple cities. hot-encoding/.
The distribution of the residual is presented in Fig. 3(b), where [4]Jeffrey Burt. 2017. Machine Learning Storms into Climate Research.
we present 12 buckets of residual. The central bucket yields count https://www.
of the test points where their residuals lie in the (-1, 0] range. The nextplatform.com/2017/04/18/machine-learning-storms-climate-
result reveals that it is better to use training data from multiple research/.
cities for training. The distribution of the residuals is clustered in [5]Aditya Grover, Ashish Kapoor, and Eric Horvitz. 2015. A deep
central hybrid model for weather forecasting. In Proceedings of the 21th ACM
SIGKDD International Conference on Knowledge Discovery and Data
Mining. ACM, 379–386.
[6]Mark Holmstrom, Dylan Liu, and Christopher Vo. 2016. Machine
Learning Applied to Weather Forecasting.
[7]Vladimir M Krasnopolsky and Michael S Fox-Rabinovitz. 2006.
Complex hybrid models combining deterministic and machine learning
components for numerical
climate modeling and weather prediction. Neural Networks 19, 2 (2006),
122–
134.
[8]Federico Montori, Luca Bedogni, and Luciano Bononi. 2017. A
Collaborative Internet of Things Architecture for Smart Cities and
Environmental Monitoring. IEEE Internet of Things Journal (2017).
[9]Y Radhika and M Shashi. 2009. Atmospheric temperature prediction
using support vector machines. International journal of computer theory
and engineering 1, 1 (2009), 55.
[10]Arne Sund. 2015. Using Amazon Machine Learning to Predict the
Weather. https://arnesund.com/2015/05/31/ using-amazon-machine-
learning-to-predict- the-weather/.
[11]Rohit Kumar Yadav and Ravi Khatri. 2016. A Weather Forecasting
Model using the Data Mining Technique. International Journal of
Computer Applications 139, 14 (2016).

UGC CARE Group-1 69