Proceedings of the 7th International
Scientific and Practical Conference
«Current Issues and Prospects for The No
Development of Scientific Research»
151
(April 19-20, 2023).
Orléans, France
INFORMATION AND WEB TECHNOLOGIES
DOI 10.51582/interconf.19-20.04.2023.067
Weather forecasting using big data
Gardashova Latafat1,
Jabrayilova Shahnaz Mirhuseyn2
1
PhD, assoc.prof.;
Azerbaijan State Oil and Industry University; Republic of Azerbaijan
2
2nd year student;
Azerbaijan State Oil and Industry University, Information Technologies and Management
Faculty, Computer Engineering Department; Republic of Azerbaijan
Abstract.
The use of big data in weather forecasting is examined in this article. Large and complicated
data collections that can be examined to find patterns, trends, and relationships are
referred to as "big data." Big data has become more crucial to weather forecasting in recent
years since it can offer insightful information about weather trends and boost forecasting
precision. The difficulties of using large data for weather forecasting are discussed in
the article along with the methods for processing and analyzing it. It also offers case
studies that show how big data has increased the precision of weather forecasts. The paper
concludes by highlighting the potential advantages of big data in weather forecasting and
by discussing its future applications. Big data can be used to improve weather predictions.
Keywords:
Big data
Weather forecasting
MapReduce
This work is distributed under the terms of the Creative
Commons Attribution-ShareAlike 4.0 International License
(https://creativecommons.org/licenses/by-sa/4.0/).
629
Proceedings of the 7th International
Scientific and Practical Conference
«Current Issues and Prospects for The No
Development of Scientific Research»
151
(April 19-20, 2023).
Orléans, France
INFORMATION AND WEB TECHNOLOGIES
Foreseeing the behavior of atmospheric conditions and
guiding decisions in a variety of industries, from agriculture
to transportation, weather forecasting is a crucial
instrument. Although conventional weather models have been in
use for many years, current technological developments and
the growth of big data have opened up new possibilities for
enhancing the precision and timeliness of weather forecasts.
We will examine how big data is transforming weather
forecasting in this post, along with any potential positive
effects it may have on society. We will analyze the main
technology and data sources that are bringing about this
transition and look into the potential and problems that come
with using big data to forecast the weather. Ultimately, we
will discuss how the use of big data in weather forecasting
has the potential to transform the way we prepare for and
respond to weather events, making us better equipped to deal
with the challenges of a changing climate.
In recent years, the fields of big data and weather
forecasting have become more and more intertwined. Big data
analytics has developed into a useful tool for enhancing the
precision and timeliness of weather forecasts as a result of
the development of technology and the proliferation of data
sources. Meteorologists may gather and analyze enormous
volumes of data from a number of sources, including
satellites, ground-based sensors, and social media feeds, by
using big data in weather forecasting. This information can
be used to create more precise weather models, which can then
be used to forecast severe weather events and lessen their
effects. Big data in weather forecasting, however, also brings
with it new difficulties, such as managing huge amounts of
data and creating the appropriate analytics tools to make
sense of it.
Processing and analyzing big data is essential for
improving weather forecasting accuracy and timeliness. There
are several techniques used in processing and analyzing big
data in weather forecasting, including:
1. Machine learning: Machine learning algorithms can
analyze large amounts of data and identify patterns that are
not easily detected using traditional forecasting methods.
For example, machine learning can help identify correlations
This work is distributed under the terms of the Creative
630 Commons Attribution-ShareAlike 4.0 International License
(https://creativecommons.org/licenses/by-sa/4.0/).
Proceedings of the 7th International
Scientific and Practical Conference
«Current Issues and Prospects for The No
Development of Scientific Research»
151
(April 19-20, 2023).
Orléans, France
INFORMATION AND WEB TECHNOLOGIES
between different weather variables and how they impact each
other.
2. Data fusion: Data fusion involves combining data from
multiple sources, such as satellite imagery and ground-based
sensors, to create a more complete picture of atmospheric
conditions. This can help improve the accuracy of weather
models and increase the lead time for severe weather warnings.
3. Cloud computing: Cloud computing provides a flexible
and scalable platform for processing and analyzing large
amounts of data. With cloud computing, meteorologists can
access computing resources on-demand, allowing them to
analyze data quickly and efficiently.
4. Visualization tools: Visualization tools can help
meteorologists interpret and communicate weather data more
effectively. For example, 3D visualizations can provide a
more intuitive understanding of atmospheric conditions, while
interactive maps can help identify areas at risk of severe
weather events.
5. Internet of Things (IoT) devices: IoT devices, such as
weather sensors and drones, can provide real-time data on
atmospheric conditions in specific locations. This data can
be used to improve the accuracy of weather models and issue
more targeted warnings to communities at risk.
Overall, these techniques are helping meteorologists to
process and analyze big data more effectively, leading to
more accurate and timely weather forecasts. As new
technologies and data sources become available, it is likely
that these techniques will continue to evolve and improve,
enabling us to better predict and prepare for the impact of
severe weather events.
Implementation
Big Data is utilized in weather forecasting through the
implementation of the MapReduce algorithm, which consists of
two important tasks - Map and Reduce. The Mapper class carries
out the mapping, while the Reducer class performs the
reduction process. The MapReduce algorithm provides a
reliable and efficient method to process data across multiple
nodes, making it easier to scale data processing. By analyzing
the available data, this approach can accurately predict
weather conditions such as thunderstorms, snowfall, or sunny
This work is distributed under the terms of the Creative
Commons Attribution-ShareAlike 4.0 International License
(https://creativecommons.org/licenses/by-sa/4.0/).
631
Proceedings of the 7th International
Scientific and Practical Conference
«Current Issues and Prospects for The No
Development of Scientific Research»
151
(April 19-20, 2023).
Orléans, France
INFORMATION AND WEB TECHNOLOGIES
weather, which are essential details that most people need to
know at the beginning of a day.
The design includes the subsequent stages: (Pic.1)
Stage 1 - Accumulation
Stage 2 - Storage
Stage 3 - Map Reduce
Stage 4 - Illustration
Accumulation: During the initial phase, the design gathers
data from multiple sources such as weather forecasts, sensor
data, market trends, and social media. For this particular
design, the data is collected from various online sources.
The collected data is then stored in a database management
system until it is transferred to a warehouse. Once the data
is transferred, it is automatically removed from the database
system.
Storage: The data collected from various sources is stored
in the Hadoop Distributed File System (HDFS), which serves as
the primary data storage system. The HDFS employs the
MapReduce programming algorithm for processing the data. The
HDFS operates as a Master-Slave system, comprising a single
name node synchronized with a backup node. The backup node is
responsible for compacting and editing the stored files in
the operating system into new files. While the data nodes can
be multiple, they divide the transmitted data into data blocks
and store them. The HDFS can multitask and perform Read/Write
operations concurrently. It is an essential component of
Hadoop, followed by the MapReduce algorithm. The HDFS is
scalable and capable of performing parallel processing to
avoid failures.
Picture 1
Flow diagram
This work is distributed under the terms of the Creative
632 Commons Attribution-ShareAlike 4.0 International License
(https://creativecommons.org/licenses/by-sa/4.0/).
Proceedings of the 7th International
Scientific and Practical Conference
«Current Issues and Prospects for The No
Development of Scientific Research»
151
(April 19-20, 2023).
Orléans, France
INFORMATION AND WEB TECHNOLOGIES
Map Reduce: This stage involves utilizing the map-reduce
framework to derive insights from the stored data. The first
step of the algorithm involves collecting different weather
predictions. Next, the input data is divided into multiple
blocks and sorted, followed by mapping the data of one set
against the other set. Once mapped, the data undergoes
shuffling, and the results are then reduced to individual
blocks. The final forecast is derived from the data with the
highest occurrences. The entire process is a highly efficient
method of organizing and restructuring a cluster of data into
individual sets. The MapReduce framework employs various
mathematical algorithms to perform these operations.
Illustration: The last stage of the process involves
presenting the filtered data, which focuses on the most
accurate and relevant weather predictions. The data can be
visualized using various graphical representations such as
line graphs, bar graphs, histograms, or pie charts.
Conclusion
In conclusion, big data is revolutionizing weather
forecasting by providing more accurate and timely information
to forecasters, enabling them to make more informed decisions
and helping society better prepare for and respond to weather
events. The vast amount of data now available, from satellite
imagery to social media feeds, is helping to create a more
complete picture of atmospheric conditions and improving the
accuracy of weather models. However, the use of big data in
weather forecasting also presents new challenges, such as the
need for advanced analytics tools and the management of
massive datasets. As the volume of data continues to grow, it
will be important for researchers and forecasters to stay
ahead of the curve and continually adapt to new technologies
and data sources. Ultimately, the benefits of using big data
in weather forecasting are clear, and the potential to save
lives and protect property from extreme weather events make
the continued investment in this area critical for the future
of our planet.
References:
[1] Sato, K., & Kuroda, K. (2021). Big Data and Weather Forecasting:
Recent Advances and Challenges. Journal of Meteorological Research,
This work is distributed under the terms of the Creative
Commons Attribution-ShareAlike 4.0 International License
(https://creativecommons.org/licenses/by-sa/4.0/).
633
Proceedings of the 7th International
Scientific and Practical Conference
«Current Issues and Prospects for The No
Development of Scientific Research»
151
(April 19-20, 2023).
Orléans, France
INFORMATION AND WEB TECHNOLOGIES
35(2), 211-222. doi: 10.1007/s13351-021-0114-4
[2] Zhang, J., Zhu, Y., & Wang, Y. (2020). Using Big Data Analytics to
Improve Weather Forecasting Accuracy. IEEE Access, 8, 116319-116329.
doi: 10.1109/access.2020.3005461
[3] Tait, A. B., Fox, N. I., & Masters, F. J. (2019). Advances in Big
Data Analytics for Weather and Climate. Annual Review of Earth and
Planetary Sciences, 47(1), 413-438. doi: 10.1146/annurev-earth-
053018-060432
[4] Hsu, P. C., Liang, X., Wu, J., & Huang, J. (2018). Weather Forecasting
with Big Data: A Review. IEEE Transactions on Big Data, 4(1), 40-54.
doi: 10.1109/tbdata.2017.2733911
[5] Kim, H. J., Kim, S. M., & Kim, S. K. (2017). Application of Big Data
Analytics in Weather Forecasting. Sustainability, 9(10), 1868. doi:
10.3390/su9101868
[6] Snook, Nathan & Kong, Fanyou & Brewster, Keith & Xue, Ming & Thomas,
Kevin & Supinie, Timothy & Perfater, Sarah & Albright, Benjamin.
(2019). Evaluation of Convection-Permitting Precipitation Forecast
Products Using WRF, NMMB, and FV3 for the 2016–17 NOAA
Hydrometeorology Testbed Flash Flood and Intense Rainfall
Experiments. Weather and Forecasting. 34
[7] Vluymans S., Sanchez-Tarrago D., Saeys Y. Fuzzy multi-instance
classifiers. IEEE Transactions on Fuzzy Systems, in press, doi:
10.1109/TFUZZ.316.2166582
This work is distributed under the terms of the Creative
634 Commons Attribution-ShareAlike 4.0 International License
(https://creativecommons.org/licenses/by-sa/4.0/).
