2024 6th International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI)

Research on Communication Resource Allocation

Strategy Optimization based on Deep Learning
2024 6th International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI) | 979-8-3315-4179-8/24/$31.00 ©2024 IEEE | DOI: 10.1109/MLBDBI63974.2024.10824006

Peiling Zuo*
School of Business, Clark University, Worcester MA, 01610, USA
* Corresponding author e-mail address: [email protected]

Abstract—As wireless communication networks become networking architecture also encounters a complex wireless
increasingly complex and communication equipment proliferates communication environment and variable interference
at a rapid pace, the demand for communication is correspondingly conditions.
rising. In order to enhance the overall performance of the network,
it is essential to utilise the available resources in an optimal In contrast, deep learning models demonstrate the capacity
manner, develop efficient algorithms, and implement effective to address complex problems. However, the training process of
policies. This work proposes a communication resource allocation such models typically requires a substantial quantity of labelled
strategy optimisation model based on deep learning, with the data, which can be challenging to obtain in the context of
objective of enhancing the overall performance of multi-cell and wireless communication due to the costs and difficulties
multi-user wireless communication networks. Initially, the model associated with gathering sufficient labelled data [3]. Furthermore,
collates and preprocesses a substantial quantity of historical the opaque nature of deep learning models hinders
communication data, encompassing the communication comprehension of their internal workings and decision-making
requirements of users, network traffic, channel state information, processes, potentially undermining the reliability and
and interference levels. Subsequently, the data is fed into a deep controllability of the models in certain applications.
neural network, and the model is trained by a backpropagation
algorithm, which continuously adjusts the network weights and Although dense deployments can markedly enhance
biases to minimise the loss function and ultimately optimise the transmission rates and overall system performance, they also
resource allocation strategy. The trained model is capable of give rise to a new set of challenges. To illustrate, high-density
generating the optimal resource allocation scheme in real time, user access has the potential to precipitate significant system
according to the prevailing network status and user requirements. disruption, which could in turn impact the performance and
This enables effective reduction of channel interference and stability of the entire system. In particular, the rapid growth in
maximisation of the efficiency and stability of signal transmission. the number of users and the increasing demand for data have
The results of the experimental analysis demonstrate that the deep resulted in a lack of high-quality information services for users
learning model not only markedly enhances the communication at the edge of the cell, which has further exacerbated the
quality of the network, but also markedly improves the efficiency performance bottleneck of the network [4].
of resource utilisation, particularly in scenarios involving high
user density and complex interference environments. Notwithstanding these challenges, the rapid development of
artificial intelligence has demonstrated the considerable
Keywords- Communication network, Allocation strategy, potential of deep reinforcement learning (DRL) to address non-
Backpropagation algorithm, Deep learning. convex optimisation problems and complex engineering
decision-making problems. The ability of DRL to autonomously
I. INTRODUCTION learn and optimise strategies in a dynamic environment, without
reliance on precise mathematical modelling, facilitates
The application of traditional optimisation techniques adaptation to complex and changing real-world application
typically necessitates the rigorous mathematical modelling of scenarios [5]. As model transparency and explanatory technology
the problem. However, in the context of actual wireless advance, the potential applications of deep learning models in
communication environments, the problem is frequently wireless communication will become increasingly diverse.
complex and dynamic, exhibiting high computational
complexity and a lack of feasibility in terms of time for solution. In order to address these issues, researchers and engineers
Furthermore, traditional methods are frequently unable to have put forward a number of technical solutions with the aim
address multi-dimensional and multi-variable dynamic of improving the performance of wireless communication
environments, and they often struggle to cope with the various networks. As an illustration, technologies such as those
uncertainties that arise in practical applications [1]. pertaining to interference management, resource optimisation,
and user enhancement at the cell edge are being progressively
In the context of 5G wireless communication networks, the implemented and enhanced [6]. These solutions are not solely
gradual mainstreaming of dense base station deployment is concerned with enhancing the overall transmission efficiency of
driven by the need to meet the requirements of high traffic the system; they are also aimed at optimising the user experience,
density, high peak rate and user experience rate. This dense particularly during periods of peak network load and in the
deployment significantly reduces the distance between adjacent presence of complex interference environments. As 5G
base stations, thereby creating a highly dense network networks continue to evolve, the effective application of these
architecture [2]. This approach facilitates efficient technologies will become a pivotal factor in driving the
communication through the utilisation of multiple access modes
and flexible frequency band resources. However, this dense

979-8-3315-4179-8/24/$31.00 ©2024 IEEE 6

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on April 08,2025 at 12:31:21 UTC from IEEE Xplore. Restrictions apply.
 sustained advancement of wireless communication network communication links and on-board equipment and roadside
performance [7]. infrastructure links. The proposed method enables multiple V2V
links to achieve efficient spectrum utilisation through the reuse
II. RELATED WORK of existing spectrum resources. The simulation results
demonstrate that the proposed reinforcement learning method is
In a recent study, Chaieb et al. [8] investigated the joint capable of approaching optimal system performance and
optimisation of user association and resource allocation in significantly enhancing the spectrum utilisation of V2X
heterogeneous networks, a process that has the potential to networks.
significantly enhance network capacity and performance.
Heterogeneous networks are characterised by a high degree of Nevertheless, while value-based deep reinforcement
complexity, given the multitude of base station types and the learning (DRL) resource allocation schemes have demonstrated
diversity of communication scenarios they encompass. This potential in enhancing network performance, they also exhibit
inherent complexity gives rise to a number of challenges in the certain limitations. In particular, value-based DRL algorithms
domains of user association and resource allocation. By typically direct the agent's decision-making process through
accurately optimising the resource allocation strategy, it is expectation modelling. However, this approach may result in the
possible to enhance network efficiency, optimise resource loss or omission of crucial information during the modelling
utilisation and significantly improve the user experience. process. This can result in the agent being unable to make the
most accurate decisions, which in turn affects the overall
In particular, the joint optimization method proposed by the performance of the system. Furthermore, the response speed and
aforementioned researchers not only considers the reasonable adaptability of DRL algorithms may be constrained in the
allocation of spectrum resources and power, but also strives to context of complex and dynamic communication environments,
optimize the access choice of users in the network. This which are characterised by frequent changes.
comprehensive optimisation strategy can be adaptively adjusted
in accordance with the specific characteristics of the network
environment and load conditions, thereby ensuring that users are III. METHODOLOGIES
able to access the optimal quality of service. In this manner, the
overall capacity of the network is enhanced, and the network is A. Deep neural network
rendered more robust and flexible in its ability to cope with high- The deep neural network model employs a multi-layered
density user access and complex interference environments. feedforward neural network structure comprising an input layer,
multiple hidden layers, and an output layer. The fundamental
Additionally, researchers Zhao et al. [9] put forth an objective of the network is to discern the intricate mapping
innovative methodology based on a Dueling Double Deep Q- relationship between the state of the communication network
Network (D3QN) to address the joint optimisation challenge of and the resource allocation strategy. This is achieved through a
user association and channel allocation in heterogeneous multi- layer-by-layer mapping and nonlinear transformation.
cell networks. This method ingeniously fuses deep
reinforcement learning technology to achieve the nuanced The input layer accepts a pre-processed, multidimensional
management of resource allocation and user association in eigenvector 𝑥 𝑥 , 𝑥 , … , 𝑥 , where 𝑚 is the number of
intricate network settings through the collaborative learning of features. Input features include the communication needs 𝐷 of
multiple agents. the user 𝑖 Channel status information 𝐻 network traffic 𝑇
interference level 𝐼 . These input features are combined into a
In particular, this method incorporates a duel network
architecture, which decomposes the value function into a state multi-dimensional vector form input network that captures
value and an advantage function. This enables a more effective complex relationships in the communication network. The
evaluation of the value of actions taken by each agent in a hidden layer is composed of multiple neurons and is nonlinearly
specific state. This architectural approach can enhance the mapped. The output ℎ of layer 𝑙 is obtained by the following
network's decision-making capacity in diverse scenarios, Equation 1.
particularly in heterogeneous network environments
characterised by constrained resources and intricate interference. ℎ 𝑓 𝑊 ℎ 𝑏 , 1
Furthermore, the incorporation of Double DQN technology where the ℎ represents the output of Layer 𝑙 1, 𝑊
effectively mitigates the issue of overestimation in Q-learning, denotes the weight matrix of Layer 𝑙, which has a dimension of
enhancing the stability and precision of the learning process 𝑛 ∗ 𝑛 , 𝑏 signifies the bias vector of layer 𝑙, which has a
through the utilisation of the target network and the behavioural dimension of 𝑛 ∗ 1 , function 𝑓  represents a nonlinear
network to calculate the Q value, respectively. This activation function, such as ReLU, Sigmoid, and so forth. In
improvement guarantees that in a multi-agent environment, each hidden layer, the output of the neurons is subjected to a
individual agents can more accurately evaluate and select the nonlinear function, which is expressed as Equation 2.
optimal resource allocation and user association strategies.
Liang et al. [10] proposes a distributed multi-agent spectrum 𝑓 𝑧 max 0, 𝑧 . 2
allocation scheme as a means of addressing the spectrum sharing An alternative approach is to utilise the sigmoid function is
problem in V2X (vehicle-to-everything) networks. This solution expressed as Equation 3.
has been designed in an innovative manner with the objective of
sharing spectrum resources between vehicle-to-vehicle (V2V)

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on April 08,2025 at 12:31:21 UTC from IEEE Xplore. Restrictions apply.
 1 𝜕𝐿
𝑓 𝑧 . 3 𝛿 ℎ 2𝜆 𝑊
1 𝑒 𝜕𝑊
The nonlinear activation functions enable the model to 𝜕𝐿
𝛿 2𝜆 𝑏 . 7
discern intricate nonlinear relationships. The function of the 𝜕𝑏
output layer is to generate resource allocation policy vectors 𝑌 As a result of repeated iterations, the network gradually
𝑦 , 𝑦 , … , 𝑦 , wherein 𝑦 represents the proportion of converges, and the resulting resource allocation strategy can
resources allocated to user 𝑖. The output layer is calculated as effectively balance user needs and network resources, minimise
follows Equation 4. the impact of interference, and improve communication
efficiency and stability. The model's capacity to handle complex
𝑦 𝑔 𝑊 ℎ 𝑏 , 4 non-linear relationships is enabled by the utilisation of nonlinear
activation functions and multi-layer structures.
where the 𝑊 represents the weight matrix of the output
layer, 𝑏 denotes the bias vector of the output layer. 𝑔  In order to guarantee the model's capacity for generalisation
signifies the activation function of the output layer. The Softmax across diverse network environments, the training process
function is typically selected to guarantee that the resource incorporates a range of communication scenario data. The
allocation ratio is non-negative and normalised and express as incorporation of the L2 regularisation term serves to mitigate the
Equation 5. potential for overfitting, thereby ensuring the model's capacity
to demonstrate robust performance in the context of evolving
𝑒 network scenarios.
𝑔 𝑧 . 5
∑ 𝑒
IV. EXPERIMENTS
The calculation of ∑ 𝑦 1 ensures that the total of the
resource allocations is equal to one. A. Experimental setups
In this experiment, a heterogeneous wireless communication
B. Loss optimization unit network environment was simulated, comprising 200 to 500
In order to optimise the resource allocation strategy, a loss users distributed evenly across multiple cells. The aim was to
function is designed that takes into account the impact of both test the performance of resource allocation under different user
resource allocation bias and interference in a comprehensive densities. The total spectrum resource is 20 MHz, with each base
manner. The loss function 𝐿 , which is defined as follows station sharing and dynamically allocating spectrum. The
Equation 6. channel model is based on the standard path loss and multipath
fading model.
Further, the experiment initiates the location of the cell base
𝐿  𝑄 𝑦  𝐼 𝑦𝑦 station, the user communication demand and the channel state,
 and employs a deep learning model for offline training. It utilises
the user's historical communication data, channel conditions and
 |𝑊|  |𝑏| , 6
interference level to optimise the resource allocation strategy.
where the parameter 𝑄 represents the target resource for The model generates an allocation scheme based on the current
user 𝑖, 𝐼 denotes the interference coefficient between users 𝑖 network state in a real-time simulation and compares it with
and 𝑗.  and  are the weight parameters, which control the several traditional methods, including fixed allocation and
weight of resource allocation error and interference impact, maximum demand priority strategies, in order to evaluate its
respectively.  and  are regularisation parameters that serve superiority in terms of improving network efficiency and user
experience.
to prevent overfitting.
Noth that, the first two terms of the loss function measure the B. Experimental analysis
accuracy of resource allocation and the impact of interference, The reachability and rate method represents a fundamental
and the last two items are L2 regularization terms, which are metric utilized for the assessment of the performance of wireless
used to constrain the complexity of the model and prevent communication networks. The term 'reachable rate' is typically
overfitting caused by excessive weights and bias vectors. employed to denote the proportion of data rate that is
To optimize the loss function, a backpropagation algorithm successfully received by users under specific network conditions.
is utilized to calculate the gradient of the loss function with This parameter is of significant importance in the assessment of
respect to the network parameters. The gradients for the weight network transmission efficiency and service quality.
matrix 𝑊 and the bias vector 𝑏 are calculated as follows Additionally, the term 'speed' is used to describe the rate of
Equation 7. data transmission, which is defined as the amount of data
transmitted per unit of time. This directly affects the quality of
the user experience. In the field of network optimisation and
resource allocation strategies, enhancing reachability and rate
results in more efficient resource utilisation and superior user

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on April 08,2025 at 12:31:21 UTC from IEEE Xplore. Restrictions apply.
 service. Figure 1 shows the the performance of the proposed spectrum resource utilization, i.e., how much of the channel
method with the traditional method under two different learning resource is used for efficient data transmission in a given amount
rates (0.001 and 0.0001) was comprehensively compared with of time. High channel utilization indicates that the network is
other methods. able to allocate and use available spectrum resources more
efficiently, improving overall network performance and data
Channel utilization refers to the degree to which channel transmission capabilities. Figure 2 shows an experimental
resources are actually used in a wireless communication network, comparison of channel utilization.
usually expressed as a percentage. It measures the efficiency of

Figure 1. Reachable Rate and Speed vs. Training Epochs with Different Learning Rates.

of Vehicles (V2X), etc., to promote the continuous innovation

and development of wireless communication technology.

REFERENCES
[1] Ajay, P., B. Nagaraj, and J. Jaya. "Algorithm for Energy Resource
Allocation and Sensor‐Based Clustering in M2M Communication
Systems." Wireless Communications and Mobile Computing 2022.1
(2022): 7815916.
[2] Xu, Yongjun, et al. "A survey on resource allocation for 5G heterogeneous
networks: Current research, future trends, and challenges." IEEE
Communications Surveys & Tutorials 23.2 (2021): 668-695.
[3] Abdu, Tedros Salih, et al. "Flexible resource optimization for GEO
multibeam satellite communication system." IEEE Transactions on
Wireless Communications 20.12 (2021): 7888-7902.
[4] Yao, Zhuohui, et al. "Resource allocation for 5G-UAV-based emergency
wireless communications." IEEE Journal on Selected Areas in
Figure 2. Comparison of Channel Utilization across Different Methods. Communications 39.11 (2021): 3395-3410.
[5] Ahsan, Waleed, et al. "Resource allocation in uplink NOMA-IoT
networks: A reinforcement-learning approach." IEEE Transactions on
V. CONCLUSIONS Wireless Communications 20.8 (2021): 5083-5098.
[6] Fang, Fang, et al. "Energy-efficient resource allocation for NOMA-MEC
In conclusion, the proposed method improves the resource networks with imperfect CSI." IEEE Transactions on Communications
utilization efficiency and system performance in heterogeneous 69.5 (2021): 3436-3449.
wireless communication networks through intelligent and [7] Jiang, Hongbo, et al. "Joint task offloading and resource allocation for
automated methods. Through the self-learning and optimization energy-constrained mobile edge computing." IEEE Transactions on
of the deep neural network model, the proposed method can Mobile Computing 22.7 (2022): 4000-4015.
generate the optimal resource allocation scheme in real time in [8] Chaieb, Cirine, et al. "On the optimization of user association and resource
the complex and changeable network environment, so as to allocation in HetNets with mm-wave base stations." IEEE Systems
Journal 14.3 (2020): 3957-3967.
improve the transmission rate and channel utilization of the
[9] Zhao, Nan, et al. "Deep reinforcement learning for user association and
network, and effectively reduce interference and resource waste. resource allocation in heterogeneous cellular networks." IEEE
In the future, with the further development of deep learning Transactions on Wireless Communications 18.11 (2019): 5141-5152.
algorithms, especially in the progress of interpretive and [10] Liang, Le, Hao Ye, and Geoffrey Ye Li. "Multi-agent reinforcement
generalization capabilities, this method is expected to be learning for spectrum sharing in vehicular networks." 2019 IEEE 20th
promoted and applied in a wider range of application scenarios, International Workshop on Signal Processing Advances in Wireless
such as 5G and 6G networks, Internet of Things (IoT), Internet Communications (SPAWC). IEEE, (2019): 1-5.

Authorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on April 08,2025 at 12:31:21 UTC from IEEE Xplore. Restrictions apply.