1

Generative AI for Physical Layer Communications:

A Survey
Nguyen Van Huynh, Jiacheng Wang, Hongyang Du, Dinh Thai Hoang, Dusit Niyato, Diep N. Nguyen, Dong In
Kim, and Khaled B. Letaief

Abstract—The recent evolution of generative artificial in- content (AIGC) era [2]. Unlike traditional AI models, which
telligence (GAI) leads to the emergence of groundbreaking focus mainly on training, analyzing, and classifying samples,
applications such as ChatGPT, which not only enhances the GAI excels in analyzing the distribution characteristics of com-
efficiency of digital content production, such as text, audio, video,
arXiv:2312.05594v1 [cs.NI] 9 Dec 2023

or even network traffic data, but also enriches its diversity. plex data across different spaces and dimensions, uncovering
Beyond digital content creation, GAI’s capability in analyzing data patterns [3]. On this basis, GAI can fully utilize the
complex data distributions offers great potential for wireless obtained features to generate outputs similar to its input data
communications, particularly amidst a rapid expansion of new and present them to users in various forms. A representative
physical layer communication technologies. For example, the example is stableDiffusion [4], which achieves state-of-the-art
diffusion model can learn input signal distributions and use them
to improve the channel estimation accuracy, while the variational scores in class-conditional image synthesis and text-to-image
autoencoder can model channel distribution and infer latent conversion. Different from existing studies focusing on image
variables for blind channel equalization. Therefore, this paper classification or segmentation, stableDiffusion focuses on the
presents a comprehensive investigation of GAI’s applications for generative, demonstrating greater flexibility and efficiency
communications at the physical layer, ranging from traditional compared to traditional content creation techniques. Through
issues, including signal classification, channel estimation, and
equalization, to emerging topics, such as intelligent reflecting the fundamental working principles of GAI models and the
surfaces and joint source channel coding. We also compare GAI- representative examples, we can see that GAI possesses two
enabled physical layer communications with those supported core capabilities. The first is the ability to analyze and capture
by traditional AI, highlighting GAI’s inherent capabilities and various features of complex data distributions. The second is
unique contributions in these areas. Finally, the paper discusses the utilization of these captured features to generate new data
open issues and proposes several future research directions,
laying a foundation for further exploration and advancement that is similar to, but distinct from, the real data. Therefore,
of GAI in physical layer communications. not only does GAI facilitate the generation of digital content,
but its potent capability for data distribution analysis also
Index Terms—Generative AI, physical layer communications,
channel estimation and equalization, physical layer security, IRS, supports research in various domains, including physical layer
beamforming, joint source channel coding. communications.
In wireless communications, a fundamental role of the phys-
ical layer communications involves converting digital data,
I. I NTRODUCTION generated by higher layers of the protocol stack, into a format
The recent surge in various large-scale datasets, combined suitable for transmitting over communication channels. This
with the ongoing progress in artificial intelligence (AI) tech- process encompasses the steps of encoding the data into a
nologies, has accelerated the development of generative AI bit sequence, modulating these bits onto a carrier wave, and
(GAI) and led to the creation of GAI based innovative applica- then propagating the modulated signal through the channel.
tions like DALL.E and ChatGPT [1]. The emergence of these Correspondingly, at the receiver, this layer undertakes the
killer applications has significantly enhanced the efficiency inverse functions, i.e., demodulating the received signal, de-
of digital content generation and enriched the variety of the coding the bit sequence, and forwarding the data to the higher
produced content, signifying the arrival of the AI-generated layers for processing [5]. Beyond these core tasks, the physical
layer is entrusted with several other key functions, such as
Nguyen Van Huynh is with the School of Computing, Engineering, and channel access, channel equalization, and multiplexing. Here,
the Built Environment, Edinburgh Napier University, Edinburgh EH10 5DT,
United Kingdom (e-mail: [email protected]) the channel access pertains to the process of determining
Jiacheng Wang, Hongyang Du, and Dusit Niyato are with the which device is authorized to transmit data over the channel
School of Computer Science and Engineering, Nanyang Technolog- at any particular moment. Equalization involves compensating
ical University, Singapore 639798 (e-mail: [email protected],
[email protected], [email protected]). for the distortion and interference that can occur during
Dinh Thai Hoang and Diep N. Nguyen are with the School of Electrical transmission over a communication channel. Multiplexing, on
and Data Engineering, University of Technology Sydney, Sydney, NSW 2007, the other hand, is the technique of amalgamating multiple
Australia (e-mail: [email protected], [email protected])
Dong In Kim is with the Department of Electrical and Computer En- data streams into a unified signal for channel transmission.
gineering, Sungkyunkwan University, Suwon 16419, South Korea (e-mail: Therefore, the physical layer is integral in shaping the overall
[email protected]) reliability, effectiveness, and performance metrics of a wireless
Khaled B. Letaief is with the Department of Electrical and Computer
Engineering, Hong Kong University of Science and Technology (HKUST), communication system [6].
Hong Kong (e-mail: [email protected]) Given its importance, researchers have conducted in-depth
 2

Section II: its key features, enhancing the system’s capability to manage
Background and Related work unknown channels [11], [12]. GAI can also improve physical
layer security, beam forming, and other various physical layer
A. Background of Generative AI
techniques. However, applications of GAI have still not been
B. Related Work
well investigated, especially for emerging technologies such as
Section III: intelligent reflecting surface (IRS), cell-free, Integrated sens-
GAI for Physical Layer Communications ing and communications (ISAC), and extremely large-scale
A. Modulation & Signal Classification MIMO. Therefore, further advancement of GAI applications
B. Channel Equalization, Modeling, & Estimation
in the physical layer communications have been receiving a
lot of attentions recently.
C. Physical Layer Security
Facing the emerging challenges in the physical layer com-
D. Intelligent Reflecting Surface
munications and considering the potential unique support
E. Beamforming offered by GAI, this paper provides a comprehensive survey
F. CSI Feedback of GAI’s applications to address diverse problems in phys-
G. Radio Map & Channel Delay Estimation ical layer communications. We further discuss comparisons
H. Joint Source Channel Coding between techniques in the physical layer that are supported by
Section IV: GAI versus those relying on traditional AI models. After that,
Open Issues and Future Research Directions we discuss the lessons learned from existing studies, empha-
sizing the key capabilities of GAI employed in these instances.
A. Security and Privacy Lastly, the paper highlights open issues and discusses future
B. Model-driven GAI research directions. The key contributions of this paper are
C. Resource-efficient Learning summarized as follows.
D. Real-time Adaptation • From our in-depth investigation, we reveal how to apply
Section V: different GAI models to solve various physical layer
Conclusion issues. These GAI models encompass not only common
ones like generative adversarial networks (GANs) and
Fig. 1. The overall structure of this paper.
variational autoencoders (VAEs), but also currently pop-
ular diffusion models. Additionally, the physical layer
studies on the physical layer, including techniques like beam- communication issues covered in our survey range from
forming, modulation and demodulation, signal detection, chan- traditional ones like modulation, signal classification, and
nel estimation, and channel state information (CSI) compres- channel equalization to emerging technologies such as
sion. These techniques are directly linked to the analysis, IRS.
compression, as well as the feature extraction of complex • We examine the problems in physical layer communica-
physical layer data. Conventional research relies on mathemat- tions supported by traditional AI models and illustrate
ically expressed models. However, in practical applications, how physical layer techniques empowered by GAI can
the systems could include unknown effects that are almost address these problems. This reveals the unique support
impossible to be expressed analytically. Therefore, AI models GAI can offer to the physical layer, beyond the capa-
have been introduced to support the physical layer functions of bilities of traditional AI, underscoring the importance of
wireless communications. For instance, deep neural networks further integrating GAI with physical layer techniques,
(DNNs) can learn the relationship between channel inputs and particularly in dealing with various emerging technolo-
outputs to enhance the accuracy of channel estimation, thereby gies.
supporting the physical layer from various perspectives, such • We provide an in-depth analysis and summary of the
as signal detection, channel equalization, and synchroniza- GAI’s applications in the physical layer communications,
tion [7]. In addition, deep learning (DL) models were also finding that these works primarily leverage three core
applied to support the physical layer communications. For ex- capabilities of GAI. These include the ability to cap-
ample, recurrent neural networks (RNNs) can assist decoding ture complex data distributions, the capability for cross-
[8], autoencoders can reduce peak-to-average power ratio [9], dimensional data transformation and processing, and the
and Convolutional Neural Networks (CNNs) can compress potential to repair and enhance data. This summary serves
the CSI in a massive multiple-input multiple-output (MIMO) as vital guidance for further advancing the applications of
system [10]. GAI in the physical layer.
• We present significant open issues when applying GAI in
Although the traditional AI models are effective, their
the physical layer communications from several perspec-
performance is limited. For example, DNNs can learn channel
tives, such as privacy, security, and resource optimization,
models, but they may struggle or even fail when dealing
and provide some directions for future research.
with channels that are unknown during training. Therefore,
researchers introduce GAI, which can not only generate more The structure of this survey is outlined in Fig. 1. Section II
channel samples to enhance the training data set, but also assist offers a review of related works, while Section III delves into
in analyzing the distribution of existing data and extracting an in-depth analysis of existing studies. Section IV discusses
 3

TABLE I
LIST OF ABBREVIATIONS

Abbreviation Description Abbreviation Description

AI Artificial Intelligence DL Deep Learning
TAI Traditional Artificial Intelligence GAI Generative Artificial Intelligence
AIGC AI-generated content RNN Recurrent Neural Network
DNN Deep Neural Network CNN Convolutional Neural Network
CSI Channel State Information ML Machine Learning
SNR Signal-to-Noise Ratio GAN Generative Adversarial Network
BER Bit Error Rate PSK Phase-Shift Keying
VAE Variational Autoencoder NF Normalizing Flow
MIMO Multiple-Input Multiple-Output QPSK Quadrature Phase-Shift Keying
mmWave Millimeter Wave UAV Unmanned Aerial Vehicle
NMSE Normalized Mean Square Error PLS Physical Layer Security
DRL Deep Reinforcement Learning RF Radio Frequency
IRS Intelligent Reflecting Surface BS Base Station
UE User Equipment FNN Fully-connected Neural Network
JSCC Joint Source Channel Coding PSNR Peak Signal-to-Noise Ratio
AWGN Additive White Gaussian Noise WGAN Wasserstein GAN
SCMA Sparse Code Multiple Access MMSE Minimum Mean Square Error

open issues and future research directions, and Section V con- input data into a latent representation, and a decoder that
cludes the paper. Additionally, Table I lists the abbreviations rebuilds the data from this latent space. These components
widely used throughout this survey. are typically multi-layer neural networks. VAEs optimize
their parameters by minimizing a loss function that assesses
II. BACKGROUND AND R ELATED W ORK reconstruction accuracy and aligns the latent space distribution
This section discusses the background knowledge about with a prior distribution. Key advantages of VAEs include their
GAI and some related surveys, and illustrates the differences ease of implementation and training, effectiveness in learning
between this survey and existing work. compressed data representations, and a probabilistic nature
that allows for uncertainty estimation and varied outputs [20],
thereby providing unique support for signal classification [21]
A. Background of Generative AI
and joint source-channel coding [22]. However, they present
This part introduces the fundamental principles and char- challenges in training and parameter tuning, with the possibil-
acteristics of four mainstream GAI models, including GANs, ity of non-interpretable compressed representations.
VAEs, normalizing flows (NFs), and diffusion models, as these
models are frequently utilized in improving physical layer 3) Normalizing Flows: NFs are generative models that
communications. transform simple probability distributions into complex ones
1) Generative Adversarial Networks: A GAN consists using reversible transformations. Unlike VAEs and GANs, they
of two main elements, i.e., a generator that produces data employ invertible neural networks for these transformations,
mimicking real data, and a discriminator that differentiates which include a deterministic mapping function and an ad-
between the real and generated data. The training process justable scaling and shifting function [23]. The representative
aims for a Nash equilibrium, where the discriminator cannot examples are the Real NVP [24], which uses affine coupling
differentiate between the two [13]. Trained GANs are capable layers, and the Masked autoregressive flow [25], based on
of reconstructing high-dimensional data from low-dimensional autoregressive models. The advantages of NFs lie in efficiently
input with fewer generator function restrictions compared sampling complex distributions, managing high-dimensional
to other models, which makes them especially proficient in data, and learning interpretable latent spaces, which can en-
channel estimation [14] and CSI compression [15]. Despite hance physical layer techniques, such as signal classifica-
these advantages, GANs’ training complexity lies in achieving tion [26]. However, the challenges include high computational
the Nash equilibrium, which is more challenging than opti- demands, lengthy training for complex distributions, and trans-
mizing an objective function. This leads to the development formation function selection. To overcome these, recent studies
of various GAN derivatives, such as StackGAN [16] and have explored optimizing architectures and training efficiency
PAN [17], focusing either on architecture or objective function through techniques like adversarial training and regularization,
optimization [18]. These models can be applied across multiple demonstrating NFs’ potential in diverse applications.
fields like image processing, sequential data handling, and 4) Diffusion Models: Unlike the above mentioned GAI
even in drug discovery and malware detection. models, diffusion models start with adding noise to training
2) Variational Autoencoders: VAEs are neural networks samples, which is known as the forward diffusion process,
designed for compressing and reconstructing data. They differ and then remove the noise to generate new samples in the
from traditional autoencoders by using probabilistic meth- inverse process [27]. They can be trained on incomplete data
ods to model and generate data from a compressed latent in a stable process, enabling them to assist in physical layer
space [19]. The VAE comprises an encoder that translates technologies like channel modeling [28]. However, diffusion
 4

Generated sample
Generator x Discriminator
Flow f ( x ) Inverse f ( z )
−1
z G ( z) x D ( x, x ) 0/1 x z x
Input noise
x Input Output
Real sample
The structure of the GAN The structure of the NF
Mapping Reconstructing Add noise to disturb the data

x
Encoder
z
Decoder x0 x1 x2 Noise
q ( z | x ) p ( x | z ) x
Latent vector Reconstructed Denoise to generate the data
input
The structure of the VAE The structure of the diffusion model

Fig. 2. The structure of four GAI models.

models face challenges such as longer sampling times, com- delays, authors in [45] discuss the need for real-time DL
plex training architectures, and limitations with certain data in the physical layer, summarizing the current advancements
types [29]. To address these issues, researchers have developed and limitations in this area. The aforementioned surveys are
optimization techniques, such as improving the training speed summarized in Table II. The existing surveys about AI-enabled
by reducing variance stochastic gradient descent, adaptive physical layer technologies and GAI, as discussed, provide two
learning rate, and weight normalization. critical insights.
In Fig. 2, we present the structures of the four aforemen- • From the perspective of GAI, the existing surveys primar-
tioned GAI models. Additionally, there are other GAI models ily discuss the principles, architectures, implementation
such as Transformers [30]. These models also possess strong methods, and the strengths and weaknesses of different
data analysis and modeling capabilities, potentially providing mainstream GAI models. Furthermore, researchers ana-
support for physical layer communications. lyze the applications of GAI in various domains such as
industrial Internet of Things and mobile networks with a
B. Relate Work variety of applications, and provide future prospects and
1) Generative AI: Given the GAI’s growing popularity, potential challenges from various aspects, such as ethical
numerous surveys have recently emerged. These surveys fo- impacts and risks.
cus primarily on the fundamental architecture [31], [32], • Regarding AI-enabled physical layer communications,
principles [33], implementation methods [34], as well as existing works review the physical layer technologies and
applications [3], [35]–[39] of GAI models. For instance, the various DL techniques. Besides, they present a detailed
authors in [32] provide a review on the GAI’s history, basic discussion of how DL supports various physical layer
components, and recent advances in AIGC across the unimodal technologies, including signal compression and detection,
and multimodal interactions. Technically, the authors in [33] coding theory, attack detection, physical layer authenti-
present a survey on various deep GAI models, comparing these cation, and so forth.
models, elucidating their underlying principles, interrelations, Despite the comprehensiveness of these surveys, a gap
and reviewing current advancements and applications. For remains in exploring GAI’s applications in the physical layer
GAI’s applications, the authors in [3] present a practical guide communications. Given the challenges posed by emerging
on using GAI for network optimization, demonstrating its technologies to the physical layer and the unique potential of
effectiveness and contributing to network design. In industry, GAI, this paper delves into how GAI underpins physical layer
the authors in [37] examine GAI’s role in the industrial Internet technologies. We further enhance this exploration by contrast-
of Things, focusing on the protection of trust-boundary and the ing GAI-assisted physical layer technologies with those reliant
prediction of network traffic, while highlighting challenges to on traditional AI models, thereby addressing current research
accelerate its adoption. Regarding the emerging Metaverse, gaps and providing insights into the ongoing evolution of GAI
the authors in [38] explore GAI’s facilitative role in its in physical layer communications.
development, providing a research roadmap and addressing
ethical implications.
2) AI Enabled Physical Layer Communications: AI mod- III. G ENERATIVE AI FOR P HYSICAL L AYER
els are crucial for advancing physical layer communications, C OMMUNICATIONS
spurring numerous research surveys. These studies primarily In this section, we provide a comprehensive review of
concentrate on the application of DL in various domains, various applications of GAI for physical layer communica-
including signal detection and compression [40], coding [6], tions. In particular, we first highlight GAI-based approaches
[41], [42], security [43], [44], and communication delay [45]. for modulation recognition and signal classification. Then,
For instance, the authors in [42] survey recent advances in the applications of GAI for channel modeling and estima-
DL-based coding, focusing on enhancing the specific coding tion, physical layer security, beamforming, and joint source
method using DL techniques. About the security, the authors channel coding (JSCC) are discussed in detail. Finally, we
in [43] offer a review of DL-based security techniques for review applications of GAI for emerging problems in wireless
addressing issues like attack detection and authentication in 5G communications, including IRS, CSI feedback, and radio map
and beyond networks. Given the importance of communication estimation.
 5

TABLE II
S UMMARY OF THE R ELATED W ORKS

Ref. Issue Key focus of survey

An overview of some GAI models and architectures, training procedures, and limitations of three typical GAI models.
[31]
Generative A summary of the history and fundamental components of GAI, along with recent progress in AIGC involving
[32] AI unimodal and multimodal interactions.
The principles, interrelations, current advancements, and applications of several GAI models.
[33]
The algorithms and implementation methods of several GAI models, as well as some guidance on selecting GAI
[34] models.
Technological development of various AIGC and application of GAI in education and creativity content.
[35]
An exploration of the advantages and disadvantages of using ChatGPT in educational contexts and some limitations
[36] of the ChatGPT.
The state of the art of GAI models and their use in industrial Internet of Things, such as trust-boundary protection,
[37] anomaly detection, and so forth.
GAI’s applications in Metaverse, such as avatars, non-player characters, and virtual world generation, automatic
[38] digital twin, and so forth.
An extensive overview of recent challenges and developments in applying GAI within mobile communications
[39] networks.
A tutorial of using generative diffusion model in network optimization.
[3]

A survey of the recent advancements in DL and its application in signal compression and detection.
[40]
An investigation about the DL-based physical layer, including using DL to redesign the modules in the traditional
[6] communication system and replace the communication system with autoencoder-based architecture.
AI Enabled Discuss some new applications of DL in the physical layer and present an autoencoder-based physical layer
[41] Physical communication system.
Layer Com-
munications An overview of recent advances of DL’s applications in coding by focusing on sequential codes and Turbo codes.
[42]
Examine the necessity of real-time DL in the physical layer and provide a summary of the current developments
[45] and their limitations.
A detailed examination of different DL and deep reinforcement learning (DRL) methods suited for physical layer
[43] security applications.
The integration of machine learning with the selection of relay nodes, antennas, and authentication processes.
[44]

A. Modulation and Signal Classification nent solution. This is because DL-based approaches can
leverage DNNs to learn the relationship between the channel
Signal classification and modulation recognition are always inputs and channel outputs, resulting in data-driven signal
among the most important components in designing receivers detection without requiring any knowledge of channel models.
of wireless communication systems [46]. Specifically, the Unfortunately, DL-based solutions require large datasets and
goal of radio signal classification is to accurately recover long training time to obtain good detection performance [47],
information transmitted from the transmitter over a noisy [48], especially when channel environments change fast due to
wireless channel by analyzing received signals based on dif- the user mobility. In practice, collecting and processing enough
ferent techniques, such as maximum-likelihood and minimum training data are costly, time-consuming, and sometimes im-
mean square error (MMSE). On the other hand, modulation possible. Moreover, DL-based approaches cannot effectively
recognition aims to detect the modulation technique used at the deal with the dynamics and uncertainty of wireless commu-
transmitter and obtain the original transmitted signals by using nications. In particular, a trained DL model only works well
the corresponding demodulation approach. Signal classifica- with some specific wireless environments that have similar
tion and modulation recognition have been extensively studied characteristics to the trained environment. In new wireless
and developed since the creation of wireless communications. environments with different conditions, e.g., channel models,
However, traditional approaches like maximum-likelihood and surrounding interference, and noise distributions, this trained
MMSE require a specific sophisticated mathematical model model will need to be retrained with a huge volume of
for each type of wireless channels and environments [6]. new training data, which may not be feasible in practice. In
In addition, perfect or highly accurate knowledge of the addition, conventional DL-based solutions are less effective
underlying channel and CSI are usually required to obtain in modeling complex wireless channels that are time-varying,
good detection performance [11]. However, these approaches non-i.i.d distributed, or non-differentiable [11], [12]. To deal
appear to be ineffective in future wireless communication with these limitations and facilitate applications of DL in
systems due to the increased complexity of signals, spectrum modulation and signal classification, GAI, with its great capa-
efficiency requirements, and the dynamics and uncertainty of bilities to understand, capture, and generate the distribution of
UEs’ behaviors and characteristics. complex and high-dimensional data [49], [50], is a promising
To overcome these challenges, DL is emerging as a promi- approach, as summarized in Table III.
 6

Specifically, the authors in [11] point out that traditional Transmitter Receiver

DL-based approaches do not perform well with non-Gaussian

Transmitter Antenna Coupling

Receiver Antenna Coupling

and time-varying channels, especially in the low signal-to-

Channel Decoder
Channel Encoder

Source Decoder
Source Encoder

Demodulation
noise ratio (SNR) regions. For that, they propose a novel

Modulation
Destination

Channel
a) Source
GAN to help the receiver intelligently adapt to the dynamics
of wireless channels without retraining DNNs. In particular,
the proposed GAN is used to efficiently learn the channel
transition probability, i.e., the likelihood function. Then, the
estimated channel transition probability is fed into the Viterbi
algorithm [51] to derive the maximum-likelihood sequence
Transmitter/ Receiver/
detection. Moreover, the authors develop an online adjustment Auto-encoder Auto-decoder
𝒔 𝒙 𝒚 𝒔ො

Channel
policy to fine-tune the proposed GAN network by leveraging b)
the soft output of the model as well as pilot signals, making
it more effective with time-varying wireless channels. The
numerical results then demonstrate that the proposed GAN
network can achieve a bit error rate (BER) of 10−2 at 8 dB
Transmitter/ Receiver/
Channel GAN
SNR while the ViterbiNet approach [51] can only obtain this Auto-encoder Auto-decoder
𝒔 𝒙 𝒚 𝒔ො
level of BER at 12 dB SNR. Moreover, the authors show that c)
by using GAN they can obtain near-optimal BER performance
under dynamic channel conditions.
Considering the same GAI method, the authors in [12]
also develop a GAN network to model unknown channels Received
pilot signals
Received
pilot signals
in end-to-end wireless communication systems. As depicted
in Fig. 3, the authors first consider an end-to-end commu- Fig. 3. Architectures of traditional wireless communication systems and
nication system in which all the signal processing blocks at end-to-end learning-based communication systems: (a) traditional wireless
systems, (b) end-to-end communication systems based on autoencoder, where
the transceivers are replaced by DNNs to jointly optimize the the transceivers are represented by DNNs, and (c) end-to-end communication
performance of the whole system. To do that, traditional DL- systems with channel GAN [12].
based approaches usually assume the availability of CSI and
prior channel knowledge which are not always available in
practice. To tackle this challenge, the authors design a novel dynamic spectrum sensing or signal sensing. The experimental
conditional GAN network to represent the channel between results then show that the proposed GAN network can preserve
the transmitter and the receiver to allow the gradient from the each modulation type’s global structure and restore up to
receiver to back-propagate to the transmitter. Moreover, the 50% missing samples in the time domain. Differently, the
pilot signals received at the receiver are used as the conditional authors in [54] propose a GAN-based modulation classification
information of the proposed GAN network, as illustrated in approach that is resilient to adversarial attacks. Specifically,
Fig. 3(c). In this way, the GAN network can generate more the authors indicate that conventional DL-based automatic
realistic coefficients for time-varying channels, and thus the modulation recognition methods are vulnerable to adversarial
end-to-end loss can be optimized to minimize the BER of attacks with well-designed perturbation injected into wireless
the system. Interestingly, the authors demonstrate that the channels. To tackle this practical issue, the authors propose
Kullback-Leibler divergence of the proposed GAN network a novel GAN network to generate plausible samples that are
can be significantly reduced when training the model over a similar to the received frames. The generated frames are then
long period, indicating that the generated data’s distribution compared with the perturbed received signals to detect the
converges to the target distribution. One potential research true class of the modulated signals. The authors then revealed
direction is to test the proposed architecture in real wireless that using only one generator may face the mode collapse
scenarios to evaluate its effectiveness in dealing with various problem when dealing with multiple modulation types. As
imperfections of wireless channels. such, multiple generators are incorporated into the proposed
Besides signal classification, GAN can also be adopted GAN network with each generator being used to deal with a
for modulation recognition. For instance, the authors in [52] specific type of modulation. In this way, the proposed solution
highlight that an application of DL for signal modulation can work well with various modulation types. Simulation
recognition is often hindered by insufficient training data results show that the proposed GAN model can significantly
and overfitting. As such, the authors propose an auxiliary increase the accuracy of DL-based modulation recognition
classifier GAN to enlarge the training dataset by generating methods under adversarial attacks. For example, the recog-
new data while maintaining high-level features learned from nition accuracy for 8 phase-shift keying (PSK) scenarios can
the original training data. The authors then demonstrate that be increased from 9% to around 70% by using the proposed
the proposed GAN solution can increase the classification GAN model.
accuracy by up to 6% compared to conventional DL-based While most existing works in the literature adopt GAN,
solutions, e.g., AlexNet. Similarly, the authors in [53] propose VAEs and NFs have been gaining attention recently [21],
a GAN network to restore missing signals due to errors in [26], [57], [58] due to their capabilities in dealing with
 7

TABLE III
S UMMARY OF GAI A PPROACHES FOR M ODULATION R ECOGNITION AND S IGNAL C LASSIFICATION

No. Problem Drawbacks of TAI Proposed Approach

Poor performance due to insufficient data and over- Using the auxiliary classifier GANs to enlarge training
[52]
fitting datasets
Modulation
[55] Require large numbers of labeled samples Use GAN to generate samples from noise and labeled data
Recognition
Propose a novel GAN network that consists of four
Vulnerable to adversarial attacks with well-designed
[54] generators to improve the model’s accuracy and robustness
perturbation
against adversarial attacks
Require a large amount of training data, and the
Propose a GAN-based method to generate missing wire-
[53] training accuracy can be greatly affected by the
less signal samples
training data’s quality
Use GAN to generate a large training dataset without
Lack of clean training dataset. Information loss dur-
[56] requiring manual annotation. Discriminative model can
ing the feature extraction process.
improve the signal classification process
Poor performance due to the dynamics of wireless
[12] Propose a conditional GAN to represent channel effects
channels
[21] Signal Clas- Not effective in estimating posterior distributions Use VAE to approximate intractable posterior distributions
sification Use VAE to simplify the maximum-likelihood estimation
[57] Enormous data labels are required
which contains latent variables
Require a huge volume of training data. Not perform
Use GAN to directly approximate the transition probabil-
[11] well when the underlying channel models are com-
ity of the underlying wireless channel
pletely unknown
Performance is not guaranteed when the noise statis- Leverage a NF to effectively learn the distribution of
[26]
tics is unknown unknown noise
Use VAE as a probabilistic model to recover transmitted
[58] Require more bandwidth resources
symbols
Use VAE as a probabilistic model to recover transmitted
[59] Sub-optimal in certain cases
short-packet symbols

Latent vector posterior probability by performing variational inference in

𝑅(𝒉 ) a VAE network. Practically, the proposed VAE network can
𝒛𝒊 ≜ 𝐼(𝒉 𝐾)
𝐾
approximate intractable posterior distributions by training an
Unsequeeze
encoder to map input data to a latent distribution and training
a decoder to estimate the inputs, making it more effective than
𝒉𝐾
conventional DL approaches in classifying received signals
K-th Flow step when approximating the posterior distribution is complex.
Normalizing Flow ×𝐾
Simulation results then reveal that the proposed approach can
obtain near-optimal maximum-likelihood performance under
1st Flow step
different single antenna settings. By using the NF technique,
𝒉0 ≜ 𝑅(𝒘𝑖 ) 𝐼(𝒘𝑖 ) the authors in [26] propose a novel signal detection frame-
Sequeeze work, which is fully probabilistic, to approximate unknown
𝑅(𝒚) 𝑅(𝒘𝑖 ) noise distributions. Specifically, the authors consider the signal
𝐼(𝒚)
≜ ഥ
𝒚 + 𝒘𝑖 ≜
𝐼(𝒘𝑖 ) detection problem in MIMO systems with unknown statistical
- Noise vector knowledge of noise which is very challenging for traditional
𝑅(𝑯) −𝐼(𝑯)
𝐼(𝑯) 𝑅(𝑯)
≜ ഥ
𝑯 x DL-based approaches. The authors then utilize the NF tech-
nique to design a flexible detection framework that does not
𝑅 𝒙𝑖
𝒙ഥ𝑖 ≜
𝐼 𝒙𝑖
𝑤ℎ𝑒𝑟𝑒 𝒙𝑖 ∈ 𝑿 require any noise statistics as depicted in Fig. 4. The proposed
NF is constructed by three major components, including
Possible signal vector
an unsqueeze layer, 𝐾 flow steps, and a squeeze layer. To
obtain the maximum-likelihood estimation, the authors first
Fig. 4. Architecture of the detection framework with NFs consisting of three
main components: (i) a squeeze layer, (ii) 𝐾 flow steps, and (iii) an unsqueeze calculate the noise vector w𝑖 = y − Hx𝑖 , corresponding to
layer. The noise vector for each possible signal vector is first calculated and signal vector x𝑖 with received signal y and channel matrix
fed into the NF. After that, the output of the NF is mapped into the latent H. The proposed NF then maps w𝑖 into the latent space
space z𝑖 [26].
which consists of latent variable z𝑖 and the log-determinant.
In this way, the corresponding likelihood 𝑝(y|xi ) can be
calculated, resulting in accurate maximum log-likelihood esti-
signals in the time domain. For example, the authors in [21] mation. Extensive simulations demonstrate that the proposed
consider the signal classification problem in MIMO orthogonal framework outperforms existing DL-based methods in terms
frequency division multiplexing with index modulation sys- of BER under non-analytical noise settings. For example, in
tems. In particular, due to the high complexity of calculating the quadrature phase-shift keying (QPSK) modulated 4 × 4
the posterior probability, the authors estimate the variational
 8

MIMO system, the proposed approach can reduce the detection encoder and then quantized by the adaptive one-bit analog-
error of the DetNet architecture [60] by 39.61% with SNR=25 to-digital converter to reduce the energy consumption of the
dB. However, the performance gap between the proposed receiver. The generator of the proposed GAN architecture is
method and the traditional maximum-likelihood approach is used to approximate the distribution of encoded signal er given
still noticeable. One potential solution is leveraging the auto- quantized signal q as its input. The discriminator then can
distribution technique to further improve the convergence of distinguish the real encoded signal er and synthetic encoded
the proposed method in unknown noise conditions. signal ef produced by the generator. Finally, the decoder will
be used to recover the transmitted signal. In this way, the
authors can construct a generalized channel equalization to
B. Channel Equalization, Modeling, and Estimation
equalize the one-bit quantization’s distortion as well as the
In wireless communications, channel equalization, model- severe channel fading of underwater environments.
ing, and estimation play essential roles in helping the receiver Due to its capabilities in analyzing signals in the time
detect the received signals more efficiently. In particular, domain, VAEs have been widely adopted for channel equal-
channel equalization refers to the process of compensating for ization recently. For example, the authors in [64] propose to
the distortions incurred when transmitting signals through the use VAEs for blind channel equalization which is challenging
communication channel. On the other hand, channel modeling for conventional AI approaches. In particular, an encoder
and channel estimation aim to create a mathematical model is used to represent the channel model and noise, and a
for the communication channel and estimate the parameters decoder is used to approximate the posterior distribution of
of the channel model, respectively. Over the past few years, transmitted symbols from the received signals. In this way,
DL has been widely adopted for channel equalization, mod- VAEs can effectively model complex channel distributions
eling, and estimation both in academia and industry [7], [61]. and perform inference on latent variables. The authors then
Unfortunately, DL-based approaches require a huge volume extend this work and propose a VAE equalizer for noisy
of labeled data to learn sufficient characteristics of a spe- linear and nonlinear channels in [65]. They demonstrate that
cific channel, and thus limiting their application in dynamic the proposed VAE equalizer significantly outperforms baseline
wireless environments with high levels of randomness and blind equalizers and can obtain similar performance to that
variability. In addition, standard neural networks can work well of a non-blind adaptive linear MMSE equalizer while not
for discriminative tasks but perform poorly when modeling the requiring prior knowledge of impulse responses as well as
full complexity of channel distributions. Finally, conventional pilot signals. Differently, the authors in [66] propose to use
DL-based methods use a general loss function that makes their a diffusion model to remove channel noise. In particular, the
predictions less accurate, especially in low SNR regions [62]. proposed channel denoising diffusion model is added as a new
For that, GAI has been adopted widely recently for equalizing, physical layer module right after the channel equalization to
modeling, and estimating wireless channels, as summarized in learn the input signals’ distributions and then leverage them
Table IV. Compared with conventional DL techniques, GAI to further remove the channel noise. Experiments demonstrate
possesses several advantages. Specifically, GAI can generate that the proposed diffusion model can significantly reduce the
synthetic training data that is similar to the data it was trained mean square error and outperform existing approaches. For
on for ML models of channel estimation. In addition, it can example, at SNR=20 dB under Rayleigh fading, the proposed
generate data following specific constraints or conditions and diffusion model can achieve a 1.06 dB gain compared to the
leverage data from a source system to generate training data joint source-channel coding system.
for a target system. All these special features make GAI 2) Channel Modeling: GAI also finds its applications in
an ideal tool for channel modeling. Moreover, GAI can be channel modeling [28], [67], [68]. In [67], the authors propose
used as an equalizer to learn the mapping from distorted to use GAN to model millimeter wave (mmWave) channels.
signals to transmitted signals as well as to model the posterior They highlight that accurately modeling mmWave channels
distribution of transmitted signals and then estimate clean is challenging due to several factors such as multiple high
signals from distorted observations at the receiver. frequencies and highly directional beams. For that, the authors
1) Channel Equalization: In [63], the authors develop a design a GAN approach to generate random profiles that
hybrid GAN and autoencoder approach for channel equal- include all information about the channel, including channel
ization of underwater wireless communications with one-bit gains, delays, angle of arrival, and angle of departure of all
quantization. Specifically, it is highlighted that underwater links between the receiver and the transmitter. Simulation
wireless communications are extremely vulnerable to severe results then show that by using GAN, the authors can generate
channel fading caused by the scattering and absorption of new channel data that have almost the same cumulative
underwater environments. Moreover, the strong nonlinearities distribution function as real data. With this newly generated
of one-bit quantization can greatly affect communication reli- data, the authors then can effectively model mmWave channels
ability. Given these challenges, using conventional DL-based by capturing the joint distribution of all links between the
approaches, e.g., autoencoder, may not yield good commu- transmitter and the receiver with multiple frequencies. Simi-
nication performance. For that, the authors propose to inte- larly, the authors in [28] also address the lack of data problem
grate GAN into their autoencoder architecture to significantly in modeling wireless channels by proposing a diffusion model.
improve the channel equalization performance as illustrated The proposed diffusion model can learn to generate new data
in Fig. 5. Specifically, input signal s is first encoded by the samples by iteratively adding noise to the previous input,
 9

Generator

Reshape

Normalization
𝒛 𝒆𝒇
… …

……

……

……

…
…

…
Fake

Decoder & Discriminator

𝒒
Encoder
Under water

Reshape
optical
Normalization

quantization
Binarization
channel

Modulation
𝒔

Adaptive
𝒆𝒓 𝒒 𝒔′

one-bit
… 𝒙

OOK
… …

……

……

……

…
…
…

…
…

𝒆𝒓
Real

Fig. 5. Training structure of the proposed hybrid AE-GAN in which the encoder encodes input signal s, the generator learns the distribution of the real
encoded signal er with the quantized signal q as its input, and finally the discriminator distinguishes the real encoded signal er and the fake encoded signal
ef generated by the generator [63].

resulting in a more stable training process compared to existing Online Training Stage The selected
GAN approaches, as demonstrated by the authors. number of
iterations

In [68], the authors introduce a distributed GAN approach Training & Receive Trained
to model mmWave channels in unmanned aerial vehicle Sequences 𝒑𝑘 , 𝒚𝑘 Input GAN Obtain Generator
𝒇𝑅 (𝒚𝑘 ) Training Network
(UAV) networks. In particular, the authors state that existing Distribution of
latent variable 𝒑𝑧
𝐺(𝒛|𝒎; 𝜽𝒈 )

approaches for channel modeling using conventional AI as

well as centralized GAN are limited by the lack of training
channel samples and environmental measurements. For that, Channel Estimation Enhancement Stage
they propose to use UAVs to collect mmWave channel data Trained
Mimic
Generate
Generator
during their aerial services. Each UAV employs GAN to train a Network 𝒇𝑅−1 (. )
Receive
Sequence 𝒚ෝ𝒊𝒌
𝐺(𝒛|𝒎; 𝜽𝒈 )
local channel model. After that, the generated channel samples (Multiple times)

produced from the local channel model will be shared with

Training & Receive
other UAVs in the networks to improve their training process. Sequences 𝒑𝑘 , 𝒚𝑘
Input LS Estimation
Channel
Estimation
Sequence
Extensive simulations show that the proposed distributed GAN Distribution of 𝒇𝑅 (𝒚𝑘 ) Enhancement ෝ𝑐𝑜𝑚,𝑘 , 𝒑𝑐𝑜𝑚,𝑘
𝒚 Enhancement
෡ 𝑮𝑨𝑵,𝒌
latent variable 𝒑𝑧 𝒉
approach can significantly improve the modeling accuracy as
well as increase the communication rate by 10% under real-
time channel estimation compared to standalone training.
Fig. 6. The flowchart of the proposed GAN-based channel estimation. The
3) Channel Estimation: Besides channel equalization and training sequence p 𝑘 and the receive sequence y 𝑘 are used as the input data
channel modeling, GAI has been widely adopted in the liter- of the GAN architecture. The generator then “mimics” receive sequences ŷ 𝑘
matching with the distribution of the true channel [69].
ature for channel estimation. For example, the authors in [70]
propose a GAN architecture for wideband channel estimation
in mmWave and THz communications. The authors highlight
that DL has been widely adopted for channel estimation GAN approach can obtain higher channel estimation accuracy
in recent years. However, conventional DL-based approaches with 70% fewer pilots compared to the traditional CNN
require long pilot sequences to achieve good estimation per- networks (e.g., ResNet). Interestingly, the proposed solution
formance. Moreover, they provide poor channel estimation can work well when changing the environment’s factors such
performance under high channel correlations and high propa- as the number of rays and clusters without retraining the GAN
gation losses. For that, the authors propose to use GAN to network. Differently, the authors in [69] adopt GAN during the
estimate frequency selective channels at low SNR regions online training phase to further improve the channel estimation
with short pilot sequences. Specifically, the proposed GAN performance as illustrated in Fig. 6. Specifically, the receive
approach can learn to generate realistic channel coefficients sequence y 𝑘 and the training sequence p 𝑘 are fed into the
based on a real-world but unknown channel distribution during proposed GAN architecture as its input data for training. The
the offline training phase. After that, the trained GAN network generator network then can “mimic” receive sequences ŷ 𝑘
is used as a prior model for online channel estimation by matching with the distribution of the true channel. After that, a
optimizing the input vector of the model based on the current newly proposed enhancement algorithm will perform channel
signal received at the receiver. By doing this, the proposed estimation based on these new receive sequences. Simulation
 10

Adversarial
Generator Global Aggregation Noise (I/Q Generator)
𝑚(𝑡)
Modulation Scheme n(𝑡)
Feedback
Discriminator
r(𝑡) Generator g(𝑡)
Signal Amplitude
𝑠𝑡 G
I(𝑡)
Signal Phase

Trusted?

Trusted Transmitter D
Fake?
I/Q Data (Through
Feature Extractor
SDR)
User 1 User 2 GAN Implementation
User 3
Local Local Local
Dataset 1 Dataset 2 Dataset 3
Fig. 8. Proposed GAN architecture for authenticating RF transmitters in
which the generator uses RF signals generated from adversaries as its input
to generate synthetic data 𝑔 (𝑡 ) and the discriminator takes input from both
Update Weights Broadcast Weights Update Regularizer
the generator and “trusted” transmitters to learn the differences between real
and fake RF signals [87].
Fig. 7. Architecture of proposed federated learning, in which only the genera-
tors’ models are sent to the global server for aggregation. The discriminators
then update their regularizer terms based on the global generator model’s C. Physical Layer Security
weights [71].
Physical layer security (PLS) is another important research
area in wireless communication systems. In general, PLS
refers to techniques that enhance the security of wireless
results indicate that the proposed GAN approach can help to communications at the physical layer by leveraging the inher-
improve the estimation accuracy of traditional training-based ent randomness of wireless communication channels. Major
channel estimation approaches, especially at low SNRs. problems in PLS include anti-jamming, anti-eavesdropping,
signal authentication, and device identification. With recent
In addition, the authors in [14] reveal that conventional DL advancements in DNNs, DL has been widely adopted to
methods perform poorly in estimating fast time-varying and improve the security at the physical layer of wireless commu-
non-stationary channels. As such, they propose a GAN-based nication systems. However, conventional DL-based approaches
channel estimation approach that can accurately estimate wire- face various challenges due to the dynamics and uncertainty
less channels in high-speed railway systems. Specifically, the of diverse physical layer attacks. Specifically, the DL model
discriminator is responsible for learning and extracting time- is usually trained on the dataset of a specific environment,
varying features of railway communication channels while and thus it cannot work well in new conditions. This is an
the generator aims to determine the training data’s implicit essential problem as attackers can always change their attack
mapping function. Through simulations, the authors show that strategies to maximize the disruption. In addition, conven-
the probability density curve of the estimated channel data is tional DL-based approaches require large datasets to obtain
highly similar to that of the ground truth channel responses, good detection performance. However, it is difficult to collect
indicating the effectiveness of GAN in estimating wireless sufficient labeled data from physical layer attacks due to their
channels. randomness and dynamics [80]–[82]. More importantly, con-
The aforementioned GAN-based solutions and many others ventional DL models are vulnerable to adversarial attacks [83],
in the literature are designed in a centralized learning manner [84]. Minor perturbations in the input data can fool DNNs
which may not be feasible in large-scale scenarios. To tackle and consequently make conventional DL-based approaches
this practical challenge, the authors in [71] propose a federated less effective in dealing with physical layer attacks. Finally,
GAN solution for channel estimation in a distributed manner, conventional DL models perform poorly with time-varying
as illustrated in Fig. 7. In particular, each client uses the channels in low SNR regions and when prior information about
estimated CSI obtained by the least square estimator as the attackers is not readily available [85], [86].
input data of GAN to learn the distribution of channels. After These challenges in physical layer security can be efficiently
that, the generator parameters are transmitted to the server for addressed by GAI, as summarized in Table V. As discussed,
aggregation. To improve federated learning performance, each GAI can be used for anomaly detection by generating data
client’s discriminator will be dynamically adjusted by using that is defined to be normal and then flagging input data that
regularizers based on the global generator’s weights. Extensive deviates significantly from these definitions. In addition, GAI
simulations suggest that the proposed federated GAN approach has been demonstrated to be effective in uncertainty estimation
is superior to conventional estimators as well as state-of-the- as well as domain adaptation [88] which are critical capabili-
art DL-based channel estimation. For example, at SNR=5 dB, ties to deal with physical layer security threats. For example,
GAN can achieve a normalized mean-squared error (NMSE) a GAN-based solution is proposed in [89] for abnormality
of 10−2 while the ChannelNet proposed in [79] can only obtain detection at the physical layer in cognitive radio networks. In
an NMSE of around 0.5. To further reduce the communication particular, the proposed GAN approach is used to generalize
overhead, model compression and multiple tasks design can the state vectors extracted from spectrum representation data
be considered to make the proposed federated GAN approach to learn the dynamic behavior of wideband signals. Based
more effective. on these state vectors, abnormal signals can be distinguished
 11

TABLE IV
S UMMARY OF GAI A PPROACHES FOR C HANNEL E QUALIZATION , M ODELING , AND E STIMATION

No. Problem Drawbacks of TAI Proposed Approach

Use VAE to efficiently learn from unknown input impulse
[64] Poor performance in blind channel equalization
sequences.
Channel High complexity and not effective without using pilot Use VAE to design a blind channel equalization that can
[65]
Equalization symbols model the unknown nonlinearity
Low performance under the scattering and absorption GAN is used to equalize the one-bit quantization’s distortion
[63]
effects of underwater communications as well as the negative effects of underwater channels.
[66] Not effective in learning signal distributions Use diffusion models to eliminate channel noise
Use GAN to learn the probability distribution functions of
[72] Only effective with simple channel models wireless channels, resulting in better channel response ap-
proximation
Use VAE to learn the distribution of channel impulse re-
Channel Suffer from the curse of dimensionality and can only
[73] sponses and generate synthetic channel response samples with
Modeling be evaluated with a simple AWGN channel model
similar properties
Limited by the lack of channel samples and environ- Propose a distributed GAN architecture to allow UAVs to
[68]
mental measurements collaboratively approximate mmWave channel distributions
Lack of training data and not effective in learning
Use GAN to generate random multi-cluster profiles that
[67] complex statistical relationships across different fre-
include all information of different frequencies
quencies
Propose a diffusion model based channel sampling approach
The collection of wireless channel data is costly and
[28] to generate synthetic channel responses based on limited
time-consuming
ground truth data
Focus on estimating mmWave channel models for Use GAN for mmWave channel modeling by effectively
[74]
specific environments with limited applications extracting useful CSI features in the spatial-temporal domain
High complexity and training overhead needed to Use GAN to learn functions of channel covariance matrices
[75]
obtain channel knowledge and environment factors
Cannot estimate channels in high-speed moving sce- Use GAN to learn and extract channel time-varying features
[14]
narios and then restore channel information
Channel Es- Need to know or model the channel distribution Use GAN to generate synthetic channel samples that have a
[70]
timation similar distribution with a true but unknown channel
Use GAN to learn from receive signals and exploit Wasser-
Poor performance and require a large number of
[69] stein distance to improve estimation accuracy without trans-
simulated samples to train DNNs
mitting long pilot sequences.
Each client uses the estimated CSI obtained by the least
[71] Low privacy due to large CSI dataset exchanging square estimator as the input data of GAN to approximate
the channel’s distribution
Do not focus on the characteristic of mmWave fre- Use GAN to learn the distribution of mmWave channels from
[76]
quencies or A2G wireless links multiple distributed datasets
Require large datasets, long training time, and less Develop a conditional GAN approach to generate channel
[77]
effective under environmental variations covariance matrices for training
Do not adequately account for the dynamics and Use GAN to generate a more realistic channel image for more
[78]
uncertainty of channels in large MIMO systems effective training under channel variations.

from legitimate signals. Similarly, the authors in [90] and [82] straightforwardly applied to RF systems due to the dynamics
aim to prevent jamming attacks as well as interference from and uncertainty of RF signals. More importantly, these ML
secondary users in cognitive radio networks. They first high- techniques may perform poorly in the presence of intelligent
light that conventional DL-based anti-jamming approaches adversaries that can spoof transmitters and inject interference
give poor performance when spectrum data is not sufficient. into the target channels, making it more challenging to capture
Unfortunately, collecting and labeling spectrum data in the the unique properties of the transmitters. For that, the authors
presence of jamming attacks are time-consuming and costly. propose a GAN-based approach to efficiently authenticate
To address this practical issue, the authors propose to use RF transmitters as GAN is well known for its capability in
GAN to generate synthetic spectrum data that can help a DRL dealing with adversarial situations, as shown in Fig. 8. In
algorithm to effectively learn and obtain the optimal dynamic particular, the GAN’s generator will use RF signals generated
spectrum anti-jamming access policy. Extensive simulations from adversaries as its input to generate synthetic data 𝑔(𝑡).
then demonstrate that the proposed GAN can help to avoid On the other hand, the discriminator learns from signals of
complex jamming attacks and outperform conventional DRL- both “trusted” transmitters and the generator to identify the
based approaches with incomplete spectrum information. The differences between real and fake RF signals. In this way,
lack of training data problem of conventional physical layer the proposed solution can achieve a detection accuracy of
security approaches is also discussed and addressed by using 99% which is much higher than those of CNN and DNN
GAN in [81], [91], [85], and [92]. approaches, i.e., 81.6% and 96.6%, respectively. Similarly,
Differently, the work in [87] aims to authenticate radio the authors in [93] also point out that the time-varying
frequency (RF) transmitters by using GAN. The authors characteristics of wireless channels introduce more difficulties
first highlight that conventional ML techniques cannot be to conventional DL-based approaches in detecting abnormal
 12

RF signals. In contrast, GAN, with its capabilities of anomaly 8188

detection and uncertainty estimation, can deal with this issue Preprocess

effectively. 𝑿 Autoencoder 𝑿ሶ
8188 8188
In [94], the authors study a more challenging scenario Fully connected layer Fully connected layer
Relu Relu
where spoofing signals are identical or similar to real signals.
1000 1000
Specifically, spoofing attacks in the global navigation satellite 𝝓(𝒉|𝒙) ሶ
𝝍(𝒙|ሶ 𝒉)
Fully connected layer Fully connected layer
system are considered in which spoofing signals are similar Relu Relu
256 256
to legitimate satellite signals in terms of pseudo-code phase
Fully connected layer Fully connected layer
and carrier Doppler values but have much stronger power Relu Relu

to lure the receiver to track them instead of real signals. ℎ 𝑯

VAE
𝑯ሶ ℎሶ
Consequently, existing detection methods, including DL-based
solutions, cannot effectively distinguish between legitimate ሶ 𝟏)
𝝓𝟐 (𝒛𝟐 |𝒉) 𝝍𝟏 (𝒉|𝒛
satellite signals and spoofing signals. The authors then design ሶ 𝟐)
𝝍𝟐 (𝒉|𝒛
𝝓𝟏 (𝒛𝟏 |𝒉)
a GAN network that is trained on a large dataset of authentic
satellite signals to accurately learn their distributions. Simula-
tion results indicate that by using GAN, the authors can obtain 𝑍1 𝑍2
better detection performance than using the conventional CNN
network. Differently, the authors in [95] leverage GAN as 𝒁𝟏 of VAE: with a Simple Gaussian Prior for Further Dimension Reduction
an effective tool for physical layer key generation. It is 𝒁𝟐 of VAE: with a Revised Double-Peak Gaussian Prior for Classification
well known that wireless communications are susceptible to
radio attacks such as eavesdropping and tampering due to
their broadcast nature. Meanwhile, conventional cryptography
Fig. 9. Proposed GAN architecture for authenticating RF transmitters in which
techniques in the upper layers may not be feasible for wireless the VAE is used as a high-level classifier with two hidden units 𝑍1 and 𝑍2 . 𝑍1
devices due to their computational complexity, especially in is constructed based on encoder 𝜙1 and decoder 𝜓1 with a Simple Gaussian
IoT networks. By leveraging the inherent uncertainties of the Prior for dimension reduction and channel impulse response reproduction. 𝑍2
is constructed based on encoder 𝜙2 and decoder 𝜓2 with a revised double-
physical communication channels, physical layer key genera- peak Gaussian Prior for authentication [87].
tion has been widely adopted. In general, DL is superior to
conventional approaches in extracting symmetric keys from
reciprocal channel responses. However, the authors in [95] peak Gaussian Prior distributions for further security and
reveal that conventional DNNs are unpredictable for physical robustness enhancement. In this way, the proposed solution
layer key generation. In addition, it is challenging to apply the can efficiently capture important features of high-dimensional
extracted high-dimensional features to generate the physical channel impulse responses for better authentication perfor-
layer key. Therefore, they propose a new key generation mance. The authors then show that the proposed solution can
method based on GAN that can efficiently extract features of improve the authentication performance by 17.18% compared
legitimate nodes. to a conventional ML approach in [105].
Besides GAN, VAEs can also be adopted for physical layer Due to the ability of generating synthetic data that is similar
security. For instance, the authors in [86] propose a hierar- to real data, GAI can also be used by adversaries to perform
chical VAE-based approach for physical layer authentication different types of physical layer attacks [83], [84], [98], [103].
in complex scenarios such as industrial IoT systems. The For example, the authors in [103] use a GAN network to
authors state that ML has been widely adopted for physi- generate synthetic wireless signals that cannot be distinguished
cal layer authentication to analyze and extract complicated from legitimate signals by conventional approaches. Experi-
properties of wireless channels for authenticating wireless ments then show that by using GAN the authors can improve
devices. Nevertheless, these methods usually require infor- the attack performance. Similarly, the authors in [84] recruit
mation about attackers available in advance to obtain good GAN to perform adversarial attacks. In particular, GAN is
detection performance which is not the case in practice. As used to generate crafted imperceptible perturbations to cause
such, the authors develop a new hierarchical VAE architecture wrong classifications of a DL-based modulation recognition
based on autoencoder and VAEs for efficient physical layer approach. Through extensive simulations, the authors then
authentication with no prior channel information of attackers, indicate that the proposed GAN-based adversarial attack can
as illustrated in Fig. 9. In particular, the VAE is used as a reduce the accuracy of the DL-based modulation classifier
classifier, consisting of two hidden units 𝑍1 and 𝑍2 . 𝑍1 is more than jamming and other adversarial attacks. For instance,
constructed based on encoder 𝜙1 and decoder 𝜓1 with a Simple at 0 dB perturbation-to-noise ratio, the proposed techniques
Gaussian Prior for dimension reduction and channel impulse can reduce the detection performance by 37% at SNR=10
response reproduction. On the other hand, 𝑍2 is constructed dB, by 56% at SNR=0 dB, and by 7% at SNR=-10 dB.
based on encoder 𝜙2 and decoder 𝜓2 with a revised double- In addition, the authors in [98] demonstrate that GAN can
peak Gaussian Prior for authentication. The conventional au- help a jammer to effectively jam a target wireless channel by
toencoder is used to further reduce the dimension of input data. generating more training data to help the jammer better learn
Finally, a new loss function is designed for the VAE module the defense policy of the legitimate receiver. To deal with these
considering both the Simple Gaussian Prior and the double- GAN-based attacks, the authors in [83] propose to use another
 13

TABLE V
S UMMARY OF GAI A PPROACHES FOR P HYSICAL L AYER S ECURITY

No. Problem Drawbacks of TAI Proposed Approach

Use GAN to extract hidden information in original signals to
[96] Inaccurate representations for the individual emitters
improve identification performance
Cannot generate high-quality synthetic spoofing sig- Use GAN to generate spoofing signals that are similar to
[97]
nals legitimate signals
Use GAN to augment the training dataset of the classifier with
[83] TAI-based approach can be cracked by using GAN
adversarial samples generated from another GAN network
Use GAN to help jammers to generate training data to improve
[98] Cannot generate high-quality synthetic samples
attack performance
Use a generative model to generate fake signals that are trained
Not effective when detecting rogue RF transmitters
[87] together with real signals by a discriminative model to better
and classifying trusted ones
identify trusted ones
Use GAN to provide accurate estimation of complex proba-
[99] Not effective in learning data distributions
bility distributions from smallish sized datasets
Physical
Use GAN to adaptively generate refined deceptive jamming
Layer
Cannot generate deceptive jamming templates under templates based on various factors such as azimuth angles,
Security
[100] constraints angles, and target types. This can help to protect a specific
area from observation and detection by adversarial radars
Only effective with a specific type of low dimen- Use GAN to effectively learn from high dimensional data of
[89]
sional data spectrum representation samples
Use GAN to generate incomplete spectrum data in multiple
[80] Lack of spectrum data
jamming patterns
Not effective in learning distributions of received Use GAN to learn the distribution of received channel data to
[101] signals authenticate a transmitting device
Use GAN to generate more samples and use VAEs to learn
[81] Lack of training data
the latent space of continuous signal samples.
Use GAN to learn the data of trusted transmitters to extract
[91] Lack of training data about malicious transmitters
RF fingerprint
[82] Lack of spectrum data Use GAN to generate missing spectrum data
Not effective in dealing with the dynamics of wire- Use GAN and LSTM to learn and predict CSI elements’
[102] less channels magnitude
Time-varying characteristics of wireless channels Incorporating an encoder network into the original GAN to
[93]
make the prediction unreliable reconstruct the spectrogram
Most detection methods cannot effectively detect
Design a GAN network that is trained on a large dataset of
[94] spoofing jamming if spoofing signals are similar to
authentic satellite signals to accurately learn their distribution
authentic signals
Use GAN to construct synthetic RF signals that are similar to
Cannot effectively use to perform attacks
[103] legitimate signals
Difficult to apply to the key generation in the phys- Propose a key generation method based on GAN to extract
[95]
ical layer features efficiently between legitimate nodes
Use GAN to identify unrecognized patterns on the model
Not effective in anomaly detection as GAI
[104] outputs and associated sequenced metadata
Use GAN to generate synthetic spectrum data that can help
Poor performance when spectrum data is not suffi-
[90] DRL to effectively learn and obtain the optimal dynamic
cient
spectrum anti-jamming access policy
[92] Lack of labeled data Use GAN to learn the distribution of collected signals
Use VAEs to extract valuable features of high-dimensional
[86] Need attackers’ information for training
channel impulse responses for authentication

GAN network to augment the training dataset of the classifier gain in a desired direction, creating a favorable wireless signal
with adversarial samples generated from adversaries’ GAN propagation environment. However, accurate CSI information
networks. Simulation results then show that by augmenting the and underlying channel models must be obtained to leverage
training data with GAN the authors can effectively improve the these advantages of IRS [62]. Unfortunately, it is challenging
classification accuracy under GAN-based adversarial attacks. to acquire BS-IRS and IRS-UE channels separately without
the help of RF chains. In addition, the cascaded channel of
D. Intelligent Reflecting Surface (IRS) BS-IRS and IRS-UE links is very high-dimensional due to the
Recently, IRS has been emerging as a promising technol- high number of reflecting elements. To overcome these issues,
ogy to significantly improve energy efficiency and spectrum various DL-based channel estimation and channel modeling
utilization with low-cost and low-power hardware [106], [107]. approaches have been proposed in the literature. Nevertheless,
In particular, a typical IRS consists of a large number of these approaches cannot accurately estimate IRS channels
reconfigurable metasurface elements that can be adjusted to since they use a general loss function that is not well designed
control the amplitude responses and phase shifts of the incom- for IRS, leading to poor estimation performance [62]. In
ing incident electromagnetic waves. By coordinating the phase addition, conventional DL-based approaches can only learn
shifts and amplitudes across the array of elements, an IRS can a limited number of channel parameters and one-dimensional
reconfigure wireless channels and obtain high beamforming channel impulse responses.
 14

TABLE VI
S UMMARY OF GAI A PPROACHES FOR I NTELLIGENT R EFLECTING S URFACE (IRS)

No. Problem Drawbacks of TAI Proposed Approach

Channel Mod- Require in-depth domain knowledge and

[106], Use GAN to generate high-dimensional channel samples
eling lack of training data
[107]
Lack of observational dimensions and mod-
Use GAN to remove noise from the estimated channel matrix
[108] Channel Es- eling capabilities
timation Use GAN to learn the channel distribution with LS estimation
Not effective with high channel dimensions
[109] as conditional input
Use a general loss function that is difficult
Use GAN to approximate cascaded channels by taking received
[62] to make the estimated IRS channels more
signals as conditional information
accurate
IRS Use GAN to support DRL by learning the action-value that is
Not effective in dealing with the dynamics
Deployment near to target-action values, resulting in a more stable learning
[110] of 6G networks
Design process

To tackle the above issues of conventional DL-based ap- beamforming matrix of an IRS-assisted 6G network. The
proaches, GAI has been adopted in various studies, as sum- authors first develop a deep reinforcement learning (DRL)
marized in Table VI. For example, the authors in [106] framework to interact with the system and gradually learn an
and [107] develop a model-driven framework based on GAN optimal joint policy. However, due to the reward function’s
for channel modeling in IRS-aided wireless communication randomness, the proposed DRL framework cannot learn all
systems. To make GAN learn the channel distribution more the dynamics and uncertainty of the considered IRS system
effectively, the authors incorporate the structure of the cas- effectively. To overcome this issue, the authors propose to use
caded BS-IRS and IRS-UE channels into the generator of the GAN to identify the action-value that is close to target-action
proposed GAN architecture. More specifically, the generative values, resulting in a more stable learning process. Specifically,
model now has three nodes: (i) BS-IRS node to learn BS- the generator aims to generate actions (e.g., adjusting phase
IRS channel distribution, (ii) IRS-UE node to learn IRS-UE shift, coordinates, and beamform) for the DRL agent that
channel distribution, and (iii) cascading node to combine the are mapped to the original dataset. Then, these generated
outputs. The discriminative model is then used to distinguish experiences and the original dataset are stored in a relay
between the generated channel samples and the real BS-IRS- buffer. After that, the discriminator randomly takes a number
UE channel samples. Moreover, the authors adopt Wasserstein of samples in the relay buffer as its input to learn how to
distance [111] to design a new loss function for the proposed distinguish the generated experiences from the generator and
GAN model for more stable training. In this way, the proposed real samples from the original datasets. Simulation results
solution can achieve much better performance than existing then demonstrate that the proposed GAN architecture can
solutions using CNNs and fully-connected neural networks help to improve the accuracy of DRL by 45%. To allow
(FNNs), as demonstrated in the simulation results. Differently, multiple IRSs to work collaboratively, the proposed approach
the authors in [62] use a conditional GAN architecture for can be extended by considering a multiagent GAN-based DRL
channel estimation in IRS-aid wireless communications. In framework.
particular, the proposed GAN takes the received signals as
its conditional information to generate channel responses with E. Beamforming
certain characteristics. Then, the discriminator and the genera-
tor compete with each other to obtain an adaptive loss function, In wireless communications, beamforming is a key tech-
making the generated channels similar to the original channels. nology to improve signal quality and transmission coverage.
With its capability in learning data distribution effectively, the To do that, the transmitter, e.g., BSs, employs an array
proposed GAN architecture can achieve much better chan- of individual antenna elements in which each element can
nel estimation performance compared to conventional DL- adjust the phase of signals to make signals at specific angles
based methods as demonstrated in extensive simulations. For experience destructive interference while others encounter
instance, at 5 dB SNR, the NMSE of GAN is around ten constructive interference. However, it is challenging to obtain
times less than that of the ChannelNet architecture proposed optimal beamforming policies due to the high computational
in [112]. Applications of GAN for channel estimation in complexity and excessive feedback overhead, especially in
IRS-aided wireless communications are also studied in [108] systems with large antenna arrays like mmWave and massive
and [109] where GAN-based convolutional blind denoising MIMO communication systems [115], [117]. DL can be used
and conditional GAN are adopted to obtain accurate CSI for to tackle this problem but it requires a large amount of
IRS-aided systems, respectively. training data and cannot efficiently deal with the dynamics and
uncertainty of wireless communications. Several researchers
GAN can also be used for the deployment design and have been adopting GAI as an alternative approach and
phase shift optimization of IRS. For instance, the authors achieving promising results, as summarized in Table VII. For
in [110] aim to jointly optimize the placement and reflecting example, the authors in [117] propose to use GAN to re-
 15

Generator 1 Generator 2 Generator 3 SE Module Discriminator

Encoder Encoder 𝑺𝑬𝒐𝒑𝒕
Block Block 𝑺𝑬′
෡
𝑯
Encoder Decoder Encoder Decoder
Block Block Block Block Conv
Encoder
Block

…
Skip Conca- Skip Conca-
connection connection

…
tenate tenate Data Inverse
Processing ReLU
Encoder Encoder

…
Conv

…
ReLU Block Block
Encoder
Normalization Block Encoder
ReLU ReLU SE layer
Maxpooling Decoder Decoder Block
Decoder Block Block
Block

…
Conv Conca- Conv Conca- Data

Conv
ReLU
… Down-
tenate
Down-
tenate Processing
Encoder
Block
Decoder sampling sampling
Normalization Block
Upsampling Up- Up-
Fully sampling sampling
Connected
Fake Real
𝑨 𝑨 𝑭
𝑯′ 𝑯′ “0” “1”

(a) (b) (c) (d) (e)

Fig. 10. Multi-GAN architecture for beamforming with (a) Generator G1, (b) Generator G2, (c) Generator G3, (d) the spectrum efficiency module, and (e)
the discriminator [113].

TABLE VII
S UMMARY OF GAI A PPROACHES FOR B EAMFORMING

No. Problem Drawbacks of TAI Proposed Approach

Lack of training data Use GAN to generate additional data for beam prediction
[114]
Beamforming Not effective in learning data dis-
[115], Use VAE to approximate the probabilistic model of beam dynamics
tribution
[116]
High feedback overhead and high Use GAN to reconstruct a low-dimensional channel fed back from the
[117] complexity receiver to perform hybrid beamforming at the transmitter
Use a GAN architecture with three generators to recover (i) rank-deficient
Suffer from the rank-deficient
channels, obtain (ii) analog beamforming and (iii) digital beamforming
[113] problem
matrices.

construct low-dimensional channel feedback from the receiver G3 is used for hybrid beamforming, as illustrated in Fig. 10.
to perform hybrid beamforming at the transmitter, resulting Generator G2 takes the estimated rank-deficient channel H′
in low communication overhead. Specifically, the generator as its input to generate analog beamforming A which is the
of the proposed GAN architecture is first pretrained offline input of generator G3. Generator G3 then estimates hybrid
with channel samples generated by a geometric channel model beamforming F. After that, H′ and F are fed into the spectrum
to learn the channel structure and correlations. In the online efficiency module to calculate the average spectrum efficiency
phase, the receiver tries to compress the channel matrix to a SE′ . The generator then learns from SE′ and the real spectrum
low-dimensional vector and feeds it back to the proposed GAN efficiency to improve the training processes of generators G1
architecture at the transmitter to recover the channel matrix and G3. Extensive simulations then demonstrate that the pro-
that will be used for beamforming design. The proposed GAN posed multi-GAN architecture can improve the beamforming
solution can help to reduce 2,048 complex channel elements performance by 47.49% compared to a conventional CNN-
to just 15 real values while maintaining good communication based method.
performance as demonstrated in simulations. One possible ex-
tension of this work is to extend the considered model/method F. Joint Source Channel Coding (JSCC)
to extremely-large massive MIMO or holographic MIMO. Coding plays a crucial role in wireless communications to
Differently, the authors in [113] consider the beamforming mitigate the negative effects of channel noise, interference, and
design in massive MIMO systems with large antenna arrays fading. Traditionally, the transmitter performs source coding
where only rank-deficient CSI can be obtained. This rank- for compression and channel coding for error correction,
deficient problem has not been fully solved in the literature separately, making it difficult to optimize the spectrum usage.
using conventional techniques. For that, the authors propose By combining the functions of source coding and channel
a multi-GAN architecture for hybrid beamforming design coding into a single process, JSCC can leverage the statistical
under rank-deficient channels. Specifically, the authors em- characteristics of the source and the channel to design a
ploy three generators in the proposed GAN architecture in more efficient coding method. JSCC also helps to reduce the
which generator G1 is used to recover rank-deficient channels, overall complexity of wireless communication systems since
generator G2 is used for analog beamforming, and generator it requires one encoder and one decoder only. However, the
 16

TABLE VIII
S UMMARY OF GAI A PPROACHES FOR J OINT S OURCE C HANNEL C ODING (JSCC)

No. Problem Drawbacks of TAI Proposed Approach

Not effective in dealing with the complexity VAE’s encoder converts the source data into a low-dimensional latent
and discontinuity of source data distribu- space while the VAE’s decoder tries to recover it to original data for
[118]
tions JSCC
JSCC Not effective when source dimension in- Use VAE to learn the source distribution by considering the noise
[119] creases channel as a sample of latent variables
Use two GAN-based networks to recover the distorted reconstructions
Have significant losses of perceptual quality
of a DL-based JSCC and to produce the latent and noise inputs for the
[120] for the edge cases
StyleGAN-2, respectively
Propose a VAE-based JSCC system with added distribution restrictions
Not stable for multivariate Gaussian source
[22] on the loss function to avoid falling into the local minimum in specific
over Gaussian multiple access channels
regions
Use diffusion models as a generative refinement component to enhance
Suffer from the cliff effect
[121] the reconstruction’s perceptual quality
Use a GAN compression method based on intermediate feature distilla-
High complexity
[122] tion

complexity and discontinuity of source data distributions in- GAN architecture. In contrast, in GenerativeJSCC, GAN is
troduce challenges to the design of JSCC. For that, the authors used as the decoder to produce latent and noise inputs for the
in [118] propose a novel JSCC approach based on VAEs StyleGAN-2 [123] generator. By jointly training the encoder
over additive noise analog channels. Specifically, the proposed and GAN decoder, GenerativeJSCC can outperform DL-based
VAE’s encoder is used to convert source data into a low- JSCC methods in terms of perceptual quality and distortion,
dimensional latent space while the VAE’s decoder recovers it as demonstrated by extensive simulations. GAI has also been
to original data for JSCC. More importantly, the authors study adopted in other studies as summarized in Table VIII.
that when the channel dimension is smaller than the source
dimension, the encoding of two neighboring source samples G. CSI Feedback
needs to be near each other for good encoding performance. With its powerful capabilities in learning data distribution
Therefore, multiple encoders are employed, and one of them and generating synthetic data, GAI has also been applied to
will be selected for sample encoding on a specific side of recover compressed CSI feedback, as summarized in Table IX.
the discontinuity. Experiments then demonstrate that using For example, the authors in [15] propose to use GAN for
the proposed VAE-based JSCC method can help to increase reconstructing CSI feedback in massive MIMO communica-
the average peak SNR (PSNR) by nearly 3 dB compared to tions systems. In particular, massive MIMO can provide high
conventional CNN-based approaches. cell throughput and reduce multiuser interference but largely
Recently, JSCC has been emerging as an effective tech- relies on exploiting the CSI feedback from UEs. To reduce the
nology for semantic communications. However, in [119], the signaling overhead of the system, the CSI feedback is usually
authors highlight that when the source dimension increases, compressed at UEs before transmitting to BSs. During the
e.g., large-scale images, the performance of DL-based JSCC compressing process, important CSI information may be re-
methods degrades significantly. Moreover, when the chan- moved unintentionally, resulting in low precoding performance
nel bandwidth ratio increases, these methods provide poor at BSs. To tackle this problem, the authors develop a GAN-
coding gain as they cannot learn the source distribution to based CSI recovery framework that can effectively generate a
determine patch-wise variable-length transmissions. To tackle CSI matrix based on its compressed version. Specifically, the
these issues, the authors design a JSCC architecture based on compressed CSI feedback will be first fed to the generator to
VAEs in which the noise channel is viewed as a sample of estimate the CSI vector. This estimated CSI vector is then fed
latent variables. In this way, the proposed architecture can to the discriminator together with the original CSI vector to
effectively learn the source distribution to provide a more determine if the reconstructed CSI is good or bad. A new loss
effective coding mechanism. Experiments then show that the function combining the adversarial loss of the discriminator
proposed solution can achieve up to 28.91% bandwidth saving and the mean square error loss between the reconstructed and
or a PSNR gain of 2.64 dB on the CIFAR10 dataset while original CSI is introduced to further enhance the recovery
the conventional deep JSCC increases the bandwidth cost by performance of the proposed GAN-based approach. Extensive
up to 54.31%. Similarly, the authors in [120] also consider simulations reveal that by using GAN, the proposed framework
JSCC for semantic image transmissions. The authors study is superior to traditional DL-based approaches. For instance,
that DL-based JSCC possesses significant perceptual quality with a compression ratio of 41 , the GAN-based framework can
losses in edge scenarios. Therefore, they propose two novel achieve an outdoor NMSE of -15.88 dB while CsiNet [130]
JSCC schemes based on GAN, namely InverseJSCC and and CsiNet+ [131] can only obtain -8.75 dB and -12.4 dB,
GenerativeJSCC. InverseJSCC aims to recover the distorted respectively.
reconstructions of a DL-based JSCC model via solving an Differently, the authors in [127] propose to use VAEs for
inverse optimization problem using a pre-trained style-based CSI compression at UEs under noisy channel conditions. The
 17

TABLE IX
S UMMARY OF GAI A PPROACHES FOR CSI FEEDBACK , RADIO MAP ESTIMATION , AND CHANNEL DELAY ESTIMATION

No. Problem Drawbacks of TAI Proposed GAI Approach

Cannot achieve performance as
[15] CSI feedback Use GAN to recover original CSI from its compressed version.
good as GAN
Cannot achieve performance as Use GAN to enhance wireless channel data, resulting in better CSI
CSI feedback
[124] good as GAN compression processes
Cell outage de-
Data imbalance issue Use GAN to generate more synthetic samples for minority classes
[125] tection
The generator aims to generate image masks while the discriminator
Radio map esti-
Lack of training data learns to distinguish the masks of the original dataset and those generated
[126] mation
by the generator
Less effective under noisy feed-
CSI feedback Use VAE to compress CSI under noisy channel conditions
[127] back channels
Poor performance due to nonuni-
Radio map esti- Use conditional GAN architecture to efficiently estimate radio maps
formly positioned measurements
[128] mation based on observations from the environment
and access constraints
Channel delay Use GAN to generate synthetic cross-correlation data and smooth it with
Lack of training data
[129] estimation a Savitzky-Golay filter

authors highlight that conventional DL-based CSI compression effective for radio map estimation in modern IoT and cellular
approaches like CsiNet in [130] are vulnerable to noisy feed- systems due to nonuniformly positioned measurements and
back channels which are common in practice. In contrast, the access constraints. For that, the authors in [126] and [128]
proposed VAE-based compressor can approximate distribution propose to use the conditional GAN architecture to efficiently
parameters for each dimension instead of estimating a point estimate radio maps based on observations from the envi-
for each dimension in the latent space (i.e., deterministic ronment. Particularly, the generator aims to generate image
latent space) as in classic DL-based solutions. As a result, masks while the discriminator learns to distinguish the masks
the compressed CSI is robust against noise in the feedback of the original dataset and those generated by the generator.
channel. To make the proposed VAE network more suitable Simulation results then demonstrate the effectiveness of GAN
for the noise conditions of the feedback channel, the authors in estimating radio maps in various outdoor environments.
modify the VAE loss by using a weighted combination of
In addition, GAN is a promising approach for channel delay
reconstruction error and KL divergence between the encoder’s
estimation as studied in [129]. The authors aim to accurately
distribution and the true distribution. The authors then test
estimate the first-arrival-path delay in wireless multi-path
the proposed solution with an additive white Gaussian noise
channels which plays an essential role in positioning and
(AWGN) feedback channel and indicate that the proposed
localization services. To do that, they first propose a CNN
VAE-based compression technique can outperform other DL-
network to learn the mapping between the cross-correlation
based techniques (e.g., CsiNet [130]) and compressive-sensing
sequence and the delay offset. However, this CNN network
based models both under noise-free and noisy channel condi-
suffers from the lack of training data. As such, the authors use
tions. Similarly, the authors in [124] adopt GAN for wire-
GAN to generate synthetic cross-correlation data and smooth it
less channel data augmentation before feeding CSI data into
with a Savitzky-Golay filter. The authors then perform various
CsiNet. Specifically, a GAN-based network is developed to
simulations to show that the proposed channel delay estimator
enrich data features of the original wireless channel data and
can outperform existing approaches. In addition, the proposed
also to generate new similar data by learning the distribution
GAN architecture can help to maintain a good estimation
of the original channel data. These GAN-generated data will
accuracy for the CNN network even with limited real cross-
be fed into CsiNet for compression before feeding back to
correlation data. Differently, the authors in [125] consider
BSs. Simulation results reveal that using GAN can achieve
the cell outage detection problem in self-organizing cellular
a 3dB performance improvement compared to existing data
networks. Several classifiers based on DL have been proposed
augmentation approaches.
in the literature for this problem. However, as highlighted by
the authors, these traditional approaches suffer from the data
H. Radio Map and Channel Delay Estimation imbalance problem in which the number of training samples in
Due to its capability of variational learning and sampling to one class is significantly larger than the number of samples in
explore the data distribution in a more versatile manner, GAI other classes. This leads to a biased classifier and degrades the
can also be used for radio map estimation [126], [128], as service quality of the system. Therefore, the authors propose
summarized in Table IX. In particular, a radio map spatially a novel GAN network with the aid of the Adaboost algorithm
shows RF signal strength distribution and network coverage to preprocess the training data to change imbalanced data to
information which are essential characteristics for resource balanced ones by generating more synthetic data for minority
management and network planning in wireless communication classes. Experimental results show that the proposed solution
systems. Unfortunately, conventional DL-based approaches can effectively address the data imbalance problem and obtain
such as RadioUNet [132] and autoencoder [133] may not be better performance compared to state-of-the-art approaches.
 18

I. Summary and Lessons Learned B. Model-driven GAI

With its capabilities in generating synthetic data under As can be observed in Section III, existing GAI-based
constraints, anomaly detection, uncertainty estimation, and models mostly focus on data-driven approaches that rely
variational learning and sampling, GAI has been widely on the availability of training data. However, in practice,
adopted in the literature to address various problems such as collecting a sufficient amount of training data may be costly,
physical layer security, channel estimation, signal classifica- time-consuming, and even impossible. To tackle this issue,
tion, beamforming, JSCC, and IRS, as discussed in this section model-driven approaches [135], [136] can be adopted. In
and summarized in Tables III, IV, V, VI, VII, VIII, and IX. particular, model-driven approaches can incorporate the prior
The lessons learned are as follows: knowledge of target domains, e.g., carrier frequencies, physical
• GAI has been mostly adopted for common issues in constraints, and noise distributions, into the training process
physical layer communications such as physical layer se- to further improve the performance of GAI-based solutions.
curity, channel estimation, and signal detection. However, For example, with prior knowledge of bandwidth and carrier
thanks to its capabilities, GAI can also be applied to other frequency, GAI-based solutions can be trained to generate
problems in the physical layer, opening new research more realistic channel samples.
directions. For example, the authors in [15] and [127]
demonstrate that GAI is a great tool for efficient CSI C. Resource-Efficient Learning
compression. In addition, GAI can significantly improve
the performance of JSCC and beamforming, as discussed The training and inference of GAI require computation, stor-
in Sections III-E and III-F. age, and communication resources, putting burdens on existing
• The majority of GAI approaches in the literature are based
communication systems, especially for resource-constrained
on GAN due to its effectiveness in variational learning devices such as IoT devices, mobile phones, and UAVs.
and sampling. Other approaches like VAEs, NFs, and dif- As such, novel GAI architectures need to be developed
fusion models have been gaining more attention recently to minimize resource consumption while maintaining good
and are expected to be common in many applications in learning performance. Distributed and federated learning can
physical layer communications soon. be integrated into GAI to offload computational tasks to
• Besides its great benefits, GAI can be used by adversaries
edge devices as well as reduce communication overhead by
to perform attacks at the physical layer as GAI can transmitting model updates instead of raw data. For example,
effectively generate fake data that is similar to real data GAI models can be trained at edge devices with local data and
from legitimate activities. However, research on counter- then aggregated at a centralized server to obtain a global GAI
measures against GAI-based physical layer attacks is still model. In addition, GAI can be used to recover compressed
limited, and more efforts from both academia and industry local model updates to reduce communication overhead while
are required. still maintaining good training performance. Incentivization
mechanisms such as dynamic spectrum access should also be
considered to utilize communication resources, especially in
IV. O PEN I SSUES AND F UTURE R ESEARCH D IRECTIONS cognitive radio networks as studied in a few papers reviewed
Although having great capabilities in complex data feature in Section III.
extraction, transformation, and enhancement, GAI is still in
its early stage of development. Thus, open issues and research
D. Real-time Adaptation
directions of GAI in physical layer communications will be
discussed in this section. Although GAI has the capability of domain adaptation that
can leverage knowledge from a source domain for training in
a target domain, it still requires a large amount of training data
A. Security and Privacy and a long training time to achieve good performance. Conse-
As discussed above, adversarial attacks can significantly quently, GAI may not effectively deal with real-time wireless
impact GAI systems. In particular, adversaries can inject channel/environment changes caused by random factors such
crafted perturbations into the input data of GAI models to as mobility, blockage, and interference. For that, it is essential
replicate these models or degrade their performance. More- to develop novel GAI approaches that can quickly adapt to
over, GAI can be exploited by adversaries to generate data track these variations. Integrating advanced ML techniques
that is similar to legitimate/trusted data, making conventional like meta-learning [137] into GAI is a promising direction
security approaches less effective in classifying these adver- to help it quickly adapt to new environmental conditions
sarial attacks. However, there is limited effort in dealing with based on a few training samples. Specifically, meta-learning
adversarial attacks, especially GAI-based attacks in physical can obtain important and useful information in the training
layer communications. One potential approach is to fight fire process of source environments and use that knowledge to
with fire by using GAI models to generate adversarial training quickly learn new environments. With meta-learning, GAI
data and learn on this synthetic data to determine statistical can obtain good accuracy with a few training data samples
anomalies that suggest potential perturbations. Moreover, GAI in new wireless systems, making it more practical in real-
can be used to recover poisoned input data to mitigate the world applications. In addition, over-the-air evaluation and
negative effects of adversarial perturbations [134]. implicit CSI feedback mechanisms should be developed to
 19

further improve the performance of GAI under the dynamics [14] D. Zhang, J. Zhao, L. Yang, Y. Nie, and X. Lin, “Generative adversarial
and uncertainty of physical layer communications. network-based channel estimation in high-speed mobile scenarios,” in
2021 13th International Conference on Wireless Communications and
Signal Processing (WCSP). IEEE, 2021, pp. 1–5.
[15] B. Tolba, M. Elsabrouty, M. G. Abdu-Aguye, H. Gacanin, and H. M.
V. C ONCLUSION Kasem, “Massive mimo csi feedback based on generative adversarial
network,” IEEE Communications Letters, vol. 24, no. 12, pp. 2805–
Generative AI is a promising technology for physical layer 2808, 2020.
communications due to its capabilities of complex data feature [16] H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, and D. N.
extraction, transformation, and enhancement. In this article, Metaxas, “Stackgan: Text to photo-realistic image synthesis with
stacked generative adversarial networks,” in Proceedings of the IEEE
we have presented a comprehensive survey of the applications international conference on computer vision, 2017, pp. 5907–5915.
of generative AI in physical layer communications. Firstly, [17] C. Wang, C. Xu, C. Wang, and D. Tao, “Perceptual adversarial
we have introduced an overview of generative AI, common networks for image-to-image transformation,” IEEE Transactions on
Image Processing, vol. 27, no. 8, pp. 4066–4079, 2018.
generative models, and their advantages compared to tradi- [18] J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye, “A review on generative
tional AI techniques. Then, we have provided detailed reviews, adversarial networks: Algorithms, theory, and applications,” IEEE
analyses, and comparisons of different generative AI tech- transactions on knowledge and data engineering, 2021.
niques in emerging problems in physical layer communications [19] C. Doersch, “Tutorial on variational autoencoders,” arXiv preprint
arXiv:1606.05908, 2016.
such as channel modeling, channel estimation and signal [20] F. P. Casale, A. Dalca, L. Saglietti, J. Listgarten, and N. Fusi, “Gaussian
detection, physical layer security, joint source channel coding, process prior variational autoencoders,” Advances in neural information
beamforming, and intelligent reflecting surface. Finally, we processing systems, vol. 31, 2018.
[21] T. Zhao and F. Li, “Variational-autoencoder signal detection for mimo-
have highlighted important open issues and future research ofdm-im,” Digital Signal Processing, vol. 118, p. 103230, 2021.
directions of generative AI in physical layer communications. [22] Y. Li, X. Chen, and X. Deng, “Joint source-channel coding for a
multivariate gaussian over a gaussian mac using variational domain
adaptation,” IEEE Transactions on Cognitive Communications and
R EFERENCES Networking, 2023.
[23] I. Kobyzev, S. J. Prince, and M. A. Brubaker, “Normalizing flows:
[1] P. P. Ray, “Chatgpt: A comprehensive review on background, appli- An introduction and review of current methods,” IEEE transactions on
cations, key challenges, bias, ethics, limitations and future scope,” pattern analysis and machine intelligence, vol. 43, no. 11, pp. 3964–
Internet of Things and Cyber-Physical Systems, 2023. 3979, 2020.
[2] J. Wang, H. Du, D. Niyato, Z. Xiong, J. Kang, S. Mao et al., “Guiding [24] L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using
ai-generated digital content with wireless perception,” arXiv preprint real nvp,” arXiv preprint arXiv:1605.08803, 2016.
arXiv:2303.14624, 2023. [25] G. Papamakarios, T. Pavlakou, and I. Murray, “Masked autoregressive
[3] H. Du, R. Zhang, Y. Liu, J. Wang, Y. Lin, Z. Li, D. Niyato, J. Kang, flow for density estimation,” Advances in neural information processing
Z. Xiong, S. Cui et al., “Beyond deep reinforcement learning: A tutorial systems, vol. 30, 2017.
on generative diffusion models in network optimization,” arXiv preprint [26] K. He, L. He, L. Fan, Y. Deng, G. K. Karagiannidis, and A. Nal-
arXiv:2308.05384, 2023. lanathan, “Learning-based signal detection for mimo systems with
[4] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, unknown noise statistics,” IEEE Transactions on Communications,
“High-resolution image synthesis with latent diffusion models,” in vol. 69, no. 5, pp. 3025–3038, 2021.
Proceedings of the IEEE/CVF conference on computer vision and [27] L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, Y. Shao,
pattern recognition, 2022, pp. 10 684–10 695. W. Zhang, B. Cui, and M.-H. Yang, “Diffusion models: A com-
[5] S. Liu, T. Wang, and S. Wang, “Toward intelligent wireless commu- prehensive survey of methods and applications,” arXiv preprint
nications: Deep learning-based physical layer technologies,” Digital arXiv:2209.00796, 2022.
Communications and Networks, vol. 7, no. 4, pp. 589–597, 2021. [28] U. Sengupta, C. Jao, A. Bernacchia, S. Vakili, and D.-s. Shiu, “Gen-
[6] T. Wang, C.-K. Wen, H. Wang, F. Gao, T. Jiang, and S. Jin, “Deep erative diffusion models for radio wireless channel modelling and
learning for wireless physical layer: Opportunities and challenges,” sampling,” arXiv preprint arXiv:2308.05583, 2023.
China Communications, vol. 14, no. 11, pp. 92–111, 2017. [29] F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah, “Diffusion
[7] S. M. Aldossari and K.-C. Chen, “Machine learning for wireless models in vision: A survey,” IEEE Transactions on Pattern Analysis
communication channel modeling: An overview,” Wireless Personal and Machine Intelligence, 2023.
Communications, vol. 106, pp. 41–70, 2019. [30] Z. Chen, F. Gu, and R. Jiang, “Channel estimation method based on
[8] R. Sattiraju, A. Weinand, and H. D. Schotten, “Performance analysis of transformer in high dynamic environment,” in 2020 International Con-
deep learning based on recurrent neural networks for channel coding,” ference on Wireless Communications and Signal Processing (WCSP).
in 2018 IEEE International Conference on Advanced Networks and IEEE, 2020, pp. 817–822.
Telecommunications Systems (ANTS). IEEE, 2018, pp. 1–6. [31] A. Oussidi and A. Elhassouny, “Deep generative models: Survey,” in
[9] M. Vahdat, K. P. Roshandeh, M. Ardakani, and H. Jiang, “Papr re- 2018 International conference on intelligent systems and computer
duction scheme for deep learning-based communication systems using vision (ISCV). IEEE, 2018, pp. 1–8.
autoencoders,” in 2020 IEEE 91st Vehicular Technology Conference [32] Y. Cao, S. Li, Y. Liu, Z. Yan, Y. Dai, P. S. Yu, and L. Sun, “A
(VTC2020-Spring). IEEE, 2020, pp. 1–5. comprehensive survey of ai-generated content (aigc): A history of
[10] S. Jo and J. So, “Adaptive lightweight cnn-based csi feedback for mas- generative ai from gan to chatgpt,” arXiv preprint arXiv:2303.04226,
sive mimo systems,” IEEE Wireless Communications Letters, vol. 10, 2023.
no. 12, pp. 2776–2780, 2021. [33] S. Bond-Taylor, A. Leach, Y. Long, and C. G. Willcocks, “Deep
[11] L. Sun, Y. Wang, A. L. Swindlehurst, and X. Tang, “Generative- generative modelling: A comparative review of vaes, gans, normalizing
adversarial-network enabled signal detection for communication sys- flows, energy-based and autoregressive models,” IEEE transactions on
tems with unknown channel models,” IEEE Journal on Selected Areas pattern analysis and machine intelligence, 2021.
in Communications, vol. 39, no. 1, pp. 47–60, 2020. [34] G. Harshvardhan, M. K. Gourisaria, M. Pandey, and S. S. Rautaray, “A
[12] H. Ye, L. Liang, G. Y. Li, and B.-H. Juang, “Deep learning-based comprehensive survey and analysis of generative models in machine
end-to-end wireless communication systems with conditional gans as learning,” Computer Science Review, vol. 38, p. 100285, 2020.
unknown channels,” IEEE Transactions on Wireless Communications, [35] C. Zhang, C. Zhang, S. Zheng, Y. Qiao, C. Li, M. Zhang, S. K. Dam,
vol. 19, no. 5, pp. 3133–3143, 2020. C. M. Thwal, Y. L. Tun, L. L. Huy et al., “A complete survey on
[13] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, generative ai (aigc): Is chatgpt from gpt-4 to gpt-5 all you need?”
S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial net- arXiv preprint arXiv:2303.11717, 2023.
works,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, [36] D. Baidoo-Anu and L. O. Ansah, “Education in the era of generative
2020. artificial intelligence (ai): Understanding the potential benefits of chat-
 20

gpt in promoting teaching and learning,” Journal of AI, vol. 7, no. 1, [59] M. A. Alawad, M. Q. Hamdan, and K. A. Hamdi, “Innovative vari-
pp. 52–62, 2023. ational autoencoder for an end-to-end communication system,” IEEE
[37] S. De, M. Bermudez-Edo, H. Xu, and Z. Cai, “Deep generative models Access, 2022.
in the industrial internet of things: a survey,” IEEE Transactions on [60] N. Samuel, T. Diskin, and A. Wiesel, “Learning to detect,” IEEE
Industrial Informatics, vol. 18, no. 9, pp. 5728–5737, 2022. Transactions on Signal Processing, vol. 67, no. 10, pp. 2554–2564,
[38] H. X. Qin and P. Hui, “Empowering the metaverse with generative 2019.
ai: Survey and future directions,” in 2023 IEEE 43rd International [61] H. Ye and G. Y. Li, “Initial results on deep learning for joint channel
Conference on Distributed Computing Systems Workshops (ICDCSW). equalization and decoding,” in 2017 IEEE 86th vehicular technology
IEEE, 2023, pp. 85–90. conference (VTC-Fall). IEEE, 2017, pp. 1–5.
[39] A. Karapantelakis, P. Alizadeh, A. Alabassi, K. Dey, and A. Nikou, [62] M. Ye, H. Zhang, and J.-B. Wang, “Channel estimation for intelligent
“Generative ai in mobile networks: a survey,” Annals of Telecommuni- reflecting surface aided wireless communications using conditional
cations, pp. 1–19, 2023. gan,” IEEE Communications Letters, vol. 26, no. 10, pp. 2340–2344,
[40] Z. Qin, H. Ye, G. Y. Li, and B.-H. F. Juang, “Deep learning in physical 2022.
layer communications,” IEEE Wireless Communications, vol. 26, no. 2, [63] C. Zou, F. Yang, J. Song, and Z. Han, “Underwater wireless optical
pp. 93–99, 2019. communication with one-bit quantization: A hybrid autoencoder and
[41] T. O’shea and J. Hoydis, “An introduction to deep learning for the generative adversarial network approach,” IEEE Transactions on Wire-
physical layer,” IEEE Transactions on Cognitive Communications and less Communications, 2023.
Networking, vol. 3, no. 4, pp. 563–575, 2017. [64] A. Caciularu and D. Burshtein, “Blind channel equalization using
[42] H. Kim, S. Oh, and P. Viswanath, “Physical layer communication via variational autoencoders,” in 2018 IEEE international conference on
deep learning,” IEEE Journal on Selected Areas in Information Theory, communications workshops (ICC Workshops). IEEE, 2018, pp. 1–6.
vol. 1, no. 1, pp. 5–18, 2020. [65] ——, “Unsupervised linear and nonlinear channel equalization and
[43] H. Sharma and N. Kumar, “Deep learning based physical layer security decoding using variational autoencoders,” IEEE Transactions on Cog-
for terrestrial communications in 5g and beyond networks: A survey,” nitive Communications and Networking, vol. 6, no. 3, pp. 1003–1018,
Physical Communication, p. 102002, 2023. 2020.
[44] A. K. Kamboj, P. Jindal, and P. Verma, “Machine learning-based [66] T. Wu, Z. Chen, D. He, L. Qian, Y. Xu, M. Tao, and W. Zhang, “Cddm:
physical layer security: techniques, open challenges, and applications,” Channel denoising diffusion models for wireless communications,”
Wireless Networks, vol. 27, pp. 5351–5383, 2021. arXiv preprint arXiv:2305.09161, 2023.
[45] F. Restuccia and T. Melodia, “Deep learning at the physical layer: [67] Y. Hu, M. Yin, W. Xia, S. Rangan, and M. Mezzavilla, “Multi-
System challenges and applications to 5g and beyond,” IEEE Commu- frequency channel modeling for millimeter wave and thz wireless
nications Magazine, vol. 58, no. 10, pp. 58–64, 2020. communication via generative adversarial networks,” in 2022 56th
Asilomar Conference on Signals, Systems, and Computers. IEEE,
[46] T. Xu and I. Darwazeh, “Wavelet classification for non-cooperative
2022, pp. 670–676.
non-orthogonal signal communications,” in 2020 IEEE Globecom
[68] Q. Zhang, A. Ferdowsi, and W. Saad, “Distributed generative adversar-
Workshops (GC Wkshps. IEEE, 2020, pp. 1–6.
ial networks for mmwave channel modeling in wireless uav networks,”
[47] C. Liu, Z. Wei, D. W. K. Ng, J. Yuan, and Y.-C. Liang, “Deep transfer
in ICC 2021-IEEE International Conference on Communications.
learning for signal detection in ambient backscatter communications,”
IEEE, 2021, pp. 1–6.
IEEE Transactions on Wireless Communications, vol. 20, no. 3, pp.
[69] T. Hu, Y. Huang, Q. Zhu, and Q. Wu, “Channel estimation enhancement
1624–1638, 2020.
with generative adversarial networks,” IEEE Transactions on Cognitive
[48] N. Van Huynh and G. Y. Li, “Transfer learning for signal detection in Communications and Networking, vol. 7, no. 1, pp. 145–156, 2020.
wireless networks,” IEEE Wireless Communications Letters, vol. 11,
[70] E. Balevi and J. G. Andrews, “Wideband channel estimation with
no. 11, pp. 2325–2329, 2022.
a generative adversarial network,” IEEE Transactions on Wireless
[49] H. Du, D. Niyato, J. Kang, Z. Xiong, P. Zhang, S. Cui, X. Shen, Communications, vol. 20, no. 5, pp. 3049–3060, 2021.
S. Mao, Z. Han, A. Jamalipour et al., “The age of generative ai and [71] Y. Guo, Z. Qin, and O. A. Dobre, “Federated generative adversarial
ai-generated everything,” arXiv preprint arXiv:2311.00947, 2023. networks based channel estimation,” in 2022 IEEE International Con-
[50] J. Wen, J. Nie, J. Kang, D. Niyato, H. Du, Y. Zhang, and M. Guizani, ference on Communications Workshops (ICC Workshops). IEEE, 2022,
“From generative ai to generative internet of things: Fundamentals, pp. 61–66.
framework, and outlooks,” arXiv preprint arXiv:2310.18382, 2023. [72] T. J. O’Shea, T. Roy, and N. West, “Approximating the void: Learning
[51] N. Shlezinger, N. Farsad, Y. C. Eldar, and A. J. Goldsmith, “Viterbinet: stochastic channel models from observation with variational generative
A deep learning based viterbi algorithm for symbol detection,” IEEE adversarial networks,” in 2019 International Conference on Computing,
Transactions on Wireless Communications, vol. 19, no. 5, pp. 3319– Networking and Communications (ICNC). IEEE, 2019, pp. 681–686.
3331, 2020. [73] L. Wei and Z. Wang, “A variational auto-encoder model for underwater
[52] B. Tang, Y. Tu, Z. Zhang, and Y. Lin, “Digital signal modulation acoustic channels,” in Proceedings of the 15th International Conference
classification with data augmentation using generative adversarial nets on Underwater Networks & Systems, 2021, pp. 1–5.
in cognitive radio networks,” IEEE Access, vol. 6, pp. 15 713–15 722, [74] I. Rasheed, M. Asif, A. Ihsan, W. U. Khan, M. Ahmed, and K. M.
2018. Rabie, “Lstm-based distributed conditional generative adversarial net-
[53] S. Lee, Y.-I. Yoon, and Y. J. Jung, “Generative adversarial network- work for data-driven 5g-enabled maritime uav communications,” IEEE
based signal inpainting for automatic modulation classification,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 2, pp.
Access, 2023. 2431–2446, 2022.
[54] E. Shtaiwi, A. El Ouadrhiri, M. Moradikia, S. Sultana, A. Abdelhadi, [75] X. Li, A. Alkhateeb, and C. Tepedelenlioğlu, “Generative adversarial
and Z. Han, “Mixture gan for modulation classification resiliency estimation of channel covariance in vehicular millimeter wave sys-
against adversarial attacks,” in GLOBECOM 2022-2022 IEEE Global tems,” in 2018 52nd Asilomar Conference on Signals, Systems, and
Communications Conference. IEEE, 2022, pp. 1472–1477. Computers. IEEE, 2018, pp. 1572–1576.
[55] M. Li, O. Li, G. Liu, and C. Zhang, “Generative adversarial networks- [76] Q. Zhang, A. Ferdowsi, W. Saad, and M. Bennis, “Distributed condi-
based semi-supervised automatic modulation recognition for cognitive tional generative adversarial networks (gans) for data-driven millimeter
radio networks,” Sensors, vol. 18, no. 11, p. 3913, 2018. wave communications in uav networks,” IEEE Transactions on Wireless
[56] C. Zhao, C. Chen, Z. He, and Z. Wu, “Application of auxiliary Communications, vol. 21, no. 3, pp. 1438–1452, 2021.
classifier wasserstein generative adversarial networks in wireless signal [77] B. Banerjee, R. C. Elliott, W. A. Krzymień, and H. Farmanbar,
classification of illegal unmanned aerial vehicles,” Applied Sciences, “Downlink channel estimation for fdd massive mimo using conditional
vol. 8, no. 12, p. 2664, 2018. generative adversarial networks,” IEEE Transactions on Wireless Com-
[57] Q. Li, Z. Xiang, P. Ren, and W. Li, “Variational autoencoder based munications, vol. 22, no. 1, pp. 122–137, 2022.
receiver for orthogonal time frequency space modulation,” Digital [78] Q. Zhang, H. Dong, and J. Zhao, “Channel estimation for high-speed
Signal Processing, vol. 117, p. 103170, 2021. railway wireless communications: A generative adversarial network
[58] M. A. Alawad, M. Q. Hamdan, K. A. Hamdi, C. H. Foh, and A. U. approach,” Electronics, vol. 12, no. 7, p. 1752, 2023.
Quddus, “A new approach for an end-to-end communication system [79] M. Soltani, V. Pourahmadi, A. Mirzaei, and H. Sheikhzadeh, “Deep
using variational auto-encoder (vae),” in GLOBECOM 2022-2022 IEEE learning-based channel estimation,” IEEE Communications Letters,
Global Communications Conference. IEEE, 2022, pp. 5159–5164. vol. 23, no. 4, pp. 652–655, 2019.
 21

[80] Y. Cai, F. Song, Y. Xu, X. Liu, X. Zhang, and H. Han, “Spectrum [99] T. Roy, T. O’Shea, and N. West, “Generative adversarial radio spectrum
waterfall completion in jamming enviroment: A general adversarial networks,” in Proceedings of the ACM Workshop on Wireless Security
networks method,” in 2020 IEEE 9th Joint International Information and Machine Learning, 2019, pp. 12–15.
Technology and Artificial Intelligence Conference (ITAIC), vol. 9. [100] W. Fan, F. Zhou, and T. Tian, “A deceptive jamming template synthesis
IEEE, 2020, pp. 1661–1665. method for sar using generative adversarial nets,” in IGARSS 2020-
[81] Y. Tang, Z. Zhao, X. Ye, S. Zheng, and L. Wang, “Jamming recognition 2020 IEEE International Geoscience and Remote Sensing Symposium.
based on ac-vaegan,” in 2020 15th IEEE International Conference on IEEE, 2020, pp. 6926–6929.
Signal Processing (ICSP), vol. 1. IEEE, 2020, pp. 312–315. [101] K. S. Germain and F. Kragh, “Physical-layer authentication using
[82] H. Han, X. Wang, F. Gu, W. Li, Y. Cai, Y. Xu, and Y. Xu, “Better channel state information and machine learning,” in 2020 14th Interna-
late than never: Gan-enhanced dynamic anti-jamming spectrum access tional Conference on Signal Processing and Communication Systems
with incomplete sensing information,” IEEE Wireless Communications (ICSPCS). IEEE, 2020, pp. 1–8.
Letters, vol. 10, no. 8, pp. 1800–1804, 2021. [102] ——, “Mobile physical-layer authentication using channel state infor-
[83] K. Merchant and B. Nousain, “Securing iot rf fingerprinting systems mation and conditional recurrent neural networks,” in 2021 IEEE 93rd
with generative adversarial networks,” in MILCOM 2019-2019 IEEE Vehicular Technology Conference (VTC2021-Spring). IEEE, 2021, pp.
Military Communications Conference (MILCOM). IEEE, 2019, pp. 1–6.
584–589. [103] Y. Shi, K. Davaslioglu, and Y. E. Sagduyu, “Generative adversarial net-
[84] P. F. de Araujo-Filho, G. Kaddoum, M. Naili, E. T. Fapi, and Z. Zhu, work in the air: Deep adversarial learning for wireless signal spoofing,”
“Multi-objective gan-based adversarial attack technique for modulation IEEE Transactions on Cognitive Communications and Networking,
classifiers,” IEEE Communications Letters, vol. 26, no. 7, pp. 1583– vol. 7, no. 1, pp. 294–303, 2020.
1587, 2022. [104] B. Barnes-Cook and T. O’Shea, “Scalable wireless anomaly detec-
[85] Y. Yang, L. Zhu, Q. He, and X. Deng, “A simple high-performance tion with generative-lstms on rf post-detection metadata,” in 2022
generation method for spoofing jamming signals,” in 2022 International IEEE Wireless Communications and Networking Conference (WCNC).
Symposium on Networks, Computers and Communications (ISNCC). IEEE, 2022, pp. 483–488.
IEEE, 2022, pp. 1–5. [105] S. Xia, X. Tao, N. Li, S. Wang, T. Sui, H. Wu, J. Xu, and Z. Han,
[86] R. Meng, X. Xu, B. Wang, H. Sun, S. Xia, S. Han, and P. Zhang, “Multiple correlated attributes based physical layer authentication
“Physical-layer authentication based on hierarchical variational autoen- in wireless networks,” IEEE Transactions on Vehicular Technology,
coder for industrial internet of things,” IEEE Internet of Things Journal, vol. 70, no. 2, pp. 1673–1687, 2021.
vol. 10, no. 3, pp. 2528–2544, 2022. [106] Y. Wei, M.-M. Zhao, and M.-J. Zhao, “Model-driven gan-based channel
[87] D. Roy, T. Mukherjee, M. Chatterjee, and E. Pasiliao, “Detection of modeling for irs-aided wireless communication,” in 2021 IEEE Global
rogue rf transmitters using generative adversarial nets,” in 2019 IEEE Communications Conference (GLOBECOM). IEEE, 2021, pp. 1–6.
wireless communications and networking conference (WCNC). IEEE, [107] ——, “Channel distribution learning: Model-driven gan-based channel
2019, pp. 1–7. modeling for irs-aided wireless communication,” IEEE Transactions on
[88] Y. Liu, H. Du, D. Niyato, J. Kang, Z. Xiong, D. I. Kim, and Communications, vol. 70, no. 7, pp. 4482–4497, 2022.
A. Jamalipour, “Deep generative model and its applications in efficient [108] Y. Jin, J. Zhang, C. Huang, L. Yang, H. Xiao, B. Ai, and Z. Wang,
wireless network management: A tutorial and case study,” arXiv “Multiple residual dense networks for reconfigurable intelligent sur-
preprint arXiv:2303.17114, 2023. faces cascaded channel estimation,” IEEE Transactions on Vehicular
[89] A. Toma, A. Krayani, M. Farrukh, H. Qi, L. Marcenaro, Y. Gao, and Technology, vol. 71, no. 2, pp. 2134–2139, 2021.
C. S. Regazzoni, “Ai-based abnormality detection at the phy-layer of [109] Y. Li and J. Chen, “Uplink channel estimation for intelligent reflecting
cognitive radio by learning generative models,” IEEE Transactions on surface aided wireless communication systems with condition gan,”
Cognitive Communications and Networking, vol. 6, no. 1, pp. 21–34, in 2023 5th International Conference on Electronic Engineering and
2020. Informatics (EEI). IEEE, 2023, pp. 328–333.
[90] H. Han, Y. Xu, Z. Jin, W. Li, X. Chen, G. Fang, and Y. Xu, “Primary- [110] F. Naeem, M. Qaraqe, and H. Celebi, “Joint deployment design
user-friendly dynamic spectrum anti-jamming access: A gan-enhanced and phase-shift of irs-assisted 6g networks: An experience-driven
deep reinforcement learning approach,” IEEE Wireless Communica- approach,” IEEE Internet of Things Journal, 2023.
tions Letters, vol. 11, no. 2, pp. 258–262, 2021. [111] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative
[91] H. Han, L. Cui, W. Li, L. Huang, Y. Cai, J. Cai, and Y. Zhang, adversarial networks,” in International conference on machine learning.
“Radio frequency fingerprint based wireless transmitter identification PMLR, 2017, pp. 214–223.
against malicious attacker: An adversarial learning approach,” in 2020 [112] A. M. Elbir, A. Papazafeiropoulos, P. Kourtessis, and S. Chatzino-
International Conference on Wireless Communications and Signal tas, “Deep channel learning for large intelligent surfaces aided mm-
Processing (WCSP). IEEE, 2020, pp. 310–315. wave massive mimo systems,” IEEE Wireless Communications Letters,
[92] L. Yang, S. X. Yang, Y. Li, Y. Lu, and T. Guo, “Generative adversarial vol. 9, no. 9, pp. 1447–1451, 2020.
learning for trusted and secure clustering in industrial wireless sensor [113] L. Pang, Y. Li, Y. Zhang, M. Shang, Y. Chen, and A. Wang,
networks,” IEEE Transactions on Industrial Electronics, vol. 70, no. 8, “Mggan-based hybrid beamforming design for massive mimo sys-
pp. 8377–8387, 2022. tems against rank-deficient channels,” IEEE Communications Letters,
[93] X. Zhou, J. Xiong, X. Zhang, X. Liu, and J. Wei, “A radio anomaly vol. 26, no. 11, pp. 2804–2808, 2022.
detection algorithm based on modified generative adversarial network,” [114] H. Ngo, H. Fang, and H. Wang, “Deep learning-based adaptive beam-
IEEE Wireless Communications Letters, vol. 10, no. 7, pp. 1552–1556, forming for mmwave wireless body area network,” in GLOBECOM
2021. 2020-2020 IEEE Global Communications Conference. IEEE, 2020,
[94] J. Li, X. Zhu, M. Ouyang, W. Li, Z. Chen, and Q. Fu, “Gnss spoofing pp. 1–6.
jamming detection based on generative adversarial network,” IEEE [115] M. Hussain and N. Michelusi, “Adaptive beam alignment in mm-wave
Sensors Journal, vol. 21, no. 20, pp. 22 823–22 832, 2021. networks: A deep variational autoencoder architecture,” in 2021 IEEE
[95] J. Han, Y. Zhou, G. Liu, T. Liu, and X. Zeng, “A novel physical layer Global Communications Conference (GLOBECOM). IEEE, 2021, pp.
key generation method based on wgan-gp adversarial autoencoder,” in 1–6.
2022 4th International Conference on Communications, Information [116] ——, “Learning and adaptation for millimeter-wave beam tracking and
System and Computer Engineering (CISCE). IEEE, 2022, pp. 1–6. training: A dual timescale variational framework,” IEEE Journal on
[96] J. Gong, X. Xu, Y. Qin, and W. Dong, “A generative adversarial Selected Areas in Communications, vol. 40, no. 1, pp. 37–53, 2021.
network based framework for specific emitter characterization and [117] E. Balevi and J. G. Andrews, “Unfolded hybrid beamforming with
identification,” in 2019 11th International Conference on Wireless gan compressed ultra-low feedback overhead,” IEEE Transactions on
Communications and Signal Processing (WCSP). IEEE, 2019, pp. Wireless Communications, vol. 20, no. 12, pp. 8381–8392, 2021.
1–6. [118] Y. M. Saidutta, A. Abdi, and F. Fekri, “Joint source-channel coding
[97] Y. Shi, K. Davaslioglu, and Y. E. Sagduyu, “Generative adversarial over additive noise analog channels using mixture of variational
network for wireless signal spoofing,” in Proceedings of the ACM autoencoders,” IEEE Journal on Selected Areas in Communications,
Workshop on Wireless Security and Machine Learning, 2019, pp. 55– vol. 39, no. 7, pp. 2000–2013, 2021.
60. [119] J. Dai, S. Wang, K. Tan, Z. Si, X. Qin, K. Niu, and P. Zhang, “Nonlinear
[98] T. Erpek, Y. E. Sagduyu, and Y. Shi, “Deep learning for launching and transform source-channel coding for semantic communications,” IEEE
mitigating wireless jamming attacks,” IEEE Transactions on Cognitive Journal on Selected Areas in Communications, vol. 40, no. 8, pp. 2300–
Communications and Networking, vol. 5, no. 1, pp. 2–14, 2018. 2316, 2022.
 22

[120] E. Erdemir, T.-Y. Tung, P. L. Dragotti, and D. Gündüz, “Generative

joint source-channel coding for semantic image transmission,” IEEE
Journal on Selected Areas in Communications, 2023.
[121] X. Niu, X. Wang, D. Gündüz, B. Bai, W. Chen, and G. Zhou, “A hybrid
wireless image transmission scheme with diffusion,” arXiv preprint
arXiv:2308.08244, 2023.
[122] D. Ye, X. Wang, and X. Chen, “Lightweight generative joint source-
channel coding for semantic image transmission with compressed
conditional gans,” in 2023 IEEE/CIC International Conference on
Communications in China (ICCC Workshops). IEEE, 2023, pp. 1–6.
[123] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila,
“Analyzing and improving the image quality of stylegan,” in Pro-
ceedings of the IEEE/CVF conference on computer vision and pattern
recognition, 2020, pp. 8110–8119.
[124] X. Liang, Z. Liu, H. Chang, and L. Zhang, “Wireless channel data
augmentation for artificial intelligence of things in industrial envi-
ronment using generative adversarial networks,” in 2020 IEEE 18th
International Conference on Industrial Informatics (INDIN), vol. 1.
IEEE, 2020, pp. 502–507.
[125] T. Zhang, K. Zhu, and D. Niyato, “A generative adversarial learning-
based approach for cell outage detection in self-organizing cellular
networks,” IEEE Wireless Communications Letters, vol. 9, no. 2, pp.
171–174, 2019.
[126] S. K. Vankayala, S. Kumar, I. Roy, D. Thirumulanathan, S. Yoon, and
I. S. Kanakaraj, “Radio map estimation using a generative adversarial
network and related business aspects,” in 2021 24th International Sym-
posium on Wireless Personal Multimedia Communications (WPMC).
IEEE, 2021, pp. 1–6.
[127] M. Hussien, K. K. Nguyen, and M. Cheriet, “Prvnet: A novel partially-
regularized variational autoencoders for massive mimo csi feedback,”
in 2022 IEEE Wireless Communications and Networking Conference
(WCNC). IEEE, 2022, pp. 2286–2291.
[128] S. Zhang, A. Wijesinghe, and Z. Ding, “Rme-gan: A learning frame-
work for radio map estimation based on conditional generative adver-
sarial network,” IEEE Internet of Things Journal, 2023.
[129] L. Xu, L. Feng, and W. Li, “Ctgan-assisted cnn for high-resolution
wireless channel delay estimation,” in 2023 IEEE 24th International
Conference on High Performance Switching and Routing (HPSR).
IEEE, 2023, pp. 1–8.
[130] C.-K. Wen, W.-T. Shih, and S. Jin, “Deep learning for massive mimo
csi feedback,” IEEE Wireless Communications Letters, vol. 7, no. 5,
pp. 748–751, 2018.
[131] J. Guo, C.-K. Wen, S. Jin, and G. Y. Li, “Convolutional neural
network-based multiple-rate compressive sensing for massive mimo
csi feedback: Design, simulation, and analysis,” IEEE Transactions on
Wireless Communications, vol. 19, no. 4, pp. 2827–2840, 2020.
[132] R. Levie, Ç. Yapar, G. Kutyniok, and G. Caire, “Radiounet: Fast radio
map estimation with convolutional neural networks,” IEEE Transac-
tions on Wireless Communications, vol. 20, no. 6, pp. 4001–4015, 2021.
[133] Y. Teganya and D. Romero, “Deep completion autoencoders for radio
map estimation,” IEEE Transactions on Wireless Communications,
vol. 21, no. 3, pp. 1710–1724, 2021.
[134] G. K. Santhanam and P. Grnarova, “Defending against adversarial
attacks by leveraging an entire gan,” arXiv preprint arXiv:1805.10652,
2018.
[135] H. He, S. Jin, C.-K. Wen, F. Gao, G. Y. Li, and Z. Xu, “Model-driven
deep learning for physical layer communications,” IEEE Wireless
Communications, vol. 26, no. 5, pp. 77–83, 2019.
[136] H. He, C.-K. Wen, S. Jin, and G. Y. Li, “Model-driven deep learning
for mimo detection,” IEEE Transactions on Signal Processing, vol. 68,
pp. 1702–1715, 2020.
[137] T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, “Meta-
learning in neural networks: A survey,” IEEE transactions on pattern
analysis and machine intelligence, vol. 44, no. 9, pp. 5149–5169, 2021.