Hybrid Quantum Neural Network Structures for

Image Multi-classification
Mingrui Shi, Haozhen Situ, Cai Zhang
College of Mathematics and Informatics, South China Agricultural
University, No. 483 Wushan Road, Tianhe District, Guangzhou, 510642,
Guangdong, China.
arXiv:2308.16005v1 [quant-ph] 30 Aug 2023

*Corresponding author(s). E-mail(s): zhangcai@[Link];

Contributing authors: shiyue519416@[Link]; situhaozhen@[Link];

Abstract
Image classification is a fundamental computer vision problem, and neural net-
works offer efficient solutions. With advancing quantum technology, quantum
neural networks have gained attention. However, they work only for low-
dimensional data and demand dimensionality reduction and quantum encoding.
Two recent image classification methods have emerged: one employs PCA dimen-
sionality reduction and angle encoding, the other integrates QNNs into CNNs to
boost performance. Despite numerous algorithms, comparing PCA reduction with
angle encoding against the latter remains unclear. This study explores these algo-
rithms’ performance in multi-class image classification and proposes an optimized
hybrid quantum neural network suitable for the current environment. Investigat-
ing PCA-based quantum algorithms unveils a barren plateau issue for QNNs as
categories increase, unsuitable for multi-class in the hybrid setup. Simultaneously,
the combined CNN-QNN model partly overcomes QNN’s multi-class training
challenges but lags in accuracy to superior traditional CNN models. Addition-
ally, this work explores transfer learning in the hybrid quantum neural network
model. In conclusion, quantum neural networks show promise but require further
research and optimization, facing challenges ahead.

Keywords: Image classification, Quantum algorithms, Quantum neural networks,

Convolutional neural networks

1
1 Introduction
Image classification is a crucial task in the field of machine learning. With the
widespread application of digital images and the generation of large-scale datasets,
significant breakthroughs have been achieved in image classification using deep learn-
ing. Deep learning leverages multi-layered neural network structures to automatically
extract features and classify images, enabling the learning of higher-level abstract fea-
tures from raw pixel-level data, thereby enhancing the accuracy and performance of
image classification[1–5]. The success of deep learning can be attributed to its power-
ful model representation capabilities and the training on massive datasets. By stacking
deep neural networks, deep learning learns multi-level feature representations, includ-
ing low-level edges, textures, and higher-level semantic information such as shapes
and structures. Specially designed structures such as Convolutional Neural Networks
(CNNs) and pooling layers enable deep learning to effectively handle spatial locality
and translation invariance in images[6–9].
Quantum Machine Learning (QML) is an advanced research field that combines
quantum computing with machine learning, aiming to leverage the characteristics of
quantum computation to enhance the processing capabilities of conventional machine
learning algorithms[10, 11]. In classical computers, the time complexity for handling
large data vectors is typically polynomial or exponential. However, quantum comput-
ing offers significantly higher storage availability than classical computing, enabling
easy storage and processing of massive data. With the rapid progress of quantum com-
puting technology, there is an urgent need to explore how to maximize its advantages
across various application domains. Implementing quantum computational features in
machine learning will lead to revolutionary changes in computer vision, particularly
in dealing with text, images, videos, and other areas. The applications of quantum
machine learning can be categorized into two primary aspects: optimizing traditional
machine learning algorithms and addressing situations where classical models are
unable to perfectly represent image feature correlations. Quantum machine learning is
based on quantum physics principles, integrating quantum computing with machine
learning to harness quantum devices’ potential for enhancing traditional machine
learning algorithm performance. By utilizing quantum computing concepts to improve
machine learning algorithms, we can enhance existing classification and clustering
algorithms. In quantum machine learning, amplitude encoding can be employed for
data preprocessing, transforming classical machine learning data into formats suitable
for quantum device processing[12, 13].
The Variational Quantum Algorithm (VQA) is a type of quantum algorithm used
in Quantum Machine Learning (QML) to solve optimization problems[14–16]. As a
specific application of VQA, the Parametric Quantum Circuit (PQC), also known as
the Variational Quantum Circuit, is a quantum circuit composed of unitary gates with
freely adjustable parameters. With the development of Parametric Quantum Circuits,
researchers have started exploring their integration with machine learning to address
classical problems[11, 16], such as numerical optimization, approximation, classifica-
tion, and more[17, 18]. Due to the resemblance of Parametric Quantum Circuits to
traditional neural networks, they can be optimized by adjusting the parameters to

2
approximate the target function. Consequently, combining Parametric Quantum Cir-
cuits with neural networks has led to the development of Quantum Neural Network
(QNN) algorithms[19–21]. One key difference between QNN and neural networks is
that all parameter gates in QNN are reversible. Additionally, PQCs do not employ
activation functions for non-linear operations; instead, they use entangling layers to
perform entanglement operations on the output quantum states, achieving non-linear
computations.
Building upon the foundation of Quantum Neural Networks (QNNs), Cong et
al.[22] were inspired by Convolutional Neural Networks (CNNs) and proposed the
Quantum Convolutional Neural Networks (QCNN) model. This model combines and
enhances multiple quantum techniques, such as multiscale entanglement and quantum
error correction, to process input data with fewer network parameters. The QCNN
has been demonstrated to be effectively trainable on near-term quantum devices. Fur-
thermore, researchers have explored the integration of quantum circuits and CNNs.
YaoChong Li et al.[23] introduced an innovative approach by integrating parametric
quantum circuits with CNNs, constructing a recognition model with quantum convo-
lutional layers and quantum classification layers, which achieved remarkable results in
experiments. In the domain of molecular modeling, K.T. Schütt et al. [24] employed
continuous-filter convolutional layers to model local correlations and applied it to
SchNet for modeling quantum interactions within molecules, enabling joint prediction
of total energy and interatomic forces. Tomohiro Mano and Tomi Ohtsuki analyzed
three-dimensional wave functions using image recognition methods based on multi-
layer CNNs[25]. They found that training the network at the center of the band allows
obtaining a complete phase diagram. Alexey A. Melnikov et al.[26] proposed a graph-
based machine learning algorithm, where a designed convolutional neural network
learns to recognize graphs with quantum advantages without executing quantum walks
or random walk simulations. Zhang et al.[27] introduced a quantum-based subgraph
convolutional neural network structure that captures both global topological and local
connectivity structures in graphs, effectively characterizing multiscale patterns present
in the data. These studies provide novel ideas and approaches for the field of quantum
machine learning and demonstrate feasibility and effectiveness through experimental
and numerical simulations.
Existing research has shown that in many cases, quantum algorithms outperform
classical computing algorithms[28, 29], but it has not been proven that quantum
computing can improve performance in all areas. Therefore, a significant amount of
research is currently focused on identifying fields where quantum machines’ parallel
computing capabilities can lead to advancements. In conclusion, quantum machine
learning is a challenging yet promising field. Leveraging the power of quantum comput-
ing can significantly enhance the performance of applications using quantum machine
learning, driving progress in various domains. However, quantum machine learning is
still in a rapid development phase, requiring further research and practice to address
technical challenges. As hardware devices continue to improve, research in the quantum
machine learning field will also advance.
The main focus of this paper is to investigate the strengths and weaknesses of sev-
eral commonly used Hybrid Quantum Neural Network (HQNN) algorithms[30]. There

3
are two main approaches studied: one using Principal Component Analysis (PCA) as
the dimensionality reduction method with angle-encoded QNN models, and the other
combining traditional CNN models with QNN models to form hybrid quantum neural
networks. The first approach is primarily applied to image multi-classification prob-
lems, followed by a comparison with the second approach. Additionally, we propose a
new PQC construction approach by combining new quantum entanglement strategies
and better combinations of quantum parameter gates. Furthermore, we study whether
the HQNN network model improves in performance after applying transfer learning.

2 Related Work
2.1 Amplitude Encoding and Angle Encoding
In machine learning, the representation of data plays a crucial role in determining
the final training effectiveness of algorithms. In classical machine learning problems,
data comes in various forms, including language, images, and various information
components, and is often represented as one-dimensional vectors or multidimensional
arrays. Such data representations facilitate appropriate processing by algorithms and
help achieve desired algorithmic outcomes. However, in quantum machine learning,
the representation of data differs from the traditional approach. For quantum machine
learning to handle classical data, we need to find suitable methods to encode it into
a quantum system. This process is referred to as quantum data encoding or quantum
data embedding. The quality of data encoding directly influences the effectiveness
of quantum machine learning. Several quantum data encoding methods have been
proposed, such as amplitude encoding and angle encoding[22, 31–34].
For a classical dataset X = x(1) , ..., x(i) , ..., x(m) containing m samples, where each
(i) (i) (i)
sample is a vector with n features, x(i) = x1 , x2 , ..., xn , the goal of amplitude
encoding is to encode the feature data of each sample into the amplitudes of a quantum
state. We can use N = log2 (n) quantum bits (qubits) to encode each data sample as
follows:
n
(i)
X
|ψ (i) ⟩ = xj |j⟩ (1)
j=1
It is important to note that the amplitudes of each computational basis in this
quantum state must satisfy the normalization condition, and in Formula 1 it is required
n
(i)
(xj )2 = 1. After encoding the data into the amplitudes of the quantum state
P
that
j=1
using amplitude encoding, subsequent algorithms will involve computations on these
quantum state amplitudes. The advantage of amplitude encoding lies in significantly
reducing the number of quantum bits required for data encoding, meeting the stringent
requirements of NISQ devices for the number of qubits. However, it is worth mention-
ing that this encoding method may have lower efficiency in the preparation process,
which is the process of preparing the quantum state with the desired amplitudes.

4
 For angle encoding, each feature of the data sample is encoded into the rotation
angle of each quantum bit. For the angle encoding of x(i) , we require n quantum bits
to complete the encoding:
n
(i)
O
|ψ (i) ⟩ = R(xj ) (2)
j=1
In angle encoding, R represents single-qubit rotation gates, which typically include
Rx , Ry , and Rz gates. Angle encoding only uses n quantum bits and a constant-
depth quantum circuit, making it suitable for current quantum hardware. However,
for samples with a large number of features, it is not feasible to directly encode the
data using angle encoding. To address this issue, a common approach is to use feature
dimensionality reduction techniques, which reduce the number of features to meet the
requirement of the available number of quantum bits.

2.2 Quantum Neural Network

As a quantum machine learning algorithm, Quantum Neural Network is a model that
combines quantum computing with neural networks. Its main purpose is to leverage
the advantages of quantum computing to handle and learn complex data patterns.
Similar to classical neural networks, a Quantum Neural Network consists of input,
intermediate, and output layers. In QNN, the input layer serves as the quantum data
encoding layer, the intermediate layer represents the parametric quantum circuit layer,
and the output layer corresponds to the quantum measurement layer.
A parameterized quantum circuit is a quantum computing model in which the
parameters of quantum gates can be adjusted based on the problem at hand. These
parameterized gates can be optimized according to the input data to perform spe-
cific quantum computing tasks. Typically, a parameterized quantum circuit consists
of a series of quantum gates with adjustable parameters. These parameters can be
real numbers and can be trained using classical optimization algorithms to maximize
or minimize specific quantum measurement outcomes. By tuning these parameters,
a parameterized quantum circuit can represent a wide range of quantum opera-
tions and quantum state transformations. Parameterized quantum circuits find broad
applications in quantum machine learning and quantum optimization.
In quantum computing, commonly used measurement bases include the Z-basis,
X-basis, Y-basis, etc. The quantum measurement layer in a Quantum Neural Network
often selects the Z-basis measurement, which corresponds to measuring the quantum
state in the Z-direction and obtaining the expectation value. The expectation value can
be understood as the average of measurement outcomes obtained in multiple Z-basis
measurements. Therefore, the output of the Quantum Neural Network is the expecta-
tion value obtained by measuring the quantum state in the Z-basis. The expectation
value of the quantum state, as the output result, possesses some characteristics:
1. Real-valued: The expectation value of a quantum state is a real number that
represents the probability distribution in a specific measurement basis.
2. Interpretability: The expectation value reflects the probability distribution of a
quantum state in a specific measurement basis, which helps us understand the model’s
processing result on input data.

5
 3. Comparability: The expectation values of different samples can be compared,
enabling tasks such as classification, regression, etc.

Fig. 1 a diagram of the quantum neural network. The quantum state is initialized as |0⟩, and
E(x) represents the quantum data encoding layer, which is the implementation to load classical data
into the quantum circuit. U (θ) denotes the parameterized quantum circuit, where θ represents the
adjustable parameters in the circuit. The final layer marked with Z represents the Z-basis measure-
ment in the quantum neural network. Finally, we obtain the output ŷ of the QNN.

Indeed, the output layer of a quantum neural network utilizes a quantum mea-
surement layer, and the output result is represented by the expectation value of the
quantum state. By choosing different measurement bases, one can obtain expectation
values along different directions, enabling the processing and prediction of input data.
As shown in Figure 1, the role of the Quantum Data Encoding Layer E is to
encode classical data x(i) into the quantum circuit for further processing by subse-
quent layers. The PQC (Parameterized Quantum Circuit) layer U performs complex
quantum computations on the input quantum state. Finally, the output is obtained
by measuring the quantum state using the Quantum Measurement Layer, and the
expectation value is used as the network’s output result. When we choose the Z-basis
as the measurement basis, the output of the Quantum Neural Network (QNN) can be
expressed as:

ŷ (i) = ⟨0|E † (x(i) )U † (θ)ZU (θ)E(x(i) )|0⟩ (3)

After obtaining the output vector ŷ, further processing can be performed on it.
Quantum Neural Networks are a practical application of variational quantum
algorithms in quantum computing.

2.3 Deep learning for image classification

Deep learning has made significant progress in image classification tasks, and Con-
volutional Neural Networks are among the most commonly used and successful deep
learning models. The design of CNNs is inspired by the working principles of the
human visual system, allowing them to automatically learn and extract features from
images, resulting in high accuracy in image classification.
A Convolutional Neural Network consists of multiple layers, and its key components
include the convolutional layer, pooling layer, and fully connected layer. In CNNs,

6
the convolutional layer plays the role of feature extraction. It performs convolutional
operations with a set of learnable convolutional kernels on the input image to capture
local features in the image. The convolutional operation slides a window over the image
and performs element-wise multiplication and accumulation with the convolutional
kernel to generate feature maps. By combining multiple convolutional kernels, the
network can learn more abstract and high-level feature representations. Following the
convolutional layer, the pooling layer acts as a downsampling method and preserves
essential information. The pooling layer aggregates features from local regions (max
pooling or average pooling), reducing the size of the feature maps. This helps to
reduce the number of parameters, improve computational efficiency, and enhance the
model’s invariance to translation, scaling, and rotation transformations. The fully
connected layer is situated between the convolutional layer and the output layer. In
the fully connected layer, each neuron is connected to all neurons in the previous
layer. By learning the weight and bias parameters, the fully connected layer transforms
the feature maps extracted by the convolutional and pooling layers into the final
class predictions. The fully connected layer establishes connections between high-level
abstract features and specific classes, enabling classification decisions.
The success of deep learning in image classification relies heavily on large-
scale annotated image datasets and powerful computational resources. By feeding
image data into deep learning models and optimizing them through backpropaga-
tion algorithms, the models can automatically learn and adjust parameters to achieve
optimal performance in image classification tasks. The application of deep learning
in image classification extends beyond traditional object recognition and encom-
passes more complex tasks such as scene understanding, facial recognition, image
generation[35–37].

2.4 Hybrid Quantum Neural Network

Hybrid quantum neural network is a machine learning algorithm that combines quan-
tum neural network algorithms with classical algorithms. Its purpose is to leverage
the quantum computing properties of QNN in classical algorithms or to use classi-
cal algorithms to assist QNN in accomplishing its target tasks[22, 37–40]. There are
common forms of HQNN, such as using QNN as the training model and employing
traditional optimization algorithms like stochastic gradient descent and Adam opti-
mization; alternatively, QNN can be integrated as a component in combination with
a traditional convolutional neural network to form a new model called CNN-QNN,
which is then optimized using classical optimization algorithms[41–43].
In the first type of HQNN model, the main component is composed of QNN[44].
Therefore, the primary concern is how to encode classical data into the QNN model.
This encoding method is known as data encoding, and angle encoding and amplitude
encoding are the main choices. Amplitude encoding requires fewer quantum bits for
data with a large number of features, making it a preferred choice. On the other hand,
angle encoding consumes a significant number of quantum bits, so to avoid resource
consumption, dimensionality reduction techniques like linear discriminant analysis or
principal component analysis are commonly used. The most crucial step in training
the model is to optimize it based on its performance. The optimization process aims

7
to find the optimal model parameters that minimize the loss function, leading to more
accurate predictions on the training data and better generalization ability. Since the
main component of the QNN model is the parameterized quantum circuit, we can use
traditional optimization techniques to adjust its parameters and ensure the model’s
convergence.
In the second type of HQNN model, known as the CNN-QNN model, QNN is
added as a component to the CNN network. Typically, QNN is added in the middle
of the hidden layers, after the convolutional blocks, similar to a fully connected layer,
and sometimes as the output layer of the entire model. For this type of HQNN model,
the training process also falls into two categories. We divide the model into the CNN
part and the QNN part. Without a doubt, the parameters in the QNN part will be
adjusted during training optimization. As for the CNN part, the first case involves
training and optimizing it together with the QNN part. In the second case, transfer
learning is applied to the CNN part, where a pre-trained CNN model is transferred
to the HQNN model without requiring additional training, reducing the training cost
and improving efficiency to some extent.

3 HQNN Model Design for Image

Multi-Classification
3.1 HQNN Model Design with PCA Dimensionality Reduction
In the current mixed quantum neural network, a common approach is to incorporate
QNN as a component into the neural network or directly use QNN as the target
training model. Regardless of the approach, a significant challenge is how to handle
the large amount of data with a limited number of quantum bits. Since image data
contains numerous features, a common solution is to use PCA dimensionality reduction
in combination with angle encoding. This process transforms the high-dimensional
data of classical image data into low-dimensional data that can be encoded with a
small number of quantum bits. Subsequently, the low-dimensional data is input to the
quantum circuit using angle encoding.
As shown in Figure 2, one common construction method for the QNN model
involves encoding the low-dimensional data xlow obtained through PCA dimensional-
ity reduction into the quantum circuit. Each value of the low-dimensional data is used
as the rotation angle for the Ry gate. The subsequent part is a typical parameterized
quantum circuit. The PQC starts with a set of rotation gate operations, consisting
of Rx, Ry, and Rz gates, applied to each input qubit. Then, quantum entanglement
operations are performed on the qubits. In the entanglement part, we require each
qubit to undergo an entanglement operation. This is achieved by using controlled
gates to connect different qubits. There are four common entanglement strategies:
linear entanglement, cyclic entanglement, star entanglement, and fully connected
entanglement[17]. In this design, we use the most commonly used cyclic entanglement
strategy, where each qubit is entangled with its next nearest neighbor qubit. After the
entanglement part, another set of rotation gate operations is typically applied to each
qubit. It is important to note that the parameters of these gates are different from the

8
Fig. 2 Schematic Diagram of HQNN based on PCA Processing.

previous ones, representing a new set of operations. We consider the first rotation oper-
ation, entanglement operation, and second rotation operation together as one layer,
and we can repeat this layer to create a multi-layered quantum circuit. It is worth
noting that the second rotation operation is not necessary in every layer; it is only
needed after the final layer. Finally, we measure the expectation value of each qubit
in the Z basis and use the expectation values from each qubit as our output vector ô.

3.2 HQNN model based on CNN

We can interpret the HQNN model based on CNN from two perspectives.
First Perspective - CNN as the Main Model Body with QNN Embedded as a
Layer. Convolutional Neural Networks (CNN) are currently the most effective and
practical method for processing image data. CNN has given rise to numerous models
designed for various image-related tasks in computer vision, such as LeNet, VGG,
ResNet for image classification, YOLO for object detection[45], and U-net for semantic
segmentation[46]. The goal of the HQNN model based on CNN is to modify the well-
designed CNN model by incorporating the Quantum Neural Network (QNN) as a
quantum circuit layer, referred to as the ”quantum layer,” and adding it to the original
model, as depicted in Figure 3. There are two common approaches in current practices:
one is directly adding the quantum layer to the CNN network, either between fully
connected blocks or as a new output layer; the other is to replace one or more fully
connected layers, utilizing the quantum layer’s output either as the final output or an
intermediate layer. In our illustration, we adopt the approach of adding the quantum
layer after the convolutional blocks and before the output layer. This method aims to
optimize and enhance the original CNN model, resulting in improvements reported in
various articles.

9
Fig. 3 HQNN Model based on CNN Architecture. X represents the input data. The red box rep-
resents the model’s convolutional block, which includes convolutional layers and pooling layers. The
subsequent blue box represents the QNN layer, and the purple box represents the fully connected
layer. Finally, ŷ denotes the model’s output

Second Perspective - QNN as the Main Model Body with CNN as Preprocessing.
In this perspective, the QNN serves as the main model body, and CNN is viewed
as a preprocessing step for the QNN model. The key structure of CNN is the con-
volutional layer, which extracts features from input image data to obtain useful and
high-level feature representations. Through end-to-end training, CNN automatically
learns feature representations, eliminating the need for laborious and subjective man-
ual feature engineering. Moreover, CNN, with convolution and pooling operations,
takes advantage of local perception and parameter sharing mechanisms, effectively cap-
turing local patterns and textures in images, which PCA often fails to do. Additionally,
CNN’s multi-layer convolution and pooling operations enable multi-scale representa-
tions, capturing both fine details and overall information in images, while PCA can
only extract global information. The introduction of non-linear activation functions in
CNN enhances the model’s representational capacity, enabling it to better model com-
plex image features and relationships. As a result, CNN provides a robust foundation
of features for computer vision tasks. After feature extraction by CNN, we continue to
use QNN for the classification task on the obtained feature data. Since QNN’s output
is constrained, we can add a final fully connected layer to control the output.
In conclusion, the HQNN model based on CNN is a novel approach that com-
bines CNN and QNN, aiming to improve the performance of CNN or serve as a
feature extraction step to fulfill QNN’s requirements for low-dimensional features.
The combination of these two models has shown promising results in various research
studies.

10
3.3 The Proposed Model
In quantum machine learning, amplitude encoding has some advantages over angle
encoding. Firstly, amplitude encoding uses the amplitudes of quantum bits to rep-
resent information, which allows it to accommodate more information. In contrast,
angle encoding only utilizes the phase information of quantum bits, resulting in lim-
ited information capacity. Secondly, amplitude encoding exhibits a certain tolerance
to noise and errors. Since amplitude encoding utilizes the amplitudes of quantum bits,
it can retain more information even in the presence of some level of noise and errors.
On the other hand, angle encoding is more susceptible to noise and errors, which may
lead to information loss and misinterpretation[47]. Furthermore, amplitude encoding is
more straightforward to implement during computation and operations. It can leverage
common quantum gate operations for manipulation and measurement, whereas angle
encoding requires more quantum gate operations, making it more complex to imple-
ment. Considering the large amount of data in image information, using amplitude
encoding is more suitable for the current quantum device environment. It can mitigate
the impact of noise to some extent and enable the storage of more information.

Fig. 4 Overall Model Diagram. In the diagram, the blue dashed box represents an example of a
PQC with four qubits.

In the QNN model, one of the most crucial challenges is the construction of Param-
eterized Quantum Circuits. Obtaining a PQC with better simulation performance
and higher expressive power has been a longstanding problem in the field of QNN.
To address this issue, we have incorporated the concept of a single quantum neural
network simulating an approximate function of a single variable[48]. This approach

11
introduces a more expressive parameterized rotation gate U, which is illustrated in
Figure 4 and can be expressed concisely as follows:

U (θ) = Ry (θ1 )Rz (θ2 )Ry (θ3 ) (4)

The parameter vector θ is represented as θ = (θ1 , θ2 , θ3 ) for gate U. After passing
through this combination gate, the initial quantum bits exhibit a broader and more
random distribution on the Bloch sphere mapping. As a result, the PQC has a higher
expressive power.
Furthermore, apart from focusing on innovative ways to design parameter gates, we
should also pay attention to the entanglement capability of each quantum bit[17]. Two
common quantum entanglement strategies are full connectivity and cyclic connectivity,
and full connectivity has some important advantages over cyclic connectivity. Full
connectivity enables direct entanglement between all quantum bits, establishing a
global entanglement relationship where each bit is directly connected to every other
bit. This entanglement strategy exhibits strong entanglement capability, flexibility,
and information transfer efficiency. It can construct entanglement networks of different
scales, catering to various quantum computing tasks and algorithm requirements. Full
connectivity also efficiently transfers and exchanges quantum information, providing
faster and more efficient information transfer. Moreover, full connectivity exhibits
good scalability, allowing easy expansion to larger quantum systems to meet complex
computational demands.
Finally, we continue to use the conventional method of calculating the expectation
value of Z-basis measurements as the output of our QNN.
Simultaneously, we embed the QNN into a simple CNN model to leverage the pow-
erful feature extraction capabilities of CNN. After the QNN, we add a fully connected
layer to obtain a more flexible model output. The overall model is depicted in Figure
4.

4 Experiment and Result Analysis

In order to explore the performance of the model in image classification, we used two
of the most commonly used datasets in the fields of machine learning and deep learn-
ing for image classification: the MNIST dataset and the FashionMNIST dataset. These
two datasets are benchmark datasets in the computer vision domain. The MNIST
dataset contains handwritten digit images with 10 classes, each representing a digit
from 0 to 9. The images are grayscale and have a size of 28x28 pixels, with a total of
60,000 training samples and 10,000 test samples. The MNIST dataset is widely used
to validate and compare the performance of various image classification algorithms,
and it is considered a relatively simple dataset. On the other hand, the FashionM-
NIST dataset is a more challenging dataset, consisting of images of 10 categories of
fashion items such as shirts, pants, and shoes. Compared to the MNIST dataset, the
FashionMNIST dataset contains more complex and diverse image styles, making it
more relevant to real-world applications. It also includes 60,000 training samples and
10,000 test samples, with images of size 28x28 pixels. In our experiments, we used
conventional training optimization methods for all models. The training process was

12
set to 50 epochs, and the commonly used cross-entropy loss function was used as the
loss function for image classification tasks. We employed the Adam optimizer with a
learning rate of 0.01. The main focus of our analysis is to compare the training process
and the final test accuracy of the models to evaluate their performance.
While binary classification has achieved 100% accuracy on many models, we
are more interested in multi-class classification problems. Multi classification better
reflects the complexity and diversity of the real world. Many image datasets involve
multiple categories, such as object recognition, face recognition, scene classification,
etc., which require more fine-grained classification of images to obtain more accurate
results. Multi classification provides richer information and better meets the needs of
real-world applications. For example, suppose we want to build an image recognition
system that can recognize different types of animals. If we simplify the problem to
binary classification, such as distinguishing between cats and dogs, then we will not be
able to distinguish other animal categories, such as birds, elephants, lions, etc. How-
ever, through multi classification, we can train the model to distinguish more animal
categories, giving it a broader range of applications.

Table 1 Results of PCA Multi Classification

PCA+QNN New HQNN

Class 2 8 10
Dimension 8 10 8 10 None
Loss 0.3940 0.4096 1.1819 1.1123 0.4086
Accuracy 0.8011 0.7954 0.2324 0.2245 0.8439

Fig. 5 Comparison between the Model Built with the New PQC Approach and the Traditional PQC
Model. The curves ”loss1” and ”accuracy1” represent the loss and accuracy of the new model, while
the curves ”loss2” and ”accuracy2” represent the loss and accuracy of the original model. The left
plot shows the training results on the MNIST dataset, while the right plot shows the results on the
FashionMNIST dataset.

13
Fig. 6 Training Performance and Comparison of the New Model. The figure illustrates the training
process and accuracy variations of the new model on the two datasets. It shows the training perfor-
mance and accuracy changes of the new model on both the MNIST and FashionMNIST datasets.

According to the model described in Section 3.1, we conducted training and testing
of PCA-generated low-dimensional data at 8 and 10 dimensions for multi-classification
tasks, as shown in Table 1. In the experiments, we first compared the use of PCA for
dimensionality reduction to obtain different low-dimensional datasets to investigate
its impact on the QNN classification problem. To demonstrate the model’s multi-
classification performance, we conducted not only binary classification comparisons
but also 8-class classification comparisons. The results indicate that the HQNN model
used for binary classification, based on PCA dimensionality reduction, exhibited mod-
erate convergence, with accuracy hovering around 80% regardless of whether the PCA
reduced the dimensions to 8 or 10. However, when faced with the 8-class classifica-
tion problem, the model encountered a training issue due to the vanishing gradient
problem, commonly known as the ”barren plateau” problem. The performance on the
FashionMNIST dataset mirrored that on the MNIST dataset, as illustrated in Figure
5, which solely presents the loss reduction process and accuracy variation for the
MNIST dataset. It is evident that the PCA-based HQNN model has demonstrated
significant drawbacks in the context of multi-classification problems.
Subsequently, we trained and tested the new model proposed in Section3.3 and
compared it with the PCA-based approach, as shown in Table 1. Both models were
utilized for the 10-classification task. Leveraging the CNN-based architecture, the new
model not only exhibited excellent training performance but also achieved an accuracy
of approximately 80%. In Figure 6, we presented the training process and accuracy

14
Fig. 7 Comparison between Transfer Learning and Non-Transfer Learning for the New [Link]
figure presents a comparison between the training performance of the new model using transfer
learning and non-transfer learning approaches on both the MNIST and FashionMNIST datasets.
The left graph displays the training results for the MNIST dataset, while the right graph shows the
training results for the FashionMNIST dataset.

variation curves for the new QNN model using CNN dimensionality reduction and
amplitude encoding on both the MNIST and FashionMNIST datasets. The model
achieved an accuracy of around 85% on the MNIST dataset and about 78.2% on the
FashionMNIST dataset. This demonstrates that the new model, employing CNN-based
dimensionality reduction and amplitude encoding, is significantly more effective than
the PCA-based approach with angle encoding in multi-classification problems. Fur-
thermore, we compared the new model with the widely used PQC-based QNN model
on both datasets. The results show that our proposed model not only outperforms the
conventional model during the training process but also achieves an accuracy improve-
ment of approximately 5%. Therefore, our proposed model proves to be more practical
and efficient for image multi-classification tasks.
In the aforementioned new model, we employed a non-transfer learning approach,
while transfer learning might potentially enhance the training process[49]. Transfer
learning is an effective method for training new models[50, 51], allowing the utilization
of existing models and knowledge to improve performance in new tasks or domains,
while also saving training time and resource costs. In transfer learning, a pre-trained
model is typically used as the initial model, which has been trained on a large-scale
dataset. Subsequently, the weight parameters of this model can be used as a starting
point for fine-tuning or further training on the new task. The benefit of this approach
is that the pre-trained model has already learned general feature representations, aid-
ing in faster convergence to the optimal solution for the new task. Hence, we initially
trained a standalone CNN model, and then employed transfer learning by fixing the
parameters of the convolutional blocks and only training the subsequent QNN lay-
ers and fully connected layers, maintaining the same model architecture as the new
model. The use of the pre-trained and fixed convolutional blocks can be likened to
utilizing an effective feature extractor. A comparison between transfer learning and
non-transfer learning for the new model is illustrated in Figure 7. Notably, the non-
transfer trained model exhibited an approximately 5% higher accuracy compared to

15
the transfer learning approach. The underlying cause of this phenomenon might be
attributed to the fact that the introduced QNN layer is less sensitive to the reduced-
dimensional data from the original convolutional blocks when trained in isolation, as
opposed to training the entire model together.

5 Conclusion
In this paper, our main objective was to explore a more practical and efficient quantum
neural network (QNN) algorithm by comparing different HQNN model construction
approaches in image multi-classification problems. Through our research, we found
that the HQNN model using PCA as the dimensionality reduction method encountered
the issue of the vanishing gradient problem in multi-classification problems, while the
combination of CNN and QNN effectively avoided this problem. Moreover, our newly
proposed QNN construction method, which combines amplitude encoding and a more
efficient PQC model, demonstrated better performance in image multi-classification
problems. However, the transfer learning approach applied to the HQNN model that
combines CNN and QNN did not prove to be suitable for this type of problem, or
transfer learning might not be appropriate for such HQNN models. Although our
model outperformed traditional HQNN models in terms of performance, its accuracy
still fell short compared to excellent CNN models in traditional algorithms.
In conclusion, quantum machine learning holds great promise and is expected to
bring revolutionary changes to the field of machine learning. By optimizing traditional
algorithms and handling feature correlations that classical models cannot perfectly
represent, quantum machine learning has the potential to improve the performance of
various applications. While there are still technical challenges and not all fields may
benefit from quantum computing, with more research and advancements, we believe
quantum machine learning will unleash tremendous potential for future scientific and
technological development.

References
[1] LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied
to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)

[2] Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep
convolutional neural networks. Advances in neural information processing systems
25 (2012)

[3] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale
image recognition. arXiv preprint arXiv:1409.1556 (2014)

[4] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D.,
Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings
of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9
(2015)

16
 [5] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recogni-
tion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pp. 770–778 (2016)

[6] Ngiam, J., Chen, Z., Chia, D., Koh, P., Le, Q., Ng, A.: Tiled convolutional neural
networks. Advances in neural information processing systems 23 (2010)

[7] Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional net-
works. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich,
Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp. 818–833 (2014).
Springer

[8] Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv
preprint arXiv:1511.07122 (2015)

[9] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T.,
Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for
mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

[10] Biamonte, J., Wittek, P., Pancotti, N., Rebentrost, P., Wiebe, N., Lloyd, S.:
Quantum machine learning. Nature 549(7671), 195–202 (2017)

[11] Havlı́ček, V., Córcoles, A.D., Temme, K., Harrow, A.W., Kandala, A., Chow,
J.M., Gambetta, J.M.: Supervised learning with quantum-enhanced feature
spaces. Nature 567(7747), 209–212 (2019)

[12] Schuld, M., Petruccione, F.: Supervised Learning with Quantum Computers vol.
17. Springer, Berlin (2018)

[13] LaRose, R., Coyle, B.: Robust data encodings for quantum classifiers. Physical
Review A 102(3), 032420 (2020)

[14] Peruzzo, A., McClean, J., Shadbolt, P., Yung, M.-H., Zhou, X.-Q., Love, P.J.,
Aspuru-Guzik, A., O’brien, J.L.: A variational eigenvalue solver on a photonic
quantum processor. Nature communications 5(1), 4213 (2014)

[15] McClean, J.R., Romero, J., Babbush, R., Aspuru-Guzik, A.: The theory of varia-
tional hybrid quantum-classical algorithms. New Journal of Physics 18(2), 023023
(2016)

[16] Kandala, A., Mezzacapo, A., Temme, K., Takita, M., Brink, M., Chow, J.M.,
Gambetta, J.M.: Hardware-efficient variational quantum eigensolver for small
molecules and quantum magnets. nature 549(7671), 242–246 (2017)

[17] Sim, S., Johnson, P.D., Aspuru-Guzik, A.: Expressibility and entangling capabil-
ity of parameterized quantum circuits for hybrid quantum-classical algorithms.
Advanced Quantum Technologies 2(12), 1900070 (2019)

17
[18] Hubregtsen, T., Pichlmeier, J., Bertels, K.: Evaluation of parameterized quan-
tum circuits: on the design, and the relation between classification accuracy,
expressibility and entangling capability. arXiv preprint arXiv:2003.09887 (2020)

[19] Farhi, E., Neven, H.: Classification with quantum neural networks on near term
processors. arXiv preprint arXiv:1802.06002 (2018)

[20] Schuld, M., Bocharov, A., Svore, K.M., Wiebe, N.: Circuit-centric quantum
classifiers. Physical Review A 101(3), 032308 (2020)

[21] Beer, K., Bondarenko, D., Farrelly, T., Osborne, T.J., Salzmann, R., Scheiermann,
D., Wolf, R.: Training deep quantum neural networks. Nature communications
11(1), 808 (2020)

[22] Cong, I., Choi, S., Lukin, M.D.: Quantum convolutional neural networks. Nature
Physics 15(12), 1273–1278 (2019)

[23] Li, Y., Zhou, R.-G., Xu, R., Luo, J., Hu, W.: A quantum deep convolutional
neural network for image recognition. Quantum Science and Technology 5(4),
044003 (2020)

[24] Schütt, K.T., Sauceda, H.E., Kindermans, P.-J., Tkatchenko, A., Müller, K.-R.:
Schnet–a deep learning architecture for molecules and materials. The Journal of
Chemical Physics 148(24) (2018)

[25] Ohtsuki, T., Mano, T.: Drawing phase diagrams of random quantum systems by
deep learning the wave functions. Journal of the Physical Society of Japan 89(2),
022001 (2020)

[26] Melnikov, A.A., Fedichkin, L.E., Alodjants, A.: Predicting quantum advantage
by quantum walk with convolutional neural networks. New Journal of Physics
21(12), 125002 (2019)

[27] Zhang, Z., Chen, D., Wang, J., Bai, L., Hancock, E.R.: Quantum-based subgraph
convolutional neural networks. Pattern Recognition 88, 38–49 (2019)

[28] Du, Y., Hsieh, M.-H., Liu, T., Tao, D.: Expressive power of parametrized quantum
circuits. Physical Review Research 2(3), 033125 (2020)

[29] Potempa, R., Porebski, S.: Comparing concepts of quantum and classical neural
network models for image classification task. In: Progress in Image Processing,
Pattern Recognition and Communication Systems: Proceedings of the Conference
(CORES, IP&C, ACS)-June 28-30 2021 12, pp. 61–71 (2022). Springer

[30] Arun Pandian, J., Kanchanadevi, K.: Quantum neural network for image classi-
fication using tensorflow quantum. In: Quantum Computing: A Shift from Bits
to Qubits, pp. 171–181. Springer, ??? (2023)

18
[31] Schuld, M., Sweke, R., Meyer, J.J.: Effect of data encoding on the expres-
sive power of variational quantum-machine-learning models. Physical Review A
103(3), 032430 (2021)

[32] Abbas, A., Sutter, D., Zoufal, C., Lucchi, A., Figalli, A., Woerner, S.: The power
of quantum neural networks. Nature Computational Science 1(6), 403–409 (2021)

[33] Lloyd, S., Schuld, M., Ijaz, A., Izaac, J., Killoran, N.: Quantum embeddings for
machine learning. arXiv preprint arXiv:2001.03622 (2020)

[34] Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap
to human-level performance in face verification. In: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)

[35] Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The
pascal visual object classes (voc) challenge. International journal of computer
vision 88, 303–338 (2010)

[36] Radford, A., Metz, L., Chintala, S.: Unsupervised representation learn-
ing with deep convolutional generative adversarial networks. arXiv preprint
arXiv:1511.06434 (2015)

[37] Fan, X., Liu, G., Wang, H., Ma, H., Li, W., Wang, S.: A new image recognition
model based on quantum convolutional neural network. Journal of University of
Electronic Science and Technology of China 51(05), 642–650 (2022)

[38] Henderson, M., Shakya, S., Pradhan, S., Cook, T.: Quanvolutional neural net-
works: powering image recognition with quantum circuits. Quantum Machine
Intelligence 2(1), 2 (2020)

[39] Bokhan, D., Mastiukova, A.S., Boev, A.S., Trubnikov, D.N., Fedorov, A.K.: Mul-
ticlass classification using quantum convolutional neural networks with hybrid
quantum-classical learning. Frontiers in Physics 10, 1069985 (2022)

[40] Trochun, Y., Stirenko, S., Rokovyi, O., Alienin, O., Pavlov, E., Gordienko, Y.:
Hybrid classic-quantum neural networks for image classification. In: 2021 11th
IEEE International Conference on Intelligent Data Acquisition and Advanced
Computing Systems: Technology and Applications (IDAACS), vol. 2, pp. 968–972
(2021). IEEE

[41] Pramanik, S., Chandra, M.G., Sridhar, C., Kulkarni, A., Sahoo, P., Vishwa,
C.D., Sharma, H., Navelkar, V., Poojary, S., Shah, P., et al.: A quantum-classical
hybrid method for image classification and segmentation. In: 2022 IEEE/ACM
7th Symposium on Edge Computing (SEC), pp. 450–455 (2022). IEEE

[42] Abdel-Khalek, S., Algarni, M., Mansour, R.F., Gupta, D., Ilayaraja, M.: Quantum
neural network-based multilabel image classification in high-resolution unmanned

19
 aerial vehicle imagery. Soft Computing, 1–12 (2021)

[43] Yang, C.-H.H., Qi, J., Chen, S.Y.-C., Chen, P.-Y., Siniscalchi, S.M., Ma, X., Lee,
C.-H.: Decentralizing feature extraction with quantum convolutional neural net-
work for automatic speech recognition. In: ICASSP 2021-2021 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6523–6527
(2021). IEEE

[44] Li, W., Chu, P.-C., Liu, G.-Z., Tian, Y.-B., Qiu, T.-H., Wang, S.-M.: An image
classification algorithm based on hybrid quantum classical convolutional neural
network. Quantum Engineering 2022 (2022)

[45] Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified,
real-time object detection. In: Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, pp. 779–788 (2016)

[46] Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No
new-net. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain
Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with
MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers,
Part II 4, pp. 234–244 (2019). Springer

[47] Schuld, M., Killoran, N.: Quantum machine learning in feature hilbert spaces.
Physical review letters 122(4), 040504 (2019)

[48] Yu, Z., Yao, H., Li, M., Wang, X.: Power and limitations of single-qubit native
quantum neural networks. Advances in Neural Information Processing Systems
35, 27810–27823 (2022)

[49] Mari, A., Bromley, T.R., Izaac, J., Schuld, M., Killoran, N.: Transfer learning in
hybrid classical-quantum neural networks. Quantum 4, 340 (2020)

[50] Bengio, Y.: Deep learning of representations for unsupervised and transfer learn-
ing. In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning,
pp. 17–36 (2012). JMLR Workshop and Conference Proceedings

[51] Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with
deep adaptation networks. In: International Conference on Machine Learning, pp.
97–105 (2015). PMLR

20