RAIRO-Oper. Res.

57 (2023) 905–912 RAIRO Operations Research

https://doi.org/10.1051/ro/2023030 www.rairo-ro.org

STEANE CODE ANALYSIS BY RANDOMIZED BENCHMARKING

Anderson de Souza Barbosa1,* , Franklin de Lima Marquezino1 and

Renato Portugal2

Abstract. Quantum error correction codes (QECC) play a fundamental role in protecting the in-
formation processed in today’s noisy quantum computers. To build good error correction schemes, it
is essential to understand how noise affects the behavior of these codes. In this research paper, we
analyze Steane code, a 7-qubit QECC, using a randomized benchmarking (RB) protocol. With RB
protocols, we can partially characterize the quality of implementation of a set of quantum gates. We
show a scenario where Steane code with one logical qubit is advantageous compared to the situation
with no quantum code. We obtained our results using a quantum simulator with custom noise models
considering different numbers of noisy qubits.

Mathematics Subject Classification. 68Q12, 81P68.

Received January 28, 2023. Accepted March 13, 2023.

1. Introduction
Quantum computers should solve certain problems exponentially faster than classical computers with the best
available algorithms. A well-known example of such a problem is integer factorization, which could be solved
efficiently on a quantum computer using Shor’s algorithm [1]. However, the noise present in current quantum
devices partially destroys the quantum information. Quantum algorithms such as Shor’s usually require many
qubits and deep circuits, which generate even more noise. Therefore, we still cannot take advantage of the full
potential of quantum computers for solving real-world problems.
Quantum error correction and detection codes are a way to attempt to get around this problem. They seek to
protect information by creating redundancies, that is, encoding one qubit into several ones, thereby being able to
detect and correct a limited set of errors, which depends on the code. It is important to completely characterize
the quantum noise in order to build good error correction schemes. That is possible through quantum process
tomography. However, this method is not scalable, which makes it unfeasible for large systems [2]. An alternative
is to partially characterize the noise acting in a quantum system and, for that, Magesan et al. [3] proposed a
scalable randomized benchmarking (RB) protocol.
This protocol is suitable for characterizing noise at the level of physical qubits, but to analyze the performance
of a quantum code, a method that evaluates it at the level of encoded logical qubits is the ideal solution. In that
Keywords. Quantum error correction, Steane code, performance evaluation, randomized benchmarking.
1 Systems Engineering and Computer Science Program, Federal University of Rio de Janeiro, Av. Horácio Macedo 2030, Rio de
Janeiro, RJ 21941-914, Brazil.
2 National Laboratory of Scientific Computing, Av. Getúlio Vargas 333, Petrópolis, RJ 25651-075, Brazil.
* Corresponding author: anderson@cos.ufrj.br

○
c The authors. Published by EDP Sciences, ROADEF, SMAI 2023

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
906 A. S. BARBOSA ET AL.

regard, Combes et al. [4] proposed a logical randomized benchmarking protocol that, as the name implies, is
capable of evaluating an error correction implementation at the logical level. In this work, we use this protocol
to evaluate the performance of Steane code, an important quantum error correction code with characteristics
that facilitate the implementation.
In addition to the logical RB, several other variations of the original protocol were proposed. Morvan et al. [5]
extended the RB to make it able to analyze qutrits. Brown and Eastin [6] examined RB protocols based on
subgroups of the Clifford group that are not unitary 2-designs. Magesan et al. [7] proposed a new protocol called
interleaved randomized benchmarking to analyze individual quantum gates. Proctor et al. [8] proposed a direct
RB protocol that avoids compiling 𝑛-qubit Clifford gates, for large 𝑛, into large circuits composed of the native
gates of the device to be characterized.
This work was inspired by the experiments performed by Harper and Flammia [9] with a quantum error
detection code. They analyzed a 4-qubit code with distance 2 that encodes two qubits with a variation of
the RB protocol called Real Randomized Benchmarking [10], which can be used with any gate set that is an
orthogonal 2-design. They observed an improvement in the fidelity of the gates from 94.2%, when no code is
used, to 99.4%, with the use of the 4-qubit code.
Our results show that, under certain circumstances, it is worthwhile to use Steane’s code, as we obtain
a higher fidelity than in the case with no code. Although the conditions under which the experiments were
performed are not those found in the normal use of quantum devices, the results indicate that in these cases we
would be able to obtain similar behaviors.
The rest of this paper is organized as follows. In Section 2, we present the background of this work. In
Section 3, we explain our experiments. In Section 4, we present our final results. Finally, in Section 5, we
summarize our conclusions.

2. Background
In this section we give some details about the code we used, Steane code, explain how to perform the
randomized benchmarking protocol, and show how the quantum channels we considered act on a quantum
state.

2.1. Steane code

Steane code is a quantum error correction code introduced by Steane [11] that encodes one logical qubit into
seven physical ones and has distance 3, which means it can correct arbitrary errors up to 1 qubit. Because of
these characteristics, we say it is a [[7, 1, 3]] code. Furthermore, Steane code is a stabilizer code, that is, its
code space can be generated by a subgroup of the Pauli group, which we call a stabilizer. Table 1 shows all
six generators of Steane code, which consist of tensor products of the Pauli 𝑋 and 𝑍 gates and the Identity
gate (𝐼).
Table 2 shows the main encoded Clifford gates in Steane code. We can see that all qubits can be implemented
in a bitwise fashion and, as all Clifford gates can be implemented as a product of the gates shown in Table 2,
all of them can be implemented in this fashion. We call this a transversal implementation. That is a feature
of a class of quantum codes called CSS (Calderbank–Shor–Steane), to which the Steane code belongs. This
class was discovered by Calderbank and Shor [12] and Steane [13] and is particularly useful due to its easy
implementation.

2.2. Randomized benchmarking

Randomized benchmarking is a protocol for partially characterizing the quality of the implementation of a
set of quantum gates. That is done by estimating the fidelity between the identity channel and the average noise
ℰ acting on the set, which is given by
∫︁
𝐹 (ℰ) = d𝜓 ⟨𝜓| ℰ(|𝜓⟩ ⟨𝜓|) |𝜓⟩ , (1)
 STEANE CODE ANALYSIS BY RB 907

Table 1. Steane code generators.

Name Operator
𝑔1 𝐼 ⊗𝐼 ⊗𝐼 ⊗𝑋 ⊗𝑋 ⊗𝑋 ⊗𝑋
𝑔2 𝐼 ⊗𝑋 ⊗𝑋 ⊗𝐼 ⊗𝐼 ⊗𝑋 ⊗𝑋
𝑔3 𝑋 ⊗𝐼 ⊗𝑋 ⊗𝐼 ⊗𝑋 ⊗𝐼 ⊗𝑋
𝑔4 𝐼 ⊗𝐼 ⊗𝐼 ⊗𝑍 ⊗𝑍 ⊗𝑍 ⊗𝑍
𝑔5 𝐼 ⊗𝑍 ⊗𝑍 ⊗𝐼 ⊗𝐼 ⊗𝑍 ⊗𝑍
𝑔6 𝑍 ⊗𝐼 ⊗𝑍 ⊗𝐼 ⊗𝑍 ⊗𝐼 ⊗𝑍

Table 2. Encoding of the Pauli 𝑋, 𝑌 and 𝑍 gates, the Hadamard gate (𝐻), and the Phase
gate (𝑆), which are some of the main Clifford gates, in Steane code.

Gate Encoding
𝑋 𝑋 ⊗𝑋 ⊗𝑋 ⊗𝑋 ⊗𝑋 ⊗𝑋 ⊗𝑋
𝑌 𝑌 ⊗𝑌 ⊗𝑌 ⊗𝑌 ⊗𝑌 ⊗𝑌 ⊗𝑌
𝑍 𝑍 ⊗𝑍 ⊗𝑍 ⊗𝑍 ⊗𝑍 ⊗𝑍 ⊗𝑍
𝐻 𝐻 ⊗𝐻 ⊗𝐻 ⊗𝐻 ⊗𝐻 ⊗𝐻 ⊗𝐻
𝑆 𝑍𝑆 ⊗ 𝑍𝑆 ⊗ 𝑍𝑆 ⊗ 𝑍𝑆 ⊗ 𝑍𝑆 ⊗ 𝑍𝑆 ⊗ 𝑍𝑆

where d𝜓 is the Haar measure on the space of quantum states |𝜓⟩. The fidelity is a distance measure, that is, it
is a way of quantifying how similar two quantum states are, that is why it is important in the context of error
correction. The fidelity between a pure state |𝜓⟩ and a mixed state 𝜌 is given by
√︀
𝐹 (|𝜓⟩ , 𝜌) = ⟨𝜓| 𝜌 |𝜓⟩. (2)

The RB protocol was proposed by Magesan et al. [2, 3] and is scalable in the number of qubits of the system.
It consists of the following steps:
(1) Choose a sequence length 𝑚 and prepare a state 𝜌.
(2) Choose uniformly at random a sequence of m Clifford gates, apply them to the circuit and apply the
inversion gate, which, in an ideal situation, inverts all the sequence and makes the circuit equivalent to the
identity.
(3) Perform the measurement to verify if the register returned to the initial state 𝜌.
After the development of this RB protocol, several modifications were proposed to adapt it to different
situations and needs. One of these modifications was the logical randomized benchmarking proposed by Combes
et al. [4], which is suitable for benchmarking quantum gates encoded in QECCs. The logical RB protocol can
be performed through the following steps:
(1) Choose an initial state 𝜌 and a sequence length m.
(2) Choose uniformly at random a sequence of m Clifford gates

𝑠𝑚 = {𝑈1 , 𝑈2 , . . . , 𝑈𝑚 }.

(3) Perform the following steps 𝑇 times:

(a) Encode 𝜌 using Steane code.
(b) For each 𝑈 ∈ {𝑈1 , 𝑈2 , . . . , 𝑈𝑚 }:
(i) Apply the encoded version of gate 𝑈 .
(ii) Apply the error correction procedure.
908 A. S. BARBOSA ET AL.

(c) Apply the gate 𝑈 −1 = 𝑈 ∈𝑠𝑚 𝑈 † encoded.

∏︀
(d) Apply the error correction procedure.
(e) Apply the decoding procedure.
(f) Measure 𝜌 with 𝐸, an operator of a POVM.
(4) Calculate the percentage 𝐹𝑚 of executions that returned to the initial state for 𝑚 gates.
Positive Operator-Valued Measure (POVM) is a formalism for the analysis of quantum measurements that
is suitable for applications where we are mainly interested in measurement probabilities associated with each
operator. In the logical RB protocol, the POVM element 𝐸 represents the situation in which the qubit returns
to the initial state, whose probability we want to estimate. The value 𝐹𝑚 obtained with the protocol above
corresponds to the probability that the initial state 𝜌 be recovered at the end of the circuit for a given sequence
length 𝜌. To estimate the average of this probability for all possible sequences, it is necessary to perform the
steps above for different sequences of 𝑚 gates. In addition, the whole procedure must also be done for different
values of 𝑚 so that we obtain several points that associate a sequence length with a probability, known as
survival probability. The curve obtained from these points is known as the fidelity decay curve, which can be
fitted to the model
𝐹𝐿 (𝑚) = 𝐴𝑝𝑚 + 𝐵, (3)
where 𝐴 and 𝐵 are constants that absorb state preparation and measurement errors and 𝑝 is given by

𝑑𝐹 (Λ𝐿 ) − 1
𝑝= , (4)
𝑑−1
where Λ𝐿 is the average noise acting on the code gate set and 𝑑 = 2𝑛 , where 𝑛 is the number of logical qubits.
That means that by finding the value of 𝑝 through the fit, we can determine the average fidelity 𝐹 (Λ𝐿 ).

2.3. Quantum channels

Below we present the three quantum channels that we used to build the noise models: the depolarizing
channel, the amplitude damping, and the phase damping, which are some of the most important. Let 𝜌 be a
mixed quantum state.
– Depolarizing channel: the action of the 𝑛-qubit depolarizing channel is given by
𝑝𝐼𝑛
ℰ𝐷 (𝜌) = + (1 − 𝑝)𝜌 (5)
2𝑛
where 𝑝 is the parameter of the channel and 𝐼𝑛 is the 𝑛 × 𝑛 identity matrix.
– Amplitude damping: the action of the 1-qubit amplitude damping channel is given by

ℰ𝐴 (𝜌) = 𝐴0 𝜌𝐴†0 + 𝐴1 𝜌𝐴†1 (6)

where the Kraus operators are given by

1 0
[︂ ]︂
𝐴0 = √ (7)
0 1−𝜆

and [︃ √ ]︃
0 𝜆
𝐴1 = , (8)
0 0
where 𝜆 is the parameter of the channel. The Kraus operators of the 2-qubit amplitude damping are given
by Guo et al. [14]
𝐴𝑖𝑗 = 𝐴𝑖 ⊗ 𝐴𝑗 , 𝑖, 𝑗 = 0, 1. (9)
 STEANE CODE ANALYSIS BY RB 909

– Phase damping: the action of the 1-qubit phase damping is given by

ℰ𝑃 (𝜌) = 𝑃0 𝜌𝑃0† + 𝑃1 𝜌𝑃1† (10)

where the Kraus operators are given by

𝑃0 = 𝐴 0 (11)
and [︃ ]︃
0 0
𝑃1 = √ , (12)
0 𝜆
where 𝜆 is the parameter of the channel. The Kraus operators of the 2-qubit phase damping are given by

𝑃𝑖𝑗 = 𝑃𝑖 ⊗ 𝑃𝑗 , 𝑖, 𝑗 = 0, 1. (13)

3. Methodology
The experiments from this work were performed in the simulators ibmq qasm simulator , available on IBM
Quantum, and qasm simulator , which can be run locally through Qiskit [15]. We decided to use simulators
instead of real quantum devices because some non-unitary instructions necessary to implement Steane code,
such as mid-circuit measurements, returning a qubit to the state |0⟩ (reset) and applying gates conditioned to
the value of the classical register (if ), were not available on IBM Quantum computers. The first two instructions
became available later, but the if remains unavailable.
To simulate noise, we created noise models based on three of the most important quantum channels, the
depolarizing channel, the amplitude damping, and the phase damping. The first one is characterized by a
parameter 𝑝 and the others by a parameter 𝜆. Our goal was to compare the performance of experiments using
one physical qubit and using one logical qubit encoded in Steane code, which corresponds to 13 physical qubits,
seven from the code block and six ancilla ones. For that, we created noise models for each quantum channel
acting on different numbers of qubits from the code block, from 2 to 7, and we sought to find a parameter
value (threshold), for each model, in which the fidelity decay curves were approximately equal for encoded and
non-encoded experiments. To each model were added 1-qubit and 2-qubit errors from the respective quantum
channel. The 1-qubit errors were associated with the gates 𝑈 1, 𝑈 2, 𝑈 3 and 𝐼, and the 2-qubit errors with the
CNOT gate. It is important to mention that the circuits to be run were initially built with the gates 𝑋, 𝑌 , 𝑍,
𝐻, 𝑆, 𝑆 † and CNOT, but later were converted into equivalent circuits with the gates 𝑈 1, 𝑈 2, 𝑈 3 and CNOT,
that were the standard gates on IBM Quantum devices. We did not add noise on ancilla qubits because the
threshold would be so low that it would require a number of gates in the thousands to generate a useful curve
and, as a result, an amount of memory much higher than the 8 GB available on IBM Quantum.
We performed experiments with initial states |0⟩ and |1⟩ and with 30 different values for 𝑚 starting at 2
and with equal intervals. The lower the threshold found, the greater the interval necessary to generate a good
curve. For each value of 𝑚, we considered the average of the values obtained for |0⟩ and |1⟩. Each circuit was
run 1024 times to estimate the probability of returning to the initial state, which means 𝑇 = 1024 in the logical
RB protocol.
The use of Steane code consists of three procedures: encoding, correction, and decoding. The encoding is done
at the beginning of the circuit, starting from the original state with which we want to work. Figure 1 shows the
circuit that we used to encode the initial state in Steane code, which includes only the seven qubits from the
code block and performs no action on the ancilla. The correction procedure is performed after each encoded
gate is applied and consists of two steps: syndrome measurement and recovery. The syndrome measurement
is performed by measuring each generator of the Steane code. To measure each generator, one ancilla qubit
is needed, and that is why we need six. The combination of the results of all the measurements is the error
syndrome. Let 𝛽𝑖 be the result of the measurement of the generator 𝑔𝑖 and {𝐸𝑗 } be a set of correctable errors
for the Steane code. We perform the recovery by applying 𝐸𝑗 such that 𝐸𝑗 𝑔𝑖 𝐸𝑗† = 𝛽𝑖 𝑔𝑖 for all 𝑖. Finally, the
910 A. S. BARBOSA ET AL.

Figure 1. Circuit to encode quantum state |𝜓⟩ in Steane code.

Figure 2. Threshold as a function of the number of noisy qubits for each quantum channel.

decoding converts the logical state encoded in Steane code into the physical state with the original number of
qubits and is performed by inverting the circuit from Figure 1. These three procedures are steps in the logical
RB protocol presented above.

4. Results
We created a different noise model for each quantum channel and each number of noisy qubits from 2 to 7,
that is, we performed experiments for 18 different models. We found the threshold for each of these models.
Figure 2 shows the thresholds obtained for each case. The graph is in logarithmic scale for better visualization.
The channel that has the highest thresholds was the phase damping channel, which shows it is the one that
causes less damage to the information for the same parameter value. The other channels have similar thresholds,
but those from the depolarizing channel were slightly lower.
We can notice that, in this scale, the points form curves that are close to lines, which indicates that a function
of the form 𝑓 (𝑥) = 𝑎𝑥𝑏 can be fitted to them. The fitted function for the phase damping was 𝑓 (𝑥) = 1.99𝑥−2.32 ,
 STEANE CODE ANALYSIS BY RB 911

Table 3. Average fidelities found. Column Configuration indicates if the fidelity is a result of
non-encoded experiments or the number of noisy qubits in the cases where the Steane code was
used.

Channel Parameter Configuration Average fidelity

5 qubits 0.9983
6 qubits 0.9977
Depolarizing channel 𝑝 = 0.007
7 qubits 0.9969
Non-encoded 0.9967
5 qubits 0.9985
6 qubits 0.9977
Amplitude damping 𝜆 = 0.01
7 qubits 0.9971
Non-encoded 0.9968
5 qubits 0.9981
6 qubits 0.9971
Phase damping 𝜆 = 0.025
7 qubits 0.9961
Non-encoded 0.9960

for the amplitude damping was 𝑓 (𝑥) = 0.73𝑥−2.30 and for the depolarizing channel was 𝑓 (𝑥) = 0.75𝑥−2.59 . The
lines shown in Figure 2 represent the fitted functions. Although our experiments were limited to simulating
noise on the code block, the curves we obtained in Figure 2 show that we can also expect to find thresholds
when simulating noise on the ancilla, which would represent a more realistic scenario where the use of Steane
code is advantageous.
Table 3 shows the main average fidelities found. They refer to the encoded and non-encoded experiments in
which the models, for each channel, were generated using the threshold for seven noisy qubits. In the case of
encoded experiments, column Configuration indicates the number of noisy qubits from the code block, and only
the cases from 5 to 7 qubits are shown. We can see that the average fidelities from the encoded experiments
with seven noisy qubits and the non-encoded ones are almost the same, with a maximum difference of 3 × 10−4 .
That indicates that, with lower parameter values, we would obtain better results when using Steane code than
when no quantum code is used. Furthermore, the fidelities show a scenario, albeit restricted, that clearly favors
Steane code, which is the one in which there are five noisy qubits with noise that can be modeled using the
thresholds corresponding to seven noisy qubits, for each channel.

5. Conclusion
Our objective in this work was to find a scenario in which using a quantum error correction code was beneficial.
We chose Steane code for our experiments because it is an influential code that is also easy to implement. To
evaluate the code and compare it to the situation without error correction, we used the fidelity metric, which
can be estimated using a randomized benchmarking protocol. Furthermore, we chose to run our experiments in
quantum simulators with custom noise models due to the limitations of the quantum devices available in IBM
Quantum.
We found the threshold for all 18 noise models created. In a scenario with the same number of noisy qubits
with noise modeled by one of the three quantum channels considered, if the parameter value is less than the
corresponding threshold, it would be advantageous to use the Steane code. The quantum channel that generated
the lowest thresholds was the depolarizing channel, which means that it is the one that damages the information
the most. On the other hand, phase damping was the channel that generated the highest thresholds. In addition,
we found that on a logarithmic scale the curve of the threshold as a function of the number of noisy qubits is
approximately a line, indicating the possibility of finding thresholds in cases where ancilla qubits are also noisy.
912 A. S. BARBOSA ET AL.

However, it is noteworthy that this curve probably has a different behavior when considering ancilla qubits,
because in this block the structure of the circuit is different.

Acknowledgements. This research has been financially supported by the Brazilian agencies FAPERJ, CAPES and CNPq.

References
[1] P.W. Shor, Algorithms for quantum computation: discrete logarithms and factoring, in Proceedings 35th Annual Symposium
on Foundations of Computer Science. IEEE (1994) 124–134.
[2] E. Magesan, J.M. Gambetta and J. Emerson, Characterizing quantum gates via randomized benchmarking, Phys. Rev. A 85
(2012) 042311.
[3] E. Magesan, J.M. Gambetta and J. Emerson, Scalable and robust randomized benchmarking of quantum processes. Phys.
Revi. Lett. 106 (2011) 180504.
[4] J. Combes, C. Granade, C. Ferrie and S.T. Flammia, Logical randomized benchmarking. Preprint arXiv:1702.03688 (2017).
[5] A. Morvan, V. Ramasesh, M. Blok, J. Kreikebaum, K. O’Brien, L. Chen, B. Mitchell, R. Naik, D. Santiago and I. Siddiqi,
Qutrit randomized benchmarking. Phys. Rev. Lett. 126 (2021) 210504.
[6] W.G. Brown and B. Eastin, Randomized benchmarking with restricted gate sets. Phys. Rev. A 97 (2018) 062323.
[7] E. Magesan, J.M. Gambetta, B.R. Johnson, C.A. Ryan, J.M. Chow, S.T. Merkel, M.P. Da Silva, G.A. Keefe, M.B. Rothwell,
T.A. Ohki, and M.B. Ketchen, Efficient measurement of quantum gate error by interleaved randomized benchmarking. Phys.
Rev. Lett. 109 (2012) 080505.
[8] T.J. Proctor, A. Carignan-Dugas, K. Rudinger, E. Nielsen, R. Blume-Kohout and K. Young, Direct randomized benchmarking
for multiqubit devices, Phys. Rev. Lett. 123 (2019) 030503.
[9] R. Harper and S.T. Flammia, Fault-tolerant logical gates in the IBM quantum experience. Phys. Rev. Lett. 122 (2019) 080504.
[10] A. Hashagen, S. Flammia, D. Gross and J. Wallman, Real randomized benchmarking. Quantum 2 (2018) 85.
[11] A.M. Steane, Error correcting codes in quantum theory. Phys. Rev. Lett. 77 (1996) 793.
[12] A.R. Calderbank and P.W. Shor, Good quantum error-correcting codes exist. Phys. Rev. A 54 (1996) 1098.
[13] A. Steane, Multiple-particle interference and quantum error correction. Proc. R. Soc. London. Ser. A: Math. Phys. Eng. Sci.
452 (1996) 2551–2577.
[14] Y.-N. Guo, Q.-L. Tian, K. Zeng and Z.-D. Li, Quantum coherence of two-qubit over quantum channels with memory. Quantum
Inf. Process. 16 (2017) 1–18.
[15] G. Aleksandrowicz, T. Alexander, P. Barkoutsos, L. Bello, Y. Ben-Haim, D. Bucher, F.J. Cabrera-Hernández, J. Carballo-
Franquis, A. Chen, C.F. Chen and J.M. Chow, Qiskit: an open-source framework for quantum computing (2019). DOI:
10.5281/zenodo.2562111.

Please help to maintain this journal in open access!

This journal is currently published in open access under the Subscribe to Open model
(S2O). We are thankful to our subscribers and supporters for making it possible to
publish this journal in open access in the current year, free of charge for authors and
readers.
Check with your library that it subscribes to the journal, or consider making a personal donation to
the S2O programme by contacting subscribers@edpsciences.org.
More information, including a list of supporters and financial transparency reports,
is available at https://edpsciences.org/en/subscribe-to-open-s2o.