A Review and Collection of Metrics and

Benchmarks for Quantum Computers:

definitions, methodologies and software
Deep Lall∗1 , Abhishek Agarwal∗1 , Weixi Zhang∗1,2 , Lachlan Lindoy1 , Tobias Lindström1 , Stephanie Webster1 , Simon
Hall1 , Nicholas Chancellor3,4 , Petros Wallden2 , Raul Garcia-Patron2,5 , Elham Kashefi2,6 , Viv Kendon7 , Jonathan
Pritchard7 , Alessandro Rossi1,7 , Animesh Datta8 , Theodoros Kapourniotis9 , Konstantinos Georgopoulos9 , and
Ivan Rungger†1,10
1
National Physical Laboratory, Hampton Road, Teddington TW11 0LW, United Kingdom
2
School of Informatics, QSL, University of Edinburgh, 10 Crichton Street, Edinburgh EH8 9AB, United Kingdom
3
School of Computing, Newcastle University, 1 Science Square, Newcastle upon Tyne NE4 5TG, United Kingdom
4
Department of Physics, Joint Quantum Centre, Durham University, South Road, Durham DH1 3LE, United Kingdom
5
Phasecraft Ltd., London, United Kingdom
arXiv:2502.06717v1 [quant-ph] 10 Feb 2025

6
Sorbonne Université, CNRS, LIP6, 75005 Paris, France
7
Department of Physics, University of Strathclyde, Glasgow G4 0NG, United Kingdom
8
Department of Physics, University of Warwick, Coventry CV4 7AL, United Kingdom
9
National Quantum Computing Centre, Didcot OX11 0QX, United Kingdom
10
Department of Computer Science, Royal Holloway, University of London, Egham, TW20 0EX, United Kingdom

Abstract
Quantum computers have the potential to provide an advantage over classical computers in a number of
areas. The quest towards such quantum advantage has required the development of methods to benchmark
the performance of quantum computers against classical computers as well as the relative performance of dif-
ferent quantum computing platforms. This is a challenging task, firstly due to the large diversity of hardware
platforms, and secondly due to the emergence of two different approaches to quantum computing, one being
a gate-based approach for universal quantum computation, and the other an analogue approach tailored to
outperforming classical computers for specific tasks. Numerous metrics to benchmark the performance of
quantum computers, ranging from their individual hardware components to entire applications, have been
proposed over the years. Navigating the resulting extensive literature can be overwhelming. Objective com-
parisons are further hampered in practice as different variations of the same metric are used, and the data
disclosed together with a reported metric value is often not sufficient to reproduce the measurements. This
article addresses these challenges by providing a review of metrics and benchmarks for quantum computers
and 1) a comprehensive collection of benchmarks allowing holistic comparisons of quantum computers, 2) a
consistent format of the definitions across all metrics including a transparent description of the methodology
and of the main assumptions and limitations, and 3) a reproducible approach by linking the metrics to
open-source software used to evaluate them. The benchmarks span aspects of a device such as architec-
tural properties, quality, speed, and stability, and also include metrics tailored to several non-gate-based
architectures.
Quantum computing technology has reached the stage where a number of methods for performance char-
acterization are backed by a large body of real-world implementation and use, as well as by theoretical
proofs. These mature benchmarking methods will benefit from commonly agreed-upon approaches. Indeed,
such agreement is the only way to fairly, unambiguously, and objectively benchmark quantum computers
across manufacturers. However, there are also benchmarking techniques that are still significantly evolving
and for which standardization would be premature. We identify five areas where international standardiza-
tion working groups could be established, namely: i) the identification and agreement on the categories of
metrics that comprehensively benchmark device performance; ii) the identification and agreement on a set
of well-established metrics that together comprehensively benchmark performance; iii) the identification of
metrics specific to hardware platforms, including non-gate-based quantum computers; iv) inter-laboratory
comparison studies to develop best practice guides for measurement methodology; and v) agreement on
what data and software should be reported together with a metric value to ensure trust, transparency and
reproducibility. We provide potential routes to advancing these areas, thereby contributing to accelerate the
progress of quantum computing hardware towards quantum advantage.

∗ These authors contributed equally

† [email protected]

1
Contents

Contents

I. Introduction 3

II. Overview and review of metrics and benchmarks 6

1. Motivation of metrics selection and categorization 6

2. Short descriptions of the comprehensive set of metrics and benchmarks 7

3. Discussion and outlook 17

III. Background 21
4. Quantum computing approaches and their performance indicators 21
4.1. Concepts for quantum computing performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2. Gate-based quantum computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.3. Measurement based quantum computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.4. Quantum annealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.5. Boson sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5. Hardware platform specific performance overview 28

5.1. Superconducting qubits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.2. Trapped ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.3. Neutral atoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.4. Photonic devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.5. Spin qubits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

6. Concepts and methodologies used for multiple metrics and benchmarks 34

6.1. Qubit states and operations in presence of noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.2. Volumetric benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.3. Gate set tomography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.4. Symbols and notation used for the metrics and benchmarks . . . . . . . . . . . . . . . . . . . . . 43

IV. Metrics and Benchmarks: definitions, methodology and software 45

M1. Hardware architecture properties 47
M1.1. Number of usable qubits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
M1.2. Pairwise connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
M1.3. Native gate set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
M1.4. Capability to perform mid-circuit measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

M2. Qubit quality metrics 52

M2.1. Qubit relaxation time (T1 ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
M2.2. Qubit dephasing time (T2 ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
M2.3. Idle qubit purity oscillation frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

M3. Gate execution quality metrics 57

M3.1. Gate set tomography based process fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
M3.2. Diamond norm of a quantum gate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
M3.3. Clifford randomized benchmarking average gate error . . . . . . . . . . . . . . . . . . . . . . . . . 60
M3.4. Interleaved Clifford randomized benchmarking gate error . . . . . . . . . . . . . . . . . . . . . . . 63
M3.5. Cycle-benchmarking composite process fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
M3.6. Over- or under-rotation angle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
M3.7. State preparation and measurement fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2
I. Introduction

M4. Circuit execution quality metrics 70

M4.1. Quantum volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
M4.2. Mirrored circuits average polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
M4.3. Algorithmic qubits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
M4.4. Upper bound on the variation distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

M5. Well-studied tasks execution quality metrics 78

M5.1. Variational quantum eigensolver (VQE) metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
M5.2. Quantum approximate optimization algorithm (QAOA) metric . . . . . . . . . . . . . . . . . . . . 81
M5.3. Fermi-Hubbard model simulation (FHMS) metric . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
M5.4. Quantum Fourier transform (QFT) metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

M6. Speed metrics 88

M6.1. Time taken to execute a general single- or multi-qubit gate . . . . . . . . . . . . . . . . . . . . . . 88
M6.2. Time to measure qubits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
M6.3. Time to reset qubits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
M6.4. Overall device speed on reference tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

M7. Stability metrics 95

M7.1. Standard deviation of a specified metric evaluated over a time interval . . . . . . . . . . . . . . . 95

M8. Metrics for quantum annealers 95

M8.1. Single qubit control errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
M8.2. Size of the largest mappable fully connected problem . . . . . . . . . . . . . . . . . . . . . . . . . 98
M8.3. Dimensionless sample temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

M9. Metrics for boson sampling devices 102

M9.1. Hardware characterization and model as metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
M9.2. Quantum advantage demonstration as metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

M10. Metrics for neutral atoms devices 106

M10.1. Analogue process fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
M10.2. Trap lifetime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
M10.3. Reconfigurable connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

V. Conclusions 109
References 109

I.Introduction
The field of quantum computation has experienced significant advances in the last decades. The elementary
units of quantum computation – qubits, have been implemented on a wide range of platforms such as supercon-
ducting circuits [1–8], trapped ions [9–16], neutral atoms [17–23], as well as photonic [24–30], and semiconductor
devices [31–37]. The number of qubits implemented in a single device has grown significantly, and devices with
hundreds of qubits have now been demonstrated on various platforms [23, 38–40]. Improvements in the hard-
ware and qubit control have allowed significant reductions in the error rates [1, 9, 21, 28, 31], and research on
reducing the noise induced errors in quantum computers is ongoing. An outstanding challenge for all hardware
platforms towards making quantum computers practically useful is to reduce error rates further. In addition to
developments in the hardware, there have also been advancements in the design of quantum algorithms and in
their resilience to noise [38, 41, 42], which have led to reduced requirements for the implementation of algorithms
on hardware [41, 43, 44].
The most prevalent model of quantum computation is denoted as gate-based (or circuit-based) quantum
computing, where a discrete set of unitary operations, denoted as gates, is used to perform the computation.
Non-gate-based (or non-circuit-based) quantum computing approaches on the other hand are typically designed
to capitalize on specific strengths of each hardware platform to provide advantage over classical computing,
with the drawback that the computations are usually not universal. Such non-gate-based approaches include
quantum annealing [45–47], boson sampling [48–51], and analogue quantum simulation [52–55]. Gate-based
and non-gate-based approaches can also be combined in an algorithm, where each approach is used for those
parts of the algorithm where it provides an advantage over the other. Although claims of quantum computers

3
I. Introduction

outperforming classical computers at certain tasks have been made for both gate-based and non-gate-based
quantum computers [40, 52, 56–59], there has yet to be a clear demonstration of quantum advantage for
practically useful tasks, which is typically denoted as practically useful quantum advantage.
Furthermore, quantum computers are now being made available to academic and industrial users in the
cloud, and it is important that such users have a transparent way to evaluate the capability of the services they
access. Therefore, the need to benchmark these devices using suitable metrics for objective comparisons arises,
both to evaluate the status of a hardware platform and to help guide it towards achieving practically useful
quantum advantage. To this aim the benchmarks need to holistically characterize all aspects of the performance
of a quantum computer that are relevant for the achievement of practically useful quantum advantage. Such
benchmarks make it possible to track the progress of the field over time, and also to guide the development of
the hardware and algorithms to eventually outperform classical computers for certain problems [60, 61].
The development and study of metrics and benchmarks for quantum computers have proliferated immensely
over the past decades, where improvements both in the hardware and in the performance metrics have led to
mutual benefits and faster development. Since its inception, errors in quantum computation due to interaction of
a qubit with its environment were identified among the main difficulties in constructing quantum computers [62,
63]. The ratio between the time taken for individual qubit operations and the time over which qubits can avoid
being perturbed by the environment was used to estimate the number of operations that the devices can be
expected to run successfully [62]. These timescales and their ratio are among the first performance metrics
proposed for quantum computers. In 1995, the first demonstration of a two-qubit gate was performed on
trapped-ions [64]. The state of the qubits with and without the application of the gate on different initial
states was measured and compared to theoretically ideal outcomes. The difference between the observed and
ideal outcomes in the absence of the quantum gate was attributed to errors in state preparation, measurement,
laser cooling, and decoherence. The addition of the gate led to further errors and a larger difference compared
to the ideal outcomes. This method for characterization constituted an early version of tomography of the
quantum state and process, and the evaluation of a fidelity between quantum computing output and the ideal
theoretical target is the foundation of a large part of metrics and benchmarks used today. In the mid-1990s the
development and implementation of tomographic methods for reconstructing quantum states and processes from
experimental data was developed [65–70], and related theoretical work introduced metrics such as entanglement
fidelity and diamond norm [71–74]. The first quantum algorithms were run in 1998 [75, 76]. In the latter,
the quality of the algorithm output was evaluated by reconstructing the quantum state and quantifying its
difference with the ideal outcomes. Variants of this methodology are still used today to evaluate the quality
of algorithm execution on a quantum computer. The fundamental challenge for such methods is that they are
difficult to scale to large number of qubits, because of the exponentially increasing complexity of carrying out
full state or process tomography.
In the late 1990s and early 2000s, research in the metrics for quantum processes led to relations being found
between metrics such as average gate fidelity and entanglement fidelity, generalizations of the metrics, and
methods for experimental determination of the metrics [77–80]. The evaluation of these metrics required the
full characterization of the quantum process, thus limiting their use to small number of qubits. An alternate
approach for characterizing the quality of quantum processes, randomized benchmarking (RB), was proposed
in 2005 [81]. RB uses random gate sequences to obtain averaged fidelities for a set of gates. Subsequently,
significant research effort, still ongoing today, has been dedicated to developing RB, both theoretically and
practically [82–98]. This has led to the development of multiple RB variants that allow characterizing different
kinds of processes, or different kinds of noise, and the creation of a framework encompassing the many kinds
of RB has been proposed [98]. Today, RB based methods are widely used to obtain metrics for the quality
of quantum gates, and often reported by the hardware manufacturers [99]. It is important to note that the
experimentally often-measured average fidelities do not directly correspond to theoretically relevant quantities
such as fault tolerance thresholds, which are quantified by the diamond norm [80, 100]. Another important
step in the development of characterization techniques was the proposal of gate set tomography (GST) in
the early 2010s [101, 102]. GST circumvents various issues faced by prior methods such as quantum process
tomography [103, 104], at the cost of increased complexity and runtime, limiting its use to only few-qubit
systems. Making GST more efficient and scalable to larger systems is an active research endeavor [103, 105–
107].
As the number of qubits in a device grew over the years, with devices having tens of qubits being accessible
to the public by the late 2010s [108], so did the need for the characterization of the performance of entire
devices, rather than just individual operations. One class of approaches towards full device characterization
utilizes randomized circuits. Such approaches include quantum volume [109–111], mirrored circuits average
polarization [94, 112–115], and upper bound of variation distance [116, 117]. Another approach characterizes
the performance of a device by evaluating the quality of outputs for chosen sets of algorithms and applications.
Various benchmarking suites, with different sets of algorithms and different metrics to quantify the quality of a
device in executing those algorithms, have been proposed in recent years [16, 99, 118–133]. With the growth of
cloud-based services, a third approach that introduces notions such as blindness to certify quantum computation

4
I. Introduction

in a distributed setting could become especially appealing [134–136]. Metrics that quantify the speed of devices,
rather than just quality, have also been proposed [137].
In recent years increasing consideration has been devoted to metrics of relevance for quantum error correction
(QEC), since QEC promises a solution to the large error susceptibility of quantum computers [138–147]. In
QEC, the information is processed in so called logical qubits, which are abstract qubits that are encoded on the
states of a larger number of physical qubits in hardware. Certain errors on the logical qubits can be detected
and corrected, and QEC involves the repeated detection and correction of errors during quantum computations.
When error levels on physical qubits are below a threshold that depends on the specific QEC framework, the
logical error rates can be suppressed to arbitrarily low levels and the device is called fault-tolerant. QEC
requires a large number of physical qubits with sufficiently low error-rates, as well as fast classical processing
and feedback to the quantum processor. Today, the improvements in quantum computing capabilities have
allowed for various proof-of-principle demonstrations of QEC [23, 148–158]. Various metrics that evaluate
how well the error-correction performs have been used in the recent literature, including the logical coherence
times [150, 153, 154], the ratio of the logical error rate to the physical error rate [149, 150, 155, 156], and error
per QEC cycle [148–150, 153, 155–157]. As QEC development is fast progressing, the development of QEC
specific metrics is ongoing.
Despite the resulting wide range of available metrics, only a few metrics are widely used in practice. These
widely used metrics are usually more mature in terms of their development, and typically include the qubit
relaxation and dephasing times, typically denoted as T1 and T2 , respectively, or RB-based gate error metrics.
The lack of widespread adoption of a number of other more recently proposed metrics reflects the lack of
consensus on these metrics compared to more mature metrics. Given the significant advancements in the
field, and the increasing need for unambiguous and objective metrics to compare devices, a number of articles
reviewing metrics and proposing various levels of standardization have been compiled recently [60, 114, 159,
160]. Standardization of metrics and benchmarks is as important as it is challenging, given the large number
of metrics available and the potential influence of a standard set of benchmarks on commercial success of a
platform. Some of the benefits of a standardized approach are that it avoids the use of different variants for
evaluating a metric, which all ultimately give the same information with some quantitative deviations. There can
be significant overlaps between the performance indicators of a device that different metrics aim to quantify, so
that the identification of a subset of benchmarks that holistically measures device performance would be useful.
A challenge to this aim is the need of ensuring that no particular hardware platform is unfairly disadvantaged
nor unduly advantaged. The set of benchmarks thus needs to be discussed in view of all the hardware platforms.
For well established metrics, a standardized approach to evaluate them will be beneficial. There are various
subtleties in the evaluation of the metrics, such as the number of measurements taken, averaging procedures,
classical circuit optimizations, and many more. Standardizing these aspects and the information that is reported
together with a metric value is important to ensure transparency and reproducibility [111, 114, 159]. Another
challenge is that, although significant research has been devoted to developing metrics for gate-based quantum
computers, much more limited research exists for non-gate-based devices such as quantum annealers, boson
sampling devices, or analog quantum simulators. It is important not just to devise metrics applicable to these
devices, but also devise metrics that enable comparisons between these devices and traditional gate-based
quantum computers. We identify this as a key area where improvements are needed. For example, there are
very few metrics for quantum annealers which can be applied across different hardware implementations, and
even those which do exist need to be modified and improved as the technology progresses and the devices move
into a more coherent operating regime.
The intent of this article is to cater to the need for objective performance comparisons by defining metrics
and the associated methodologies by which the performance of emerging quantum processors can be holistically
measured and compared. We review the literature and use it to identify a collection of metrics that forms a
comprehensive benchmarking suite covering the relevant aspects of device performance such as speed, quality,
and stability, as well as performs different levels of benchmarking: from component-level benchmarking to
application-level benchmarking. Along with short definitions and longer descriptions of the metrics, we provide
detailed methodologies to evaluate them, their assumptions and limitations, relevant references, and finally
demonstrator software that implements the methodology to measure the metric. In the first part of the document
we also give a general introduction to quantum computing and of some of the widely used hardware platforms,
introducing all the required concepts also for the non-expert readers. The provided collection of metrics and
the associated software package can contribute to the development of a standardized approach for measuring
the performance of quantum computers. By design we restrict the selected metrics in the collection to a limited
number. Considering the fast advancement of the field, the document and software is set up as dynamic online
resource that is regularly updated with new metrics and improved insights on existing ones. As metrics develop,
these can be added in future versions. The methodologies presented in the document for obtaining metrics are
based on what is commonly employed in practice, and as research on metrics and benchmarking progresses,
these methodologies may also be updated.
The linked software package implements the methodologies to evaluate the metrics and is designed to be used

5
1. Motivation of metrics selection and categorization

in conjunction with this document, both as tutorial to provide all information on the implementation details
of the described methodology, as well as a practical tool to evaluate the collection of metrics on a quantum
computing emulator or on a hardware platform. The software either implements the methodology directly or
links to further open source libraries such as Qiskit [161] and PyGSTi [162]. It provides a platform that allows
one to evaluate all the metrics in the collection in a consistent manner, for example by using a single consistent
noise model for the evaluation of all metrics. The effects of changing the noise in the model on all metrics can
then be evaluated and potential correlations across metrics investigated. The software directly implements the
methodology described in this document, focusing on clarity and transparency and hence avoiding optimizations
that may obscure the relation between the described methodology and the software. Having such open-source
software enables reproducibility of the metrics by making visible any parameters used to obtain them, which
may not always be reported otherwise, as well as providing enough output data to be able to others to reproduce
the calculations.
Structure of document: The remainder of this document is divided into four parts: Part II provides an
overview of the chosen metrics across all categories, Part III provides the background, and Part IV presents the
database of metrics, and finally Part V provides the conclusions.
In Part II we both review the relevant literature on performance metrics and list the collection of metrics,
alongside with the motivation of the choice of each metric and of their categorization. We provide a short
summary for each of the metrics included in the collection and what performance related information each
metric provides. At the end of this part we provide a discussion and an outlook on future research in the field
as well as the proposed steps for standardization of metrics and benchmarks.
Part III provides the background with all the required information for the more detailed definitions of the
metrics and methodologies given in Part IV. In Sec. 4 we provide a brief overview of the different quantum
computing approaches and their performance indicators. In Sec. 5 we provide a description of some of the
most widely used hardware platforms including the hardware-specific performance aspects. In Sec. 6 we give a
description of the concepts and methodologies that are used to obtain various metrics presented in the document.
The concepts in this part are presented in the context of quantum computing performance, and also accessible
to the non-expert to equip those new to the field with the knowledge needed for understanding the technical
details of the metrics presented in the database.
In Part IV the detailed definitions of the metrics are presented. Here, the individual sections correspond to
the different categories and metrics presented in part II. This part contains, for each metric, a definition, a
description, the measurement methodology listed as a step by step guide, assumptions and limitations, a link
to the associated software, and references. This part goes together with the software provided at the QCMet
software repository (qcmet.npl.co.uk).

II.Overview and review of metrics and benchmarks

1.Motivation of metrics selection and categorization

The drive to develop quantum computers stems from their potential to be able to perform computations sig-
nificantly faster than classical computers, or even to perform computations that are impossible to perform on
classical computers. A number of performance metrics used to benchmark classical computers can therefore
also directly be applied to quantum computers, such as the speed of operations, the number of processing units,
as well as the probability of errors to occur in the computation. This last aspect is particularly important
in emerging quantum computing hardware, where the error probability is still too large to perform very large
computations for real-world applications.
The collection of metrics selected in this document is designed to comprehensively benchmark quantum
computers across all relevant performance criteria. The metrics are grouped in five categories, each of which
aims to answer questions related to the performance, which combined allow obtaining a comprehensive evaluation
of the performance of a device:
• M1. Hardware architecture properties: What are the device’s architectural properties and what
operations is it able to perform?
• M2-5. Quality metrics: How accurate are the outputs of computations on the device?

• M6. Speed metrics: How fast are operations and algorithms executed on the device?
• M7. Stability metrics: How stable and reliable is the device over time?

6
2. Short descriptions of the comprehensive set of metrics and benchmarks

• M8-10. Non-gate-based quantum computer metrics: What algorithms are non-gate-based quan-
tum computers targeting and what is their performance?
Reporting the values of metrics in categories M1-7 allows obtaining a comprehensive performance evaluation
of a device for general quantum computers; the categories M8-10 on the other hand only apply to specific
non-gate-based quantum computing architectures.
A large number of metrics are dedicated to measuring the quality of the device. This is because measuring
quality is a complex task which can be performed at different levels of abstraction, ranging from the intrinsic
quality of the qubits as components of the quantum hardware to the quality of output of entire applications.
The quality metrics are therefore divided into the following set of categories:

• M2. Qubit quality metrics: How much are idle qubits affected by errors?
• M3. Gate execution quality metrics: How accurately can operations on qubits, denoted as gates, be
performed?
• M4. Circuit execution quality metrics: How accurately can general quantum circuits be executed?

• M5. Well-studied tasks execution quality metrics: How accurately can real-world applications be
executed?
There is a variety of different non-gate-based quantum computing technologies. In this article performance
metrics for three widely studied types of non-gate-based quantum computing approaches are provided:
• M8. Metrics for quantum annealers: How can one characterize the performance of quantum anneal-
ers?
• M9. Metrics for boson sampling devices: How can one characterize the performance of boson
sampling devices?
• M10. Metrics for neutral atoms devices: How can one characterize the performance of neutral atom
devices for analogue simulation?
The full set of metrics is schematically illustrated in Fig. 1.0.1. In the remainder of this section we provide a
summary and overview of the literature for each metric in a category, motivating the relevance of a metric and
explaining which performance criteria it addresses. While the metrics are intended to benchmark physical qubits,
when error-corrected quantum computers are available [138–147], most of the metrics are directly applicable to
the logical qubits generated within quantum error correcting codes as well. For example, the fidelity of a qubit
operation is applicable to both a physical and a logical qubit.

2.Short descriptions of the comprehensive set of metrics and

benchmarks

M1. Hardware architecture properties

The hardware architecture properties determine general capabilities of the hardware. In this document a set
of properties is specified that determines general device performance across hardware platforms. Individual
hardware platforms may specify additional architectural device specific architectural properties.

M1.1: Number of usable qubits

One of the key metrics that determines whether or not a given quantum computer can be used for a par-
ticular task is the number of qubits available in the device [20, 138, 163–165]. Thus, the number of usable
qubits is a key metric that quantifies the capabilities of the hardware [15, 16, 42, 56, 137, 139, 166–171].

M1.2: Pairwise connectivity

To be able to perform complex computations across the whole quantum computer, one needs to ensure that
the states of individual qubits can be entangled across different qubits [172]. Entanglement across different
qubits is the property of a quantum system, where the state of the full quantum system across multiple
qubits cannot be described by the states of individual qubits alone. Entanglement across qubits is generated
by applying multi-qubit gates, typically two-qubit gates. The pairwise connectivity of qubits is specified by
a list of pairs of qubits between which entangling operations can be applied directly. Qubits can also be

7
2. Short descriptions of the comprehensive set of metrics and benchmarks

Performance Metrics for Quantum Computers

M1. HARDWARE ARCHITECTURE PROPERTIES
M1.1. Number of usable qubits
M1.2. Pairwise connectivity
M1.3. Native gate set
M1.4. Capability to perform mid-circuit
measurements

M2. - M5. QUALITY METRICS

HARDWARE MANUFACTURER END-USER

M2. QUBIT QUALITY M3. GATE EXECUTION​QUALITY M4. CIRCUIT EXECUTION​​ M5. WELL-STUDIED TASK
METRICS METRICS QUALITY METRICS EXECUTION​​QUALITY
​ METRICS
M2.1. Qubit relaxation time M3.1. Gate set tomography- M4.1. Quantum volume​
(T1 )​ based process fidelity​ M4.2. Mirrored circuits M5.1. Variational Quantum
M2.2. Qubit dephasing time M3.2. Diamond norm of a average polarization​ Eigensolver​metric
(T2 )​ quantum gate​ M4.3. Algorithmic qubits​ M5.2. Quantum Approximate
M2.3. Idle qubit purity M3.3. Clifford randomized M4.4. Upper bound on the Optimization Algorithm metric​
oscillation frequency​ benchmarking average gate error​ variation distance​ M5.3. Fermi-Hubbard model
M3.4. Interleaved Clifford simulation metric​
randomized benchmarking gate M5.4. Quantum Fourier
error​ Transform​metric
M3.5. Cycle-benchmarking
composite process fidelity
M3.6. Over- or under-rotation
angle
M3.7. State preparation and
measurement fidelity​

HARDWARE COMPONENTS APPLICATIONS

M6. SPEED METRICS

M6.1. Time taken to execute a general single- or
multi-qubit gate
M6.2. Time to measure qubits
M6.3. Time to reset qubits
M6.4. Overall device speed on reference tasks

M7. STABILITY METRICS

M7.1. Standard deviation of a specified metric
evaluated over a time interval

M8. - M10. NON-GATE-BASED QC METRICS

M8. METRICS FOR QUANTUM M9. METRICS FOR BOSON M10. METRICS FOR NEUTRAL
ANNEALERS SAMPLING DEVICES​ ATOM DEVICES​
​
M8.1. Single qubit control errors​ M9.1. Hardware characterization M10.1. Analogue process fidelity
M8.2. Size of largest mappable and model as metrics M10.2. Trap lifetime
fully connected problem​ M9.2. Quantum advantage M10.3. Reconfigurable connectivity​
M8.3. Dimensionless sample demonstration​as metric
temperature​

Figure 1.0.1.: Schematic of the collection of metrics presented in this document, with each category illustrated in a
different color.

entangled indirectly by swapping the states of qubits, then applying the direct entangling operation between
a pair of directly connected qubits, and then swapping the states of qubits back. The swapping operations
can be done either by physically moving around the qubits [9, 19, 23, 115] or by applying entangling opera-
tions that swap the information in the qubits [173–175]. The overhead of swapping qubits can be large [129,
176–178], leading to longer execution times and worse performance due to noise. Thus, a high degree of con-
nectivity can be very advantageous. Knowledge of the pairwise qubit connectivity can be used to optimize
quantum circuits in order to improve performance [179–186]. Thus, the pairwise connectivity of a device is
an important architectural property [16, 38, 56, 109, 125, 129, 187].

M1.3: Native gate set

For a quantum computer to be universal it needs to be able to perform arbitrary unitary operations [163,

8
2. Short descriptions of the comprehensive set of metrics and benchmarks

172]. Any multi-qubit unitary operation can be decomposed into a circuit consisting of general single-qubit
and two-qubit gates [163, 188]. It is important to determine the gates that the hardware can perform na-
tively [125, 189], and whether these can be combined to generate general single-qubit gates and a two-qubit
gate allowing for universal computation. The set of these native gates is denoted as the native gate set. Quan-
tum circuits for a specific hardware can be designed to predominantly use the native gates, thereby avoiding
more general gates that can require expensive decompositions in terms of number of native gates [186, 190–
194]. Decomposing general operations into native gates can introduce a significant overhead [125, 190, 195].

M1.4: Capability to perform mid-circuit measurements

Quantum algorithms can utilize qubit re-use in order to improve the resource requirements of the algo-
rithms [196–201]. Qubit re-use involves measuring and then resetting a qubit multiple times within a single
quantum circuit execution. Thus, the capability to perform mid-circuit measurements determines whether
such quantum algorithms can be run on a device. This capability is also important because it is required
to perform quantum error correction [142] and for applying gates conditioned on previous measurement
results [202–206].

M2-5. Quality metrics

Compared to existing classical computers the current generation of quantum computers is more error-prone,
limiting the usefulness of the devices. The possibility to apply quantum error correction also requires error
rates to be below some code dependent thresholds. The quality metrics are designed to quantify the error
rates of quantum computing hardware, which is achieved by quantifying various differences of the outputs of
a device when compared to the outputs expected for an ideal noise-free quantum computer. In evaluating the
performance of a quantum computer, quantifying the quality is one of the most challenging aspects [165], since
there are various different sources of errors, and these sources can be different for different hardware platforms.

M2. Qubit quality metrics

Qubits are not isolated from their environment, and their interaction with the environment can significantly
perturb their quantum state, potentially leading to a full loss of information on the state of the qubit [3, 163,
207–212]. The effects of the environment are generally described by various types of noise acting on the qubit.

M2.1: Qubit relaxation time (T1 ) and M2.2: Qubit dephasing time (T2 )
Two dominant sources of noise due to interactions with the environment are amplitude damping (also denoted
as relaxation), and dephasing: amplitude damping occurs due to the qubit losing energy to the environment
and changing its state, while dephasing predominantly occurs due to fast fluctuations of the qubit frequency
due to the environment interaction. The qubit relaxation time, T1 , and the qubit dephasing time, T2 , quantify
the effects of amplitude damping and dephasing, respectively [2, 9, 22, 163, 212–220]. These metrics quantify
the major noise contributions that typically determine the duration over which a qubit preserves its state
and stays coherent, which is denoted as the coherence time. They are therefore central metrics that quantify
the quality of single qubits. Longer coherence times allow running more operations on the qubit before the
results are significantly affected by decoherent noise such as amplitude damping and dephasing, which reduce
the qubit coherence.
The T1 and T2 taken in isolation provide only a partial picture for qubit performance in quantum circuits,
since what determines the number of gates that can be successfully applied is the ratio between these times
and the duration of a quantum gate. Different hardware platforms can have gate durations that differ by
orders of magnitude [23, 38, 115, 129, 148]. For example, ion-trap based devices typically have orders of
magnitude longer coherence times than devices such as superconducting qubits, but also typically have much
longer gate durations [129].

M2.3: Idle qubit purity oscillation frequency

In quantum computing hardware, the qubit-environment interaction typically changes over time, both over
fast and slow time-scales. If the time-scale of such variations is comparable to the circuit execution time,
then the noise affecting the qubits can vary significantly during one execution of a quantum circuit. In such
cases the noise due to the environment cannot be accurately modelled as Markovian noise, which assumes
time-independence of the noise. Instead, one must account for the non-Markovianity due to the changing
environment [221, 222]. The idle qubit purity oscillation frequency is a metric that indicates the presence of
non-Markovian effects in the time evolution of a qubit [223–225]. Various other methodologies for charac-
terizing more general non-Markovian noise have been proposed [226–231]. However, they typically require

9
2. Short descriptions of the comprehensive set of metrics and benchmarks

running a large number of circuits as part of the characterization.

M3. Gate execution quality metrics

The quality of individual gates that are applied is determined by different factors such as qubit decoherence [122,
208, 213, 219], miscalibrations [92, 232–237], or errors such as leakage [1, 9, 238–245] and crosstalk [17, 21, 245–
262]. The metrics included in category M2. Qubit quality metrics also quantify some of the dominant sources
of errors that affect the quality of the gates [148, 263–265]. Often, there is a trade-off between the gate control
parameters: longer gate durations can allow reducing the effects of leakage but come at the cost of increased
qubit decoherence during gate execution [233, 239].
The effects of these errors are that the state resulting from the application of a gate is different from the
ideal expected state in the absence of errors. The metrics included in this category are designed to quantify
the overall difference between the ideal noise-free gate and the gate executed on hardware. The metrics do not
directly relate gate execution qualities to the individual noise sources, which would require accurate modelling
of the noise in the devices and its environment, which is a challenging task. Instead, they provide averaged
effects on the gate operations generated by such noise. Since the level and type of averaging is not unique, there
are many metrics for gate quality.
The noisy gates are not described by fully unitary quantum operations due to the effects of decoherence.
Characterizing these effective non-unitary operations can be a challenging task [103]. One method which allows
obtaining such descriptions of the effective quantum processes is known as quantum process tomography [66, 67,
103, 266–270]. Process tomography consists of determining the quantum states resulting from the application
of the target gate on a set of input states that spans the space of all possible input states. The quantum states
are measured using quantum state tomography [70, 103, 271–279], which involves measurements of the quantum
state in different bases, such that the measurements can be used to reconstruct the quantum state. This method
is sensitive to errors such as state preparation and measurement errors, limiting its usefulness [101, 103, 271]. A
variant of process tomography that takes into account state preparation and measurement (SPAM) errors is Gate
Set Tomography (GST) [101–104, 107, 280–282]. It involves solving an optimization problem that characterizes
the quantum gates, state preparation errors, and readout errors simultaneously. GST requires running a large
number of circuits and is time-consuming, and so is only feasible for few-qubit operations. Once one has obtained
a description of the quantum process using GST, one can quantify the difference with the ideal process. There
are various metrics that can be used to quantify the difference [77, 78, 80, 283]. A fundamental assumption in
GST is that the noise is constant over the measurement acquisition time, or more generally that the noise is
Markovian. This is often not the case in quantum computing hardware. The assumption of Markovian noise
can be a significant shortcoming in devices with a large amount of non-Markovian noise, and other methods
which can account for the temporal or system-environment correlations causing the non-Markovianity must be
used in these cases. Some examples include process tensor tomography [226], non-Markovian quantum process
tomography [227], non-Markovian gate set tomography [230], fast Bayesian tomography [228], and machine
learning based methods [229, 284]. The drawback of these methods is that they typically require running a
larger number of circuits and solving more challenging classical optimisation problems.
In what follows we describe the metrics included in this category, some of which are based on GST.

M3.1: Gate set tomography based process fidelity

The process fidelity, also known as entanglement fidelity, quantifies the deviation of the gate executed on
hardware from the ideal noiseless gate, and can be calculated using GST [71, 77, 80, 113]. It is related to
the average fidelity of a quantum gate [77–79, 113], which quantifies the average closeness, over all possible
pure input states, between the states resulting from the application of the actual and the ideal process [77,
80, 285].
M3.2: Diamond norm of a quantum gate
In contrast to the average fidelity, which averages over all initial states, the diamond norm [74, 80, 280] is
the minimum trace distance, over all possible initial states, of the states resulting from the application of the
actual and ideal process. Thus, this metric measures the maximum distinguishability of the ideal and actual
process. Such worst-case measures are especially relevant when comparing fidelities against fault-tolerant
thresholds [74, 100, 280, 286].

M3.3: Clifford randomized benchmarking average gate error

The process fidelity, average fidelity, and the diamond norm can all quantify the quality of a specific gate.
However, in a quantum circuit, one typically uses a wide set of gates, so that for a performance indicator
for a quantum computer to execute general quantum circuits one may consider an averaged fidelity over a
set of gates included in typical circuits. Clifford Randomized Benchmarking (RB) is a method that allows
calculating such an average error rate, where the considered set of operations are the Nq −qubit Clifford op-

10
2. Short descriptions of the comprehensive set of metrics and benchmarks

erations [81–83, 86, 91, 92, 287–290]. The advantage of RB is that it does not require expensive tomographic
methods to find the effective operators. It instead runs random circuits and evaluates the probability of ob-
taining the correct output. The error rate is estimated from the exponential decay of this success probability
with the circuit length. This error rate corresponds to the gate fidelity averaged over all possible input states
and all Nq −qubit Clifford gates [98]. Note that this only holds if the noise in the quantum computer is com-
pletely Markovian and gate independent[98, 289]. It is therefore important to also quantify the amount of
non-Markovianity of the noise, for example using the the metric M2.3 Idle qubit purity oscillation frequency.

M3.4: Interleaved Clifford randomized benchmarking gate error

Variants of RB have been proposed and used to quantify the quality of individual operations. Interleaved
randomized benchmarking (IRB) [90–92, 95, 98, 291] is one such protocol that can be utilized to estimate the
average error of individual Clifford gates, or even individual non-Clifford gates with some variants [98, 291].
In this approach one interleaves multiple repetitions of the gate of interest within the RB circuits, so that the
difference between the fidelity of the RB circuits and the interleaved circuits allows estimating the fidelity
of the interleaved gate. Other methods that have been proposed to characterize the quality of individual
operations include random circuit sampling [292] and direct randomized benchmarking [87]. Methods based
on RB can also used to evaluate the quality of mid-circuit measurements [97]. For an overview of different
variants of RB, see Ref. [98].

M3.5: Cycle-benchmarking composite process fidelity

Running GST to obtain the process fidelity of a quantum gate can be costly in terms of the number of circuits
required to run, and the amount of time spent on classical optimization. Cycle benchmarking [96, 293–297]
has been proposed as a way to more efficiently measure the process fidelity of a quantum gate, or a layer
of quantum gates, by using randomized compiling [297, 298]. Cycle benchmarking has also been proposed
to benchmark multi-qubit operations while also benchmarking their effects on other qubits. This allows for
the quantification of an important kind of noise that affects multiple qubits simultaneously called crosstalk.
Crosstalk describes how qubits affect each other, both in the idle state and also when gates are applied. This
noise can take various forms, and various metrics have been proposed to quantify the different effects [247].
There are metrics that quantify the averaged effect of the crosstalk errors on quantum gates rather than
describing the individual sources of crosstalk noise. Some examples of this are simultaneous randomized
benchmarking [88, 288] or extensions of it, such as correlated randomized benchmarking [299]. These are
variants of randomized benchmarking that can be used to characterize locality and weight of crosstalk errors.
Methods such as cycle benchmarking [96, 293–297] have been proposed to benchmark multi-qubit operations
while taking into account correlated noise sources such as crosstalk.

M3.6: Over- or under-rotation angle

Limited gate calibration precision, drifts on device properties, or undesired interactions that cannot be turned
off can lead to systematic or coherent errors in the device [92, 232, 234, 280, 287, 300–304]. Single-qubit
unitary gate operations can be represented as rotation gates on the Bloch sphere. One measure of the quality
of such gates is how accurately these rotations are implemented. Control errors can lead to over-rotation
or under-rotation errors, where the actual rotation amount corresponding to the operation differs from the
desired rotation. Characterizing these kinds of errors often involves repeated application of these gates in
order to amplify the errors before measuring their effects [92, 300].

M3.7: State preparation and measurement fidelity

Aside from the typical unitary operations applied to the qubits, it is also important to quantify the quality of
the state preparation and measurement (SPAM) [237, 305–314]. Similar to how quantum state tomography
can be used to characterize quantum states, quantum measurement tomography [308, 315–318] can be used
to characterize the native measurement operations. However, state tomography assumes an ideal readout,
and measurement tomography assumes ideal state preparation. Thus, when both state preparation and
readout errors are non-negligible, it is challenging to directly characterize the two processes independently
without other assumptions, such as ideal qubit gates [319]. Since GST involves the characterization of the
qubit gates together with SPAM errors, it circumvents the issue of characterizing operations independently
by jointly characterizing the operations during the optimization procedure [103]. The SPAM fidelity can
therefore be obtained using the GST results.

The methods discussed above can differ significantly in the required execution time. For example, RB based
methods involve running random operations such as the application of random Clifford gates. On the other
hand, evaluating the diamond norm typically requires a complete description of the process, which can be
obtained by performing GST. Typically, GST requires running more circuits than RB methods, and involves
a solving a much more difficult classical optimization problem, so that it takes significantly longer to run.

11
2. Short descriptions of the comprehensive set of metrics and benchmarks

The longer execution times of GST also reduce its ability to provide time-resolved fluctuations of metrics. On
the other hand, the advantage of GST is that it provides a much more detailed characterization the quantum
processes, so that the results of GST can be used to evaluate multiple metrics unlike RB based methods. Thus,
when choosing which metrics to measure, one must consider factors such as the number of quantum circuits
that need to be executed and the classical computational time overhead.

M4. Circuit execution quality metrics

The effect of individual gate errors on the full circuit execution result is difficult to estimate, since each quantum
circuit amplifies individual gate errors in a different way. The quality estimates of individual quantum gates
obtained with the metrics in category M2 can therefore only give a very approximate indication of how well a
device executes entire quantum circuits. Instead, to evaluate the quality of circuit execution one can use metrics
that characterize the quality of execution of entire circuits directly [94, 109, 320, 321].

M4.1: Quantum volume

A single-number metric that aims to quantify the performance of a device in running certain kinds of quan-
tum circuits is the quantum volume [109–111, 137, 322]. The metric is calculated by running randomized
square circuits, where the number of circuit layers are equal to the number of qubits, for increasing numbers
of qubits. The quantum volume is evaluated by finding the largest square circuit that can be run on the
device and that also passes an acceptance criterion that quantifies the quality of the output. Importantly,
this requires classical simulation of the quantum circuits, which limits scalability. A further drawback of
quantum volume is that it is not necessarily indicative of the performance of a device in running circuits
which are not square, such as circuits which have depths that scale non-linearly with the number of qubits.
Extensions to quantum volume that generalize from square circuits to rectangular circuits with different
scaling of widths and depths have been proposed [320, 323].

M4.2: Mirrored circuits average polarization

A method that can be used to perform characterization of entire circuits without needing to perform expen-
sive classical simulations is mirror benchmarking [94, 112–115]. In order to avoid the classical simulations,
mirror benchmarking transforms the circuits so that their outputs are easy-to-predict and have simple mea-
surement outcomes. The main idea of mirror benchmarking involves appending the circuit of interest with
its inverse mirror copy, with a layer of random single qubit Pauli operations in between. Specific noise con-
tributions can cancel out when the inverse mirror copy is applied, thus a set of circuits with different random
Pauli operations must be used to ensure that these noise contributions are not cancelled out. The known
ideal output of the circuits involved in these benchmarking methods makes the results easily verifiable, hence
making the benchmarking methods scalable to large numbers of qubits. The quality metric is the average
polarization after circuit execution, which is a rescaled form of the success probability of circuit execution.
This method can be used with Clifford circuits as well as general non-Clifford circuits, although in the latter
case the mirrored circuit needs to modified in order to account for the layer of random Pauli operations [109,
113, 321].

M4.3: Algorithmic qubits

Ref [320] generalizes quantum volume and provides a framework for quantifying how well a device executes
circuits with different depths and widths. This is denoted as volumetric benchmarking. The algorithmic
qubits [324, 325] benchmarking method uses circuits corresponding to six chosen quantum algorithms and
combines the volumetric benchmarking results for those algorithms and outputs a single-number metric that
is obtained by finding the number of qubits at which all 6 benchmarks pass a specific success criteria.

M4.4: Upper bound on the variation distance

To evaluate the quality of noisy quantum circuit execution one may use quantum accreditation protocols [116,
117], which do not involve computationally expensive classical simulation of quantum circuits. The upper
bound on the variation distance is a metric within the accreditation protocol that bounds the distance be-
tween the probability distributions corresponding to a quantum circuit run on a noisy quantum computer
and its ideal noiseless counterpart.

Given the many possible types of circuits, there are a number of other potential circuit execution quality metrics.
One such example is cross-entropy benchmarking (XEB), which measures circuit execution quality by comparing
the outputs of random quantum circuits and then calculating how likely the measurement outcomes would be
if measured on an ideal noiseless quantum computer [326]. One key limitation of circuit execution benchmarks
such as quantum volume [109] and XEB [56, 98, 326] is the requirement of expensive classical simulations that
scale exponentially with the number of qubits. Thus, once quantum computers reach the threshold where the

12
2. Short descriptions of the comprehensive set of metrics and benchmarks

classical simulations become infeasible, other methods, which do not rely on exponentially-scaling simulations,
need to be used.

M5. Well-studied tasks execution quality metrics

Metrics that quantify the performance of a device in executing specific well-studied tasks can provide a view of
the overall capabilities of a device. Such tasks can include entire applications, algorithms, and subroutines of
applications. There are a number of articles that propose algorithm or application level benchmarking suites
for gate-based quantum computers [16, 99, 118–131]. Most such proposed benchmarking suites quantify the
performance of a quantum computer in solving specific algorithms, with the inputs being derived from target
applications. In Ref. [327] the authors present a database of qubit-based quantum Hamiltonians which can
be used to benchmark quantum computers. The main differences between the various proposed benchmarking
suites are both the choice of algorithms and of the metrics used to quantify the quality of outputs. The
algorithms that appear commonly in these suites include the Quantum Fourier Transform [118–122, 163, 328],
Grover’s search [118–120, 123–125, 329], the variational quantum algorithm (VQE) [99, 118–120, 124, 126–128,
330], the Bernstein-Vazirani algorithm [16, 118–120, 125, 129, 331], the Hidden-shift problem [16, 118, 119, 125,
129, 332], the Deutsch-Jozsa algorithm [118–120, 124], Shor’s period finding algorithm[118, 119, 124], Quantum
Phase Estimation [118–120, 123, 124, 202], the Quantum Approximate Optimization Algorithm (QAOA) [99,
119, 120, 122, 124, 130, 333], amplitude estimation [118, 119, 124], and quantum communications [131, 136, 334].
Other algorithms and applications that have been considered include quantum singular value transform [335],
error correcting circuits [336], encoding and decoding of cat states [337], and the Harrow–Hassidim–Loyd (HHL)
algorithm [338]. One may also consider general variational quantum machine learning algorithms, which may
be somewhat tolerant to noise in the hardware.
The fidelity between the output on an ideal noise-free quantum computer and the output on hardware is
commonly used as the metric to quantify performance, typically averaged over a set of chosen circuits. It re-
quires knowing the exact solution, and thus limits the scalability of these benchmarking methods [118]. One
may also restrict the tasks to problems that can be solved efficiently classically also for large system sizes.
However, performance of a device on simple problems or circuits may not be indicative of the performance of
the device for complicated problems or circuits [118]. Other kinds of metrics that have been considered include
runtime [128], largest circuit that can be run successfully [126, 128, 324], and application specific metrics [127,
130, 333]. In Ref. [99, 119] the authors compare the different algorithms used in benchmarking methods and
evaluate what aspects of device performance the methods measure.

Four of the well-studied tasks are considered as metrics in this category, and are described in what follows.

M5.1: Variational quantum eigensolver (VQE) metric

On near term devices the rather large error rates only allow short quantum circuits to run successfully. A
representative class of such circuits are those used within the VQE to prepare a wavefunction on the quan-
tum computer, which allows obtaining the ground state energy of a given Hamiltonian, and which is a key
task in many types of applications [330]. VQE is therefore widely used on current noisy devices, typically in
chemistry and materials science applications [42, 330, 339–344].

M5.2: Quantum approximate optimization algorithm (QAOA) metric

Another important type of applications for quantum computers is in the area of optimization. Representative
examples are combinatorial optimization problems, where one aims to find an optimal object out of a set
of objects. The quantum approximate optimization algorithm (QAOA) [345–347] targets such problems. It
can also be executed at low circuit depths, which makes it suitable for noisy devices, similarly to VQE. An
advantage of using QAOA for benchmarking is that it can be run using both gate-based and non-gate-based
quantum computing approaches, such as quantum annealing devices [347–349]. The so-called Q-Score [130]
is a representative example for quantifying the performance of a device in executing QAOA, and is used as
metric in this document.

M5.3: Fermi-Hubbard model simulation (FHMS) metric

The 1-D Fermi-Hubbard model is exactly solvable using the Bethe ansatz, and the outputs of methods in-
volving the simulation of that system can be verified [126]. The quality of the computation of this task on a
quantum computer therefore provides a scalable benchmark. Solution of such Hamiltonian simulation prob-
lems is part of the set of applications where quantum advantage may be found. It is challenging to run on
noisy devices, and fault tolerance may be required to achieve quantum advantage for Hamiltonian simulation.

M5.4: Quantum Fourier transform (QFT) metric

13
2. Short descriptions of the comprehensive set of metrics and benchmarks

The quantum Fourier transform (QFT) is an algorithm that forms a central role in many applications that
are expected to provide quantum advantage, for example for the factorization of integers within Shor’s
algorithm. The quality of the a QFT output on a device is an indicator for the performance of the device
for such applications. It typically requires larger circuit depths for its execution, so that it is challenging
to run successfully on noisy devices, and hence representative of the device performance towards quantum
advantage.

M6. Speed metrics

Device speeds can vary over orders of magnitude, depending on the device hardware platform [129]. Variations
in the speed over orders of magnitude can therefore make the difference between an algorithm finishing in a
day or within many years of runtime. Thus, the speed of operations is an important consideration for the
achievement of practical quantum advantage.
There are many contributions to the total time taken to run a particular algorithm on a quantum computer,
and the magnitude of the times taken for the individual contributions vary across hardware platforms. These
contributions include not only the actual time to run the quantum circuits, but also delay times between circuit
execution, compilation and other pre-processing times, data transfer times between control equipment, and
other similar times [137, 350].

One of these contributions is the time taken to run the quantum circuit itself, which can also be further
decomposed into the individual times taken to run specific operations that constitute the circuit. The provision
of these individual times therefore allows estimating runtimes for general circuits on a given hardware. These
contributions consist of the following.

M6.1: Time taken to execute a general single- or multi-qubit gate

Typically, the time to execute all quantum gates in a circuit forms the dominant contribution to the total
time taken to run the circuit. However, different quantum gates require different execution times, which
are set by the duration of a pulse sent to the hardware to execute the gate. Some single-qubit gates do not
need an actual pulse to be sent to the hardware, but can be performed virtually by modifying parameters
of subsequent gates, so that their execution time is effectively zero [233]. It is therefore useful to provide a
metric that averages timings over all single qubit gates. Such a general metric is the time taken to perform
a general single- or multi-qubit gate on the device, which effectively averages over all possible single- or
multi-qubit gates.

M6.2: Time to measure qubits

At the end of a quantum circuit, a subset of the qubits are measured in order to extract the results of the
circuit execution. The time taken to perform the measurement on each qubit is thus another contribution
to the total time taken to run the quantum circuit.

M6.3: Time to reset qubits

After the execution of a quantum circuit, typically the qubits need to be reset before the execution of the
next quantum circuit. This time to reset qubits contributes to the total time to run a set of quantum circuits.

M6.4: Overall device speed on reference tasks

In order to evaluate how fast a quantum computer is at running particular algorithms, one can directly
evaluate the time taken to run those algorithms [123, 128, 132, 133, 351]. Thus, the overall device speed on
reference tasks can provide estimates of device speeds in real-world usage scenarios.

By using the gate time metrics M6.1-3 one can estimate the time taken to run a set of quantum circuits. To
account for factors such as pre-processing times and data transfer times, rather than quantifying the contri-
butions individually one can quantify the total time taken to run specific quantum algorithms using M6.4. In
practice evaluating time metrics and using them for cross-platform comparisons can be challenging [118]. This
is due to the differences in the way timing information is reported by different hardware vendors, and also due
to the fact that often only limited, if any, timing information is provided.

One can also evaluate the rate at which a device executes layers of quantum computation [128, 137]. One
proposed metric for this is Circuit Layer Operations per Second (CLOPS) [137]. The CLOPS metric aims to not
only factor in the time taken to perform the individual operations mentioned above, but also other contributions
such as delays between circuits and classical processing times. Although such a single-number metric can be
useful in comparing the speeds of different devices, it may not always be indicative of the speed of the device

14
2. Short descriptions of the comprehensive set of metrics and benchmarks

in running particular algorithms because different algorithms can have different kinds of circuits or different
classical processing and feedback. For example, the VQE algorithm [330] involves a feedback loop between the
quantum and classical device, where the results obtained from the QC are processed classically to generate new
circuits. Such an algorithm would benefit more from a device with fast communication speeds than algorithms
which generate and submit all quantum circuits required for execution at once.

M7. Stability metrics

Qubit and device properties can change significantly over different time-scales [213, 216, 217, 223, 304, 352–364].
The stability over time of a device is therefore an important metric to quantify the reliability of the device as
well as confidence in the obtained metrics. The typically observed changes include random fluctuations around
an average value [213, 304, 353, 362], slow drifts of the average value over time [365–367], as well as sudden
abrupt changes of the values at random times [216, 223, 362, 363, 365, 366]. In what follows a number of
aspects that affect the methodologies to capture the different aspects of the stability of the device performance
are outlined.
The changes of metrics values over time typically exhibit one of the following types of behaviours:
1. Randomly distributed fluctuations around an average value [213, 304, 353, 362]: the values of a metric
fluctuate randomly around the average within a continuous distribution. The standard deviation can
provide a good metric for the stability in such cases, especially if the distribution is Gaussian.
2. Random fluctuations between discrete values [216, 223, 362, 363, 365, 366]: over a given time interval a
metric can fluctuate between a discrete set of values. In this case the resulting distribution over a time
interval is not a single continuous distribution, but consists of sharp peaks around the possible values of
the metric. While the standard deviation can still provide overall information on the fluctuations, a more
detailed description of the specific peaks in the distribution can be beneficial.
3. Drifts of the metrics over time [365–367]: the distributions of metrics over a given time-interval obtained
in points 1 and 2 above can further exhibit drifts over longer time-scales. While also such drifts can be
captured for a given time interval by reporting the standard deviation, it can be beneficial to provide
information on the detailed drifting behavior.
A key aspect in the metric fluctuations described above is the considered time scale. One can mainly distin-
guish three main types of time-scales:
1. Time scales significantly shorter than the circuit execution time: fluctuations of metrics that occur at
significantly shorter time scales than the circuit execution time are typically not measurable, and the
acquired metric value effectively averages over such fast fluctuations [304, 359, 368].
2. Time scales similar to the circuit execution time: for such time scales, the acquired measurements after
each execution of the circuit, denoted as a shot, can vary due to the variation of the metric from one circuit
execution to the next. When acquiring a metric over many, often thousands of shots, the resulting value
is an average over the underlying fluctuations of the metric between different circuit executions [304]. In
such cases, an evaluation of the metric with a smaller number of shots may be performed to obtain time-
resolved information on the fluctuations. Such information can provides insights on the noise sources in
the device and on how they affect the execution of quantum circuits. The drawback of a reduced number
of shots is a higher measurement uncertainty.
3. Time scales much longer than the circuit execution time: these can be accurately tracked by systematically
evaluating a metric at specific time intervals, which can be minutes, hours, days or months [217, 223, 352].
The type of time interval that needs to be considered varies depending on the aim. For example, for many
quantum algorithms it is assumed that the qubits remain stable while the algorithm is executed. Changes in
behavior over time can lead to worse algorithm performance. For example, if the over- or under-rotation error
changes significantly during execution of the minimization process in a variational quantum eigensolver (VQE),
convergence may not be achievable [330]. In this case the time scale to be considered is the duration of the
execution of the algorithm. For hardware manufacturers that aim to understand and eventually reduce the
noise in the devices, the evaluation of the fluctuations over all the time scales is important.
Since the times taken to evaluate some of the metrics can be large, it is important to report, together with
the value of each metric, the start and end times of its measurement, as that allows estimating how much the
metrics have fluctuated over time while measuring the metric.

M7.1: Standard deviation of a specified metric evaluated over a time interval

15
2. Short descriptions of the comprehensive set of metrics and benchmarks

A metric to capture the overall stability is the standard deviation of metrics evaluated over a specified time
interval. For all types of fluctuations this provides a first estimate of their magnitude. To obtain more
detailed insights into the types of instabilities of the metrics and of their origins, the methodologies to char-
acterize the stability of the device over time need to be adapted to the specific type of time-dependence of
the fluctuations.

M8-10. Non-gate-based quantum computer metrics

Benchmarking methods and metrics for non-gate-based quantum computers such as annealers, boson sampling
devices, and neutral atom quantum simulators are less developed than for gate model quantum computers.
The main reason for this is that each of these approaches has very specific requirements, which may even vary
significantly from one manufacturer to the next, so that it is more difficult to find a unique set of metrics that
applies to all. Furthermore, over many years the number of manufacturers for these methodologies was relatively
small, so that there was less incentive to develop benchmarks to allow for performance comparisons. In quantum
annealers, for example, for a long time there has been only one manufacturer of large scale quantum annealing
systems, D-Wave Systems Inc.1 . However, the landscape is now changing, and a number of companies are seeking
to provide similar analogue solvers, for example Pasqal2 , Qilimanjaro3 , Orca 4 and Quantum Computing Inc5 .
Therefore, there is a clear need for metrics for quantum annealers and other non-gate-based quantum computers.
Nevertheless, there is much less literature on metrics for such devices to draw from. Hence, the sections below
provide a general overview of the needs for metrics in these areas, and propose a number of metrics based on
existing work. The goal of these sections is to provide a starting point for understanding how to assess non-
gate-based devices, which can be used in the future development of well defined sets of metrics. We highlight
this as an important area for future work.

M8. Metrics for quantum annealers

While formally adiabatic quantum computation is a type of universal quantum computation [369], the annealers
that are currently available are non-universal machines. For such non-universal machines, there may not be a
single set of metrics that is applicable to all devices. As a result, metrics based on demonstrations of quantum
effects specifically tailored to the D-Wave devices [45, 370–372] are not usually portable to other manufacturers,
since it is unlikely that other devices have the controls available to perform comparable experiments. This
category instead includes metrics which assume only basic general annealing protocols. Another option for
assessing annealers is to directly measure their performance on optimization problems, such as for example
done for the QUARK framework6 , and for the Q-Score metric7 . Such measurements also allow for cross-
platform comparisons, which is explicitly the goal of the Q-Score [130, 373, 374] (metric M5.2). Ref. [375]
generalizes randomized benchmarking methods to programmable analogue quantum simulators, and Ref. [376]
proposes guidelines for reporting and analyzing performance of quantum annealers.
The purpose of this section is to present lower level characterization of the annealer devices. In what follows
we describe the metrics for annealers.

M8.1: Single qubit control errors

These are measures of the level of miss-specification of the problem on each qubit, and can be caused by
calibration errors or fundamental lack of precision in the device.

M8.2: Size of the largest mappable fully connected problem

This measures how large of a problem could be solved, assuming it is fully connected. This measure is
distinct from number of qubits, since many annealers may not allow arbitrary connectivity and therefore
incur overhead when more highly connected problems are mapped.

M8.3: Dimensionless sample temperature

This is a measure of the statistics of the distribution returned by the device. Lower temperature implies
lower energy states are found more often. This metric is built on the implicit assumption that the device is
operating in a regime where the states returned follow an approximately thermal distribution.

1 https://www.dwavesys.com/
2 https://www.pasqal.com/
3 https://www.qilimanjaro.tech/
4 https://orcacomputing.com/
5 https://www.quantumcomputinginc.com/
6 https://github.com/QUARK-framework/QUARK
7 https://atos.net/en/solutions/q-score

16
3. Discussion and outlook

M9. Metrics for boson sampling devices

Similar to other quantum computing technologies, boson sampling device performance characterisation can
range from the characterization of the hardware, as traditionally done in experimental labs, to quantification of
how complex it is to simulate the device operation using a classical computer. There are also metrics for photonic
quantum processors analogous to those for gate-based quantum computers, such as quantum volume [377].
These are the considered metrics.
M9.1: Hardware characterization and model as metrics
The characterisation of the individual components of a boson sampling device, such as linear-interferometers
and detectors, can allow estimating the overall performance of the device as a whole. These metrics are
chosen on a hardware-specific basis.

M9.2: Quantum advantage demonstration as metric

Quantifying how challenging it is to classically simulate sampling from the device provides a metric for
the overall capability of the boson sampling device. This is especially relevant in the context of quantum
advantage, as this metric effectively quantifies the distance to quantum advantage for the device.

M10. Metrics for neutral atoms devices

Neutral atoms offer the combined modalities of performing both digital gate operations and analogue computa-
tion in the form of coherent quantum annealing [19]. When used as a gate-based digital quantum computer, the
metrics presented for gate-based quantum computers all apply to the device. However, when used as an analogue
quantum simulator [378–381], there are other metrics that can be used to quantify the performance of the device.

M10.1: Analogue process fidelity

The analogue process fidelity measures the quality of the analogue computation by running quantum simu-
lation on the device and comparing the obtained results with the ideal results obtained via classical simula-
tion [382]. Note that this metric is not limited to neutral atom devices, but is applicable to a wide class of
analogue quantum simulators [382].

M10.2: Trap lifetime

Due to the relatively shallow trap depth of neutral atom tweezers, it is possible for atoms to be ejected
from the trap during computation. The trap lifetime metric is a measure of the characteristic lifetime of the
trap [383].

M10.3: Reconfigurable connectivity

Neutral atom quantum computers can also offer the ability to dynamically reconfigure the atom array ge-
ometry in order to implement arbitrary couplings. The number of mobile tweezers available and hardware
restrictions on the allowed moves limit the degree of reconfigurability. Thus, the reconfigurable connectivity
is an important architectural property that characterizes the capability.

3.Discussion and outlook

The aim of this document is to provide a comprehensive collection of metrics allowing the evaluation of all
aspects of quantum computing performance relevant on the pathway of achieving quantum advantage. The
included metrics measure performance going from the individual quantum computing components, mostly of
interest to hardware manufacturers, all the way up to well-studied tasks in applications, more relevant to end
users. For thorough benchmarking of a given quantum computing hardware system, it is crucial to evaluate
a comprehensive and holistic collection of metrics in order to determine advantages and disadvantages across
performance indicators and avoid focusing only on a subset of metrics that may lead to biased conclusions.
This compendium provides a consistent format for the definitions of all metrics in the collection, including the
methodology, assumptions and limitations, and a link to software implementing the methodology. The inclusion
of methodology and software in the metric description is needed to ensure transparency of the obtained metrics
values. When reporting a metric, inclusion of information on how well the assumptions are met by the specific
hardware provides trust in the results. The included limitations describe aspects such as how scalable a metric
is and where the metric may not be applicable. The linked open-source software, together with the associated
tutorials, is necessary to ensure the reproducibility of reported metrics values, as no parameters of the metrics

17
3. Discussion and outlook

evaluation are hidden. Each metric in the collection can be evaluated by running its associated software. The
software uses or is built on open software libraries with permissive licenses such as Qiskit [161] and PyGSTi [282].
The collection of metrics is aimed at a wide number of hardware platforms. Since the relevance of each metric for
a particular hardware platform varies, background information for a number of widely used hardware platforms
is provided including a description of the most relevant metrics for each platform. Platform-specific metrics
are typically also needed for non-gate-based approaches, and in this document we provide an initial collection
of such metrics. In these cases the metric description format is partly adapted to the specific systems, since
metrics and benchmarks for these systems are at the early stage of their development and may lead to to
more specific metrics in the future. We also suggest that, when reporting metrics, all parameters set in the
measurement procedure are reported. Additionally, in the case of metrics containing randomized circuits, one
should also report the circuits executed to obtain the metric. If classical optimization is used, the user should
then also report the un-optimized and classically optimized circuits, along with all other metric parameters,
and the metric value.
Extensive research in the development of metrics and benchmarks has led to a number of well established
and mature methodologies. Nevertheless, due to the rapid advancement of quantum hardware and algorithms,
research is still needed and is rapidly progressing in a number of areas, such as the following:
• Significant research is needed to improve efficiency with which a number of metrics can be evaluated.
As discussed in the Sec. 2, some metrics can take significantly longer to evaluate than others. Metrics
that take very long to evaluate can only measure properties averaged over such long time scales, and
are therefore not suitable for the evaluation of potentially occurring fast changes in device properties
and performance. Speeding up the evaluation time of metrics is therefore an important area of research,
as it will allow gaining significantly more information on the time-dependent fluctuations of hardware
performance and with it enable fast on-the-fly re-calibration.
• Research is also needed to improve the scalability of a number of metrics. As the number of qubits
increases, it will become progressively more important to establish metrics that can scale-up to thousands
of qubits and beyond. Many existing metrics do not scale to such large number of qubits, often because they
require a classical computation of the exact solution, which becomes intractable for large qubit numbers.
Ideally, the computational resources demanded by the benchmark should grow at most polynomially with
increasing quantum computer size. Furthermore, when scaling up quantum computers to thousands of
qubits, some component level metrics may start to affect the performance more than others, which may
result in the development of new metrics tailored to such scaled-up systems.
• A significant amount of past research was devoted to the development of quality metrics, while speed
metrics and stability metrics are much less developed. Therefore, research into these metrics categories
will be important. Development of such benchmarks will also guide the development of quantum computer
characterization techniques aimed to improve the hardware performance in these benchmarks.
• It will be important to provide metrics that characterize specific aspects of the occurring errors, such as
correlated errors, cross-talk errors, or non-Markovian noise. This will allow optimizing quantum error
mitigation and error correction techniques for these hardware errors, as well as guide hardware design to
reduce them.
• It will be important to further develop metrics devoid of any noise model assumptions, accommodating
all potential noise disturbances. In many of the currently used metrics there are a number of assumptions
regarding the impact of noise on computations. For example, for the evaluation of gate fidelities it may
be implicitly assumed that there is no cross-talk, or more generally no non-Markovian noise.
• Research is required into benchmarks for quantum software, especially low level compilers and optimizers,
as well as error correction algorithms
• Another area for research is the development of metrics for non-gate-based quantum computing ap-
proaches, which are typically hardware and application specific and also difficult to directly compare
to gate-based approaches, and are hence difficult to communicate to non-experts. A resulting risk is that
these may not be considered promising approaches compared to their gate-based counterparts. It is also
important to develop metrics for combined gate-based and non-gate based algorithms.
• It will be important to advance the development of metrics for distributed quantum computers [384–
390]. These offer a solution to the challenge of scaling devices to a large number of qubits by using
a network of quantum computers which can communicate with each other, and hence effectively form
a larger quantum combined computer. One of the primary challenges with such devices is transmitting
entanglement across different quantum computers. Typically, this is performed by the use of entangled Bell
pairs [386]. Various metrics that measure the rate and fidelity at which these Bell pairs can be generated

18
3. Discussion and outlook

and transmitted have been proposed [391–394], as well as other metrics that measure the performance
of such quantum networks [386, 395–397]. Such metrics, in conjunction with the metrics quantifying the
performance of individual quantum computers, and with application-level metrics that are agnostic to
hardware implementations, can be used for thorough evaluation of the performance of such networked
quantum computers.

• With the aim of developing fault-tolerant (FT) quantum computers [23, 148–158], there is the need to
develop metrics that guide and track the progress towards that goal. Most of the metrics presented in
this document are largely transferable to logical qubits, for example the various gate fidelities and gate
times apply also to logical qubits. New metrics are needed that specifically evaluate metrics relevant to
quantum error correction (QEC). This can include, for example, time for each QEC cycle [148, 150, 155,
158], amount of change in error probability per gate provided by QEC when compared to physical error
rates [149, 150, 155, 156], and error-decoding latencies [155]. Given the large scale required for these
devices, metrics would also be needed to quantify the performance of error correction implemented on
networked quantum computers. The combination of metrics for the physical qubits, QEC capabilities,
and logical qubits can then together be used in developing intermediate milestones towards the FT era.

Since the end user is typically not familiar with the details of hardware platform and algorithms, from an end
user perspective there are a number of properties that should be considered for metrics and benchmarks:
• Hardware agnostic: For the end user to be able to objectively compare across hardware platforms, the
results of benchmarks aimed at end users should remain unaffected by any provider’s specific operations
or instructions, and may be framed to ensure this.

• Algorithm agnostic and future proof: when providing algorithm level benchmarks it is important to
develop performance indicators across a multitude of applications envisaged for the long term future
including those targeting quantum advantage.
• Predictive and certifiable: quantum benchmarks should also offer insights into the performance of quantum
computations and routines that are not explicitly included in the metric evaluation, and hence have a
predictive capability. In addition, there should be a level of assurance and a guarantee of accuracy.
Such paradigms have been recently introduced in Ref. [398].
As research on metrics and benchmarks progresses, the collection of metrics for performance evaluation is
expected to evolve. Hence, this document and its associated software are envisaged at being a living online
resource, updated at regular intervals to account for community driven developments in the field.
While research about the above-mentioned aspects is important, many of the metrics, in particular for gate-
based approaches, have a high level of maturity and are widely used across platforms. Examples of such metrics
are the qubit relaxation time (T1 ), the qubit dephasing time (T2 ), or the randomized benchmarking average gate
fidelity. Systematic laboratory inter-comparison studies for each of these metrics, where a quantum computing
chip is characterized independently by different laboratories either by remote or direct access to the device, can
be a useful step in the direction of achieving agreement of the community on the methodologies to evaluate these
well established metrics. This requires identification and agreement on a set of metrics that can be considered
mature, of relevance for multiple platforms, and widely used. These should be separated from metrics and
benchmarks that may still significantly evolve due to future progress of hardware platforms, and where it would
be too early to define an agreed methodology to measure them. Analogously, many benchmarks for individual
quantum computing components are also still rapidly evolving; this may for example apply to qubit couplers or
interconnects. In these cases, it is likely too early to aim for an internationally agreed approach.
A similar separation of benchmarks can be applied to the end user perspective: many applications where
quantum computers promise advantage are well-established and may hence be suitable for selection into an
agreed set of application or algorithm level benchmarks. These may include subroutines of full algorithms such as
Shor’s algorithm or Hamiltonian simulation, such as the quantum Fourier transform, quantum phase estimation,
and also the variational quantum eigensolver. One problem with such metrics is that even though a specific
small-scale quantum computer may perform well in these metrics, the quantum computer implementation may
not be scalable to the large systems needed for quantum advantage. As a result, care must be taken when
extrapolating these application level metric results for small or medium scale devices to abilities of future large
scale devices to achieve quantum advantage. Furthermore, many algorithms are being developed alongside the
hardware platforms, both for gate-based and non-gate-based approaches, and it is important not to exclude
these in the consideration of the best approach towards quantum advantage. Since algorithms are still heavily
evolving, algorithm level benchmarks need to be taken with care to avoid prematurely fixing the design of
machines optimized solely for the current known algorithms, potentially overlooking emergent solutions. When
benchmarking applications for quantum advantage it is also important to identify and optimize the most efficient
classical approaches.

19
3. Discussion and outlook

Independently of the specific metrics and methodologies, an international agreement may be sought to en-
sure trust in benchmarks by providing transparency and reproducibility. An initial step this direction may
involve agreements on which information to disclose together with metrics reports to ensure transparency and
reproducibility across different manufacturers, making sure that results can be objectively compared. In this
approach each manufacturer can still decide which metric to report, the benefit is that for the reported metrics
the values are well defined and trusted.
The importance of objective and relevant performance benchmarks for quantum computers has led to several
international standardization bodies initiating work for the evaluation of which specific areas are now ready for
standardization. As this may then inform funding decisions, care must be taken to ensure that standardization is
beneficial to the development of the field and facilitates work towards quantum advantage, while still providing
end users and investors an informed evaluation of the performance of a specific quantum computing platform.
Based on the discussion in the preceding paragraphs, listed below are some items that should be considered by
international standardization bodies:
• identification and agreement on the categories of metrics that comprehensively benchmark device perfor-
mance;

• identification and agreement on a set of well-established and mature metrics of relevance across multiple
hardware platforms that together benchmark the different aspects of performance such as quality, speed,
and stability,;
• identification of hardware platform specific metrics, including metrics for non-gate-based quantum com-
puters;

• inter-laboratory comparison studies for a set of such well-established metrics to develop best practice
guides on describing the measurement methodology;
• agreement on what data and software should be reported together with a metric value to ensure trust,
transparency and reproducibility.

A discussion may hence be held within international standardization bodies to identify the initially restricted
set of commonly agreed benchmarks that are expected to remain relevant even as the technology fully matures,
while leaving room for further development in metrics for areas that are still rapidly progressing. An agreement
may also be sought for metrics categories that need to be considered to ensure a comprehensive evaluation of
quantum computing performance.
An open question in the field is whether fault tolerant quantum computing based on quantum error correction
will be needed for practically useful quantum advantage. To date a number of different avenues for quantum
advantage have been considered: i) quantum advantage for specific tasks using gate-based devices without error
correction, ii) quantum advantage using fully error-corrected gate-based quantum computers, iii) quantum
advantage for specific tasks using gate-based devices with partial error correction, and iv) quantum advantage
on specific problems with non-gate-based quantum computers. While for all considered approaches in order
to achieve quantum advantage the noise levels need to be small, and speed of operations and device stability
high, the importance of individual metrics for the success the various approaches differs. Hence the device
improvements required to achieve quantum advantage depend on the quantum advantage approach that a
hardware manufacturer aims for. For example, reducing noise during quantum operations to very low levels is
essential if quantum advantage is to be gained without error correction, while increasing the number of low-noise
qubits by orders of magnitude alongside finding and implementing the most suitable error-correcting codes for
a device are the key priorities on the path to error-corrected quantum computers. As the goal of the field is to
achieve practical quantum advantage using one of the approaches above, one needs to evaluate a collection of
metrics to quantify qubit performance across all categories relevant for quantum advantage, such as the ones
presented in this document. For example, a commonly reported metric is the noise level when compared to
thresholds required for fault tolerance. However, for practical advantage it is also important to evaluate the
time take to perform the operations, and it is also important to consider if such a technology architecture is
scalable both from a hardware and control perspective, or whether it is stable over long time scales. A collection
of metrics, such as the one presented in this article, together with the associated open-source software needed to
evaluate them, will guide the development of standardized benchmarks for quantum computers for all of these
approaches and speed up the progress of the field towards practical quantum advantage.

20
4. Quantum computing approaches and their performance indicators

III.Background

4.Quantum computing approaches and their performance indicators

In this section the gate-based and non-gate-based models of quantum computation are described. The first
subsection presents concepts of quantum computing relevant for their performance. These include the types
of operations on qubits, the effects of noise in the hardware, quantum error correction, fault tolerance and
quantum advantage. This is followed by a detailed description of gate-based quantum computing. In the
subsequent subsections a number of non-gate-based approaches are presented: measurement based quantum
computation, quantum annealing, boson sampling.

4.1. Concepts for quantum computing performance

Universal quantum computer
The concept of a universal quantum computer builds on its classical counterpart. Computation is the trans-
formation of sequences of symbols by precise rules, and a computer is a physical device that performs these
transformations. Turing defined a class of abstract machines, now called Turing machines, that perform a
computation by manipulating symbols on a strip of tape according to a list of rules. A Turing machine that
can simulate an arbitrary Turing machine on arbitrary inputs is called a universal Turing machine. When
the transformations obey the laws of quantum mechanics, the physical device is called a quantum computer.
In 1985, Deutsch proposed a quantum Turing machine, which introduces the notion of a universal quantum
computer as a universal Turing machine augmented by unitary transformations [399]. A qubit is the unit of
quantum information encoded in a two-level system, and is the quantum analogon to the classical bit of infor-
mation. In this document both universal and non-universal quantum computing approaches are considered. A
non-universal quantum computer can only solve a restricted set of tasks, and it may provide an advantage over
classical computers for these tasks.

Models of quantum computation

In 1989, Deustch presented the quantum circuit model, a model of quantum computation equivalent to the
quantum Turing machine [400]. Circuit-based quantum computation is also known as gate-based quantum
computation, where a small and finite set of well defined operations, denoted as gates, are performed on the
quantum states in a quantum computer. Gates are unitary transformations, produced by evolving a Hamiltonian
for a certain amount of time. In addition to the circuit model, several other models of quantum computation have
been proposed, such as measurement-based quantum computation, quantum annealing, and boson sampling.
These will be discussed in more detail in the next sections. A quantum device that implements a model of
quantum computation is called a quantum processing unit (QPU), in analogy to the central processing unit
(CPU) in classical computation. The metrics and benchmarking methods presented in this document are for
gate-based quantum computers, as well as for selected non-gate-based models.

Noise and errors in the physical realization of quantum computers

Quantum computing hardware platforms create qubits by manipulating physical systems with at least two
discrete energy levels. Operations on the state of such a qubit are physically realized using electromagnetic
pulses on the qubit [2, 401]. Individual discrete operations on qubits are denoted as gates. The pulse can be
tuned by altering the pulse shape, frequency, amplitude, and duration to implement the gate that is required [2,
402]. At the hardware level, a quantum circuit is compiled into a pulse sequence that generates the evolution
of the states of the qubits as described by the quantum circuit [2].
Any quantum system seeking to realize a physical quantum computer is subject to disturbances from its
surrounding environment. These result in the quantum computer producing erroneous outputs with some
probability. Errors also result from inaccuracies in the physical implementation of the qubit operations, which
are continuous and inherently analogue operations. Errors from both these sources are generally referred to
as noise. Noise can accumulate in the course of a long computation, rendering its outputs unreliable. The
coherence of a qubit refers to the qubit’s ability to preserve the quantum information in the face of noise [403],
and the coherence time provides an estimate for how long a qubit can retain phase information [404] and hence
used to perform computations.

21
4. Quantum computing approaches and their performance indicators

Noisy intermediate-scale quantum (NISQ) devices

Currently available devices are typically referred to as noisy intermediate-scale quantum (NISQ) computers.
This term reflects the fact that quantum computers are affected by noise in the hardware, which for example
limits the coherence time of the states of the qubits. A number of metrics presented in this document can be
used to quantify how strongly the quantum computing hardware is affected by noise. The number of qubits in
the intermediate scale lies between very small quantum computers of significantly fewer than ten qubits and
large quantum computers with millions of qubits.

Quantum Error Correction (QEC)

Quantum error correction (QEC) is based on the use of multiple qubits and gates to detect and correct errors
that occur due to noise and imperfections in the hardware. This is achieved through redundancy provided by
the use of multiple qubits when storing and processing information.
In 1996 Shor proposed a scheme of error correction for the quantum circuit model of quantum computa-
tion [405]. Quantum error-correcting codes map so-called logical qubits, which are error corrected, into blocks
of physical qubits, such that a small number of errors in the physical qubits of any block has effects that can
be detected and corrected. This mapping is implemented by quantum circuits, where QEC involves correcting
errors in the physical qubits, as well as a means for computing with the encoded logical qubits. If these circuits
are implemented with slightly noisy gates, with only a small probability of errors occurring in the physical
qubits in each block over short time-scales, the encoded logical qubits are effectively not disturbed by noise.

Fault-tolerant quantum computing

Any physical implementation of QEC is subject to noise. If the noise is smaller than a threshold value, then
arbitrarily large quantum computations can be performed reliably by concatenating quantum error-corrected
circuits. This is referred to as the quantum threshold theorem, and results in fault-tolerant quantum comput-
ing [286, 406, 407]. Fault-tolerant quantum computation requires poly-logarithmically more resources. If the
noise is larger than the threshold, then errors accumulate at a rate faster than they can be corrected, resulting
in an unreliable output, and no fault-tolerance is achieved.
The threshold and resource overhead of fault-tolerant quantum computation depend on the model of quantum
computation, the actual computation at hand, the type of noise affecting it, and the quantum error-correcting
codes employed to counter them. Depending on these, estimates for the threshold noise levels range from values
corresponding to fidelities of 99% to 99.9999% [147, 408, 409]. Those for the overheads range from thousands
to millions of qubits and gates [148, 406, 408, 410, 411].
It is important to benchmark how far a given quantum hardware is away from being able to perform fault-
tolerant quantum computation. Such an ability depends on the fidelity of the gates, on the ability to perform
mid-circuit measurements to detect errors, and on the ability to apply measurement-dependent gates to correct
potentially found errors, which depends on the speed of measurement and processing of the measurement results.
The topology and connectivity of the qubits in the hardware determines which QEC codes can be efficiently
executed on such hardware. A number of metrics outlined in this document can be used to quantify these
aspects, and the combination of the information provided by such metrics characterizes how far away a specific
hardware platform is from being able to implement fault-tolerant quantum computation.

Quantum advantage
Quantum advantage is achieved when a quantum computer can solve a task faster than large conventional
high performance computers. Any limited, for example constant or polynomial speedup, can be considered a
quantum advantage. The notion of quantum supremacy is sometimes used either interchangeably for quantum
advantage, or to emphasize a large quantum advantage, such as an exponential rather than polynomial speedup.
Any claim of quantum advantage is always based on the comparison with the best classical algorithms
and hardware available at the time that claim is made. Given the always ongoing improvements in classical
computing hardware, and the development of novel algorithms implemented on them, the requirements on
quantum computers to achieve quantum advantage become more stringent with time. Two instances of claims
of quantum advantage with NISQ devices are those based on random circuit sampling [56, 57] using the gate-
based model of quantum computation, and those based on boson sampling [412]. Improvements on classical
algorithms have subsequently reduced the resources required to simulate such algorithms on classical computers,
and the race between improved classical algorithms and improved quantum computing hardware demonstrations
is actively developing [58, 413–417].

22
4. Quantum computing approaches and their performance indicators

|0⟩ X •

|0⟩ H

|0⟩ Ry (π) •

•
X Ry (π) H
X gate Ry (π) gate Hadamard gate CX gate Measurement
Figure 4.2.1.: Example circuit diagram for a three-qubit quantum circuit. Each horizontal wire represents a qubit,
and the initial state is shown at the left end of the circuit. Symbols and boxes represent X, Ry , Hadamard and CX
gates, with measurements at the end of the circuit. The mathematical definitions for these gates and their operation
are described in section 6.1. Multi-qubit gates are drawn vertically across wires, with dots or other symbols placed on
the wires that participate in the multi-qubit gate. For controlled gates like the CX gate, a solid dot denotes the control
qubit and a vertically connected symbol denotes the target qubit (⊕ for CX ).

4.2. Gate-based quantum computing

Gate-based quantum computing is also known as circuit-based quantum computing. In this model, qubits are
described by complex vectors, and quantum operations on qubits are denoted as quantum logic gates, which
correspond to unitary matrices. Quantum circuits are constructed by applying a sequence of gates to qubits, in
an analogous way to how classical circuits are constructed by classical logic gates. First proposed by Deustch
in 1989 [400], this model is used in algorithms like the Shor’s algorithm for prime factorization [328] and in
variational quantum eigensolvers [330].
As opposed to classical bits, which can be in either 0 or 1 but not both at the same time, qubits can be in
a quantum superposition of orthonormal basis states |0⟩ = [ 10 ] and |1⟩ = [ 01 ]. With regard to the quantized
energy levels of a quantum system, it is a convention that |0⟩ represents the lowest-energy (ground) state and
|1⟩ represents the first excited state. An arbitrary state of a qubit is written as |ψ⟩ = α |0⟩ + β |1⟩, where α and
β are complex numbers, and |α|2 + |β|2 = 1. A measurement of the qubit state in the |0⟩ and |1⟩ basis causes the
qubit’s wavefunction to collapse, outputting |0⟩ with probability |α|2 , or outputting |1⟩ with probability |β|2 .
By repeatedly preparing a qubit in the same state and then measuring it, a statistic of measurement outcomes
is gathered, and one may estimate the values of |α| and |β|. The number of circuit executions to prepare the
state followed by measurements is usually referred to as shots.
A quantum circuit is a collection of quantum gates applied to an initial quantum state. In a quantum circuit
diagram, sequential applications of gates on the same qubit are connected with horizontal wires, and multi-
qubit gates are drawn with vertically spanning boxes or with vertical lines. Fig. 4.2.1 shows an example circuit
diagram for a three-qubit quantum circuit containing a number of gates.
If a set of quantum gates can be used to construct an equivalent of any other quantum gate, then this set
is called a universal gate set. √There are many universal gate sets, examples include {CNOT and all one-qubit
gates} [418] and {CNOT, H, S} [419]. The mathematical definitions for these gates and their operation are
described in section 6.1. To achieve universal quantum computation within the gate based model, a quantum
computer needs to physically implement a universal gate set.
There are different technologies of realizing gate based quantum computing, examples including supercon-
ducting qubits, trapped ions, neutral atoms and photonic devices. An overview of these technologies is presented
in section 5. As a result, different quantum hardware may have different gates that are physically implemented,
known as native gates. Other quantum gates, not in the set of native gates, need to be decomposed into native
gates. Typically, the native gates are chosen to form a universal gate set, so that universal quantum computation
can be achieved.

4.3. Measurement based quantum computation

In measurement based quantum computation (MBQC) [420], instead of evolving a quantum state through a
circuit, one starts with a large entangled state, denoted as resource state, and performs the computation by
adaptively measuring the qubits, one by one with local single-qubit measurements. The main idea is that by
measuring one part of an entangled state, one can teleport the quantum information of one layer to the next layer
with an added gate applied. Since measurements on quantum states have fundamental randomness, in order to
recover the deterministic unitary evolution one needs to adaptively modify the measurements performed, based
on the previous measurement outcomes. It is possible to show that MBQC is complexity-wise equivalent to

23
4. Quantum computing approaches and their performance indicators

the circuit model, in that one can map a MBQC computation to a circuit one and visa-versa with a constant
overhead at most.
MBQC may offer a number of advantages for some applications. It may be easier to implement such a model
for some hardware approaches, such as photonic quantum computers [421]. MBQC is also the natural quantum
computation model for applications such as blind quantum computing [422, 423] and verifiable blind quantum
computing [134], which is a secure protocol for quantum computation whereby a server performs quantum
computation for a client while keeping all their inputs, outputs, and intermediate steps completely private. A
number of error-correction methods can also be adapted to MBQC. From a more foundational perspective, the
role that non-classical resources such as contextuality play for providing quantum advantage may be clearer in
the language of MBQC [424]. On the other hand, MBQC generally requires significantly more qubits for the
same algorithms. Additionally, the necessity of measurements in the middle of the computation, as well as the
need for adaptive measurements, put stringent limitations on using MBQC for certain hardware approaches.
Most digital devices can perform MBQC, so that most of the metrics described in this document are relevant,
but the order of importance and specifics vary when compared to the circuit model. Of relevance are the
following qualities: the ability to prepare high fidelity entangled states is important, the state preparation and
measurement (SPAM) errors need to be small, and the number of qubits needs to be large. Beyond these,
measures that quantify the performance of a hardware in running applications/algorithms are also relevant.
Finally, a crucial metric is the ability and quality of performing mid-circuit measurements, as well as the ability
to condition mid-circuit measurements on the outcomes of earlier measurements.

4.4. Quantum annealing

Quantum annealers constitute a class of devices which have been originally envisioned to solve combinatorial
optimization problems [47, 425] and related problems in machine learning. These devices rely on directly
mapping a problem to a Hamiltonian Function and utilize a variety of physical mechanisms to solve these
problems. Currently the most technologically mature and largest scale quantum annealers are those produced
by D-Wave Systems Inc. with over 5,000 superconducting flux qubits at the time of writing [426]. However, it
is worth distinguishing the computational paradigm of quantum annealing from the physical devices produced
by a single company. In fact, Ryderg-based quantum annealing has become commercially available through
the platforms of companies such as QuEra and PASQAL. Recent work has shown that, while not their original
intended purpose, the flux-qubit quantum annealers can also act as highly effective simulators of condensed
matter systems. In fact, while there is currently no evidence of a scaling advantage for exact optimisation,
there has been for approximate optimisation [427], and compelling evidence for an advantage over path-integral
quantum Monte Carlo, which is a classical algorithm (despite the potentially misleading name) used in the
simulation of certain condensed matter systems [428].
There are a number of reviews and perspectives on quantum annealing and related topics. A list of some of
them along with a short summary can be found below:
• Mathematically focused review article on topics related to quantum annealing, particularly in the adiabatic
limit [429]

• Forward-looking perspective on the potential of new methods in quantum annealing, focusing strongly on
non-adiabatic methods [430]
• Perspective on hybrid quantum-classical algorithms in general, with considerable discussion on hybrid
methods for quantum annealing [431]
• Review of potential industrial applications of quantum annealing [432]

• Review of the mathematics related to quantum annealing [433]

In the simplest version of a typical implementation of quantum annealing, which is the one usually adopted
in commercial systems, the objective Hamiltonian is expressed as a classical Ising Hamiltonian,
n
X
HIsing = Jij Zi Zj + hi Zi , (4.4.1)
i,j=1

where the problem is encoded into the values of h and J, and Zi is a Pauli z matrix acting on site i. Since
the problem of finding the ground state of this Hamiltonian is NP-hard [434], any combinatorial optimization
problem can be stated in this form[435, 436]. Quantum computation can be achieved by adding fluctuations, the

24
4. Quantum computing approaches and their performance indicators

simplest version of which being single-qubit flipping operations, so that one obtains the annealer Hamiltonian

Hanneal = A(t)Hdrive + B(t)HIsing , (4.4.2)

Xn
Hdrive = − Xi , (4.4.3)
i=1

where Xi is a Pauli x matrix acting on site i. Initializing in an easy-to-prepare state, and using appropriate
controls on the time dependent parameters A(t) and B(t), protocols can be constructed to find highly optimal
states of HIsing . The conceptually simplest version of this protocol is to initialize in the ground state of Hdrive ,
and then slowly reduce A and increase B, relying on the adiabatic theorem of quantum mechanics to reach
the ground state of HIsing [437]. There are however numerous other mechanisms which could potentially aid
a quantum annealer in solving problems. For faster protocols, either a mechanism related to energy conserva-
tion [438–440] or a generalization of the adiabatic theorem may play a role [430]. For some devices, thermal
dissipation may also aid computation, and unlike the previous two mechanisms, can do so regardless of the state
in which the system was initialized.
Especially in older parts of the literature, adiabatic quantum computation is sometimes used as a synonym
for quantum annealing. However adiabatic quantum computing is more properly considered as a subset of
annealing protocols, namely those where the adiabatic theorem approximately holds. In practice, assuming NP-
hard optimisation problems are not solvable in polynomial time on an annealer (as is widely suspected [438]),
then adiabatic quantum computing would require exponentially growing coherence times, which are unlikely
to be practically achievable. However, more generally quantum annealing could be performed with a mildly
growing runtime (and therefore required coherence time), but sampled many times [438, 441]. Since the theory
of adiabatic quantum computation is much more mature than general annealing, more can be proven in that
setting [429].
It is important to note that a quantum annealer operating as previously described is not capable of universal
quantum computing, because it can only implement Hamiltonians with negative off-diagonal elements, known
as stoquastic Hamiltonians. These Hamiltonians cannot capture important quantum phenomena in quantum
physics, such as the exchange statistics of electrons. However, quantum annealing can be made universal with
a slight generalization of Eq. (4.4.3) and introducing time dependent control of h and J in Eq. (4.4.1). If in
addition to local control, two-body terms are added to the driver, in other words
X X
Hdrive = − ∆i Xi + Kij Xi Xj , (4.4.4)
i ij

with parameters ∆i and Kij , then quantum annealing becomes a fully universal model of quantum computa-
tion [442–444], able to in principle perform any computation which can be performed on a gate based machine.
It is worth noting that the one aspect where quantum annealing is lacking in comparison to gate based quantum
computing is that fault tolerance has not been proven to be possible within this model.
The method of mapping optimization or related problems in an analog way and solving them using natural
dynamics of a quantum system potentially extends beyond quantum annealing, but it is likely to be worth
classifying such related models similarly to quantum annealing. One example is a class of optical devices known
as coherent Ising machines, which use non-linear optics and feedback to achieve this type of computation [445,
446] (and the related but distinct paradigm of entropy computing [447]). Due to experimental limitations,
currently all large-scale coherent Ising machines involve feedback based on classical measurements taken directly
in the basis which is used to encode the solutions. Such measurements by definition destroy any multi-qubit
quantum coherence. In principle this feedback could be performed with delay lines which would maintain
coherence, but this is a substantial engineering challenge. Furthermore, this shows the possibility for other
techniques, which follow similar operating principles to quantum annealing, to emerge. For this reason it is
appropriate to include the possibility of such devices when discussing the future of quantum annealing.

4.5. Boson sampling

Boson sampling is an umbrella term for proposals of non-universal quantum computing photonic devices which
take advantage of the bosonic statistics of photons. The name originates from the first proposal to demonstrate
quantum advantage by Aaronson and Arkhipov in 2010 [49]. Since the main motivation of boson sampling is
based on achieving quantum advantage for a specific task rather than on solving practical applications, in what
follows we provide a more extended overview of this method and eventually connect it to potential practical
applications. The difficulty to link boson sampling to practically useful applications is at difference to the
annealing approach presented in the previous section, where the potential for practical applications is the core
motivation, while provable quantum advantage is difficult to demonstrate.

25
4. Quantum computing approaches and their performance indicators

Current near-term quantum computing implementations based on quantum optics, similarly as for other
platforms, can be decomposed in three stages: (i) initialization of quantum state; (ii) quantum evolution; (iii)
measurement at the end of the circuit. It typically consists of three layers of experimental hardware: a set of
parallel sources of quantum states of light, followed by a linear-optic interferometer, and ending with a set of
measurements of each output mode. The different variants usually correspond to changing either the sources
at the input or the measurements at the output, all sharing the intermediate stage of linear-interferometry.
The technology and design of the linear-interferometer can vary, the most commonly used types are integrated
circuits, free-space optical experiments and fiber-optics. In the case of quantum optics, the quantum information
is encoded into quantum states spanning over multiple modes, but there are other physical systems that can
be modeled as harmonic oscillators, and that can therefore be used to implement the same boson sampling
experiments. Examples include exploiting the vibration of ions in an ion trap [448], and the microwave resonators
in superconducting circuits [449]. Different near-term quantum optics differentiate themselves via the choice
of the quantum states or hardware technology used on each of the three stages of the implementation. As we
discuss below the most common initialization is either single-photon sources or squeezed vacuum states, where
the measurement are divided into photon number resolving detectors and photo detectors that can only detect
the presence of photons but not count them. The quantum evolution usually consists in a linear-interferometer
that can be implemented either on free-space, fiber-optics or integrated chips.

Main proposals
The first theoretical proposal of a boson sampling device was by Aaronson and Arkhipov in 2010, which was
followed by a series of alternative proposals [51].
Boson Sampling The original proposal for boson sampling consists in sending exactly N indistinguishable
photons over a linear-optic interferometer of M modes. A mode is a degree of freedom of the electromagnetic field
where photons can be located, this can be eigenmodes of its spatial distribution, but also polarization, angular
momentum, and any other alternative degree of freedom that is experimentally accessible and distinguishable
from the other modes. Ideal linear-interferometers are characterized by a unitary that transforms input to output
modes, and for boson sampling the unitary is chosen randomly from the uniform distribution of unitaries, also
denoted as Haar-random distributed unitaries. The output photons are detected at the output of the linear-
interferometer using photodetectors. The proposal requires a regime of few photons per mode M = O(N 2 ) in
order to guarantee that the probability of having two photons leaving at the same output port is negligible.
Then, the required detectors do not need to be able to count the number of photons, but only to discriminate
zero from one or more (photodetectors), which simplifies the experimental requirement.
Scattershot Boson Sampling Until very recently, most photon sources were probabilistic, making the genera-
tion of N indistinguishable photons in parallel impractical. To alleviate this experimental problem, scattershot
boson sampling was proposed in 2014 [50]. It uses M sources of an entangled two-mode states, one per input of
the linear-interferometer. Each two-mode entangled state has a pair of modes, the first (signal) is sent through
the interferometer, and the second (idler) is used to herald the presence of a single-photon in the signal mode.
Their photon number distribution being governed by a geometric decay, and by properly tuning the average
photon number of the two-mode entangled state one can ensure that most heralding detections correspond to
single photons in the signal modes and higher number of photons have negligible probability, while at the same
time being in a few photon regime of M = O(N 2 ) modes per heralded single-photon. In practice, scattershot
boson sampling can be seen as a version of the original boson sampling, where the input of the N photons is
randomized over the M possible input modes, instead of the first N as in the original proposal.
Gaussian Boson Sampling This proposal consists of measuring the photon number in each of M modes con-
taining a Gaussian state of light [48]. The same interference procedure is performed, but the initial states are
Gaussian rather than number states, that is they contain an uncertain number of photons but are squeezed
in a different way. A Gaussian state corresponds to the ground state of a Hamiltonian composed of quadratic
terms in the creation and annihilation operators of the M modes. The name comes from the fact that their
phase-space representation is a Gaussian distribution, therefore fully characterized by its first and second order
moments. Gaussian operation corresponds to time-evolution with a Hamiltonian also composed of quadratic
terms, which evolves a Gaussian state into another Gaussian state. The transformation corresponds to a simple
transformation of the phase-space, which can be tracked with simple matrix-matrix and matrix-vector multipli-
cations. It is known that any pure Gaussian state can be generated by a sequence of sources of squeezed states,
states given by a superposition of even number of photon of a specific form and obtained through parametric
down conversion processes [450], followed by a linear-interferometer and local displacements. The local displace-
ment being a local operation does not add any non-locality, and therefore is usually not used in proposals of
quantum advantage, but may play a role in some application of Gaussian boson sampling. See [450] for details
on Gaussian states.

26
4. Quantum computing approaches and their performance indicators

Hardness of Boson sampling

Boson sampling and its variants are used in a series of proposals for quantum advantage demonstration. The
idea behind those proposals is that classically simulating boson sampling devices for randomly selected inter-
ferometers has a computational complexity beyond the capability of classical devices. The intuition is that for
boson sampling a given output probability is related to a permanent of a submatrix of the unitary representing
the linear-interferometer. Because computing permanents of complex matrices is hard, with best known algo-
rithms running in exponential time on the number of photons [451], the scaling of its simulation is impractical
beyond a given threshold size. Similarly, in the case of Gaussian boson sampling, the output probability is given
by the a Haffnian, another combinatorial object similar to the permanent. In some special cases the Haffnian
resulting from a Gaussian boson sampling output reduces to a permanent, as in the case for scattershot boson
sampling.
Compared to the intuitive reasons for quantum advantage outlined above, the actual hardness proofs are
intricate, and involve technical concepts of computer science. The proof works by contradiction. It is shown
that if a classical computer could efficiently solve these sampling problems, it would imply the collapse of the
polynomial hierarchy of complexity classes to its 3rd level, which is widely believed not to be the case in a
similar way as computer scientists consider the conjecture N = N P to be extremely unlikely. The result is a
probabilistic statement, which means that it is proven to be hard on average over a distribution of randomly
selected circuits. For more information on quantum computational advantage see [452].
Hardness in practice The hardness proofs of boson sampling, but also for similar qubit based proposals, have
only limited robustness to small deviation from the output of an ideal circuit. This places current devices
with significant levels of errors and imperfections outside the regime of the hardness proofs. It is therefore not
excluded that classical algorithms that exploit the presence of noise in real implementations could efficiently
simulate current boson sampling devices.
Validation challenge Another challenge for quantum advantage demonstration based on the sampling of ran-
dom circuits is the fact that verifying that we are in the hardness regime is itself not computationally tractable.
For example, the cross-linear entropy benchmark (XEB) used for qubit random circuits [56] needs the compu-
tation of the output probabilities of the ideal circuit, which is supposed to be hard in the quantum advantage
regime. There is growing evidence that no efficient verification of quantum advantage using solely classical com-
putation is possible, or if possible then it would imply the existence of classical algorithms that can simulate the
device [453]. It is becoming more clear that to prove quantum advantage, a departure from sampling problems
is necessary. Focusing on problems where the solutions can be classically verified or provide a quantifiable
advantage for an end-user application seems a more promising and useful route. Despite this, random circuit
experiments remain an interesting area for testing the capabilities of quantum computing platforms, including
boson sampling devices, to compete against the performance of classical devices.

Implementations and imperfections

While small-scale proof-of-principle demonstration of boson sampling had already been done more than 10 years
ago [51], in the last years there were a number of experimental demonstrations of large-size boson sampling
devices [40, 412, 454]. These experiments managed to implement highly complex setups beyond 100s of modes
of light with different sets of sources, photon measurements and hardware for the linear-interferometer. The
quantum advantage claims made by some of these experiments have been contested [455, 456], and the debate
of whether they provide quantum advantage remains open. Nevertheless, they are remarkable experimental and
technological achievements. We also note that the best known classical algorithms for their simulation are not
yet scalable with system size.
Free-space demonstration The first large-scale boson sampling device demonstration was done at USTC in
China with a device named Jiuzhang 1 implementing a Gaussian boson sampling demonstration [454]. It was
composed of a single source generating 50 multiplexed squeezed states injected into a free-space interferometer
with 100 modes and detected at the output with 41 photon detections. The linear-interferometer was not re-
configurable, but had a random configuration chosen during its fabrication. This was later improved to Jiuzhang
2, with now 144 modes and up to 66 photodetections [412], followed by the very recent Jiuzhang 3 [59]. It has
been shown that Jiuzhang 1 and 2 and 3 in the regime of a high number of photons detection can be classically
simulated, and a recent report on the simulation of Jiuzhang 2 and 3 using a supercomputer and very intensive
computational resources has been published [456].
Fiber-optics platform Free-space interferometers, despite being an ideal way of limiting losses, are not a
scalable technology to build a quantum computer. The alternative of using re-configurable integrated chips
suffers from serious impact of losses as the size of the circuit is scaled up, which makes boson sampling not
scalable in such a platform. Therefore, an alternative approach using fiber-optical loops, initially proposed
in [457], has inspired a novel approach to boson sampling that was recently demonstrated by Xanadu, reaching
216 modes and detecting up to to 219 photons (one can have more than one photon per mode)[40]. The advantage

27
5. Hardware platform specific performance overview

of this setup is that it is also a building block for universal-quantum computation using continuous-variables
measurement-based quantum computation techniques. To date there is no classical algorithm challenging this
quantum advantage demonstration.
Imperfections The main imperfections that occur on a photonic platform used for boson sampling can be
divided into three types: losses, thermal noise, and distinguishability of photons. Losses result from the coupling
with unwanted modes on the propagating medium, such as optical fibers or waveguides of an integrated chip.
Also small imperfections in the coupling between elements of the setup, such as the coupling of the sources to
the linear-interferometer, generate losses. Detectors have two main types of errors, detection efficiency (false
negative detections) and dark counts (false positive detections), which can be modelled by losses and interaction
with a thermal bath, respectively. If one assumes that all imperfections are uniformly distributed among the
different components, one can show that using properties of the theory of Gaussian states and operations, the
circuit will be equivalent to an ideal circuit where all losses and thermal noise happen right before the detection
or after the sources, where one is free to choose the option that makes the theoretical analysis simpler. For
example, this can be used to show that a circuit with losses η changes the scaling of the classical simulation
of N single-photon boson sampling device from 2N to 2ηN . Losses therefore decrease the effective hardness of
boson sampling from an ideal circuit of size N to at least ηN . Ideal photons are indistinguishable particles, but
errors in the source can result in the generation of photons that are partially distinguishable, and this can be
modeled by appending additional modes to the ideal circuit to emulate those extra degrees of freedom. It can
be shown that this has a similar effect on the hardness of simulating the device. See Ref. [458] for more details.

Applications
One of the main motivations for manufacturing a quantum computing device is to use it for practical applications.
In the last years there was significant development and discussion of several applications that are native to
boson sampling devices, and that exploit mathematical connections between the problems of interest and the
theoretical description and statistics of the device, such as with applications to graph optimization, molecular
docking, graph similarity, point processes, and quantum chemistry [459].
Molecular vibronic spectra The vibration of the nuclei in a molecule are modeled by quantum oscillators
of bosonic nature. This provides a direct mathematical link between the theory of quantum optics and the
vibrations of molecules that can be exploited to design quantum algorithms. It was shown in Ref. [460] that
a relevant problem in spectroscopy of molecular vibronic spectra can be exactly mapped into the statistics of
Gaussian boson sampling, with the addition of local displacement. The sampling output is transformed into
an energy via a classical post-processing that generates the vibronic spectra of molecules under a harmonic
approximation [461]. The quantum advantage in this scenario is not fully convincing as there are classical
algorithms that can efficiently solve problems of similar size as those accessible to current experimental Gaussian
boson samplers [461].
Graph problems Many problems of practical relevance can be stated in terms of a graph. Dense subgraph
problems are an example of such graphs, which search for subgraphs that contain many connections between
their nodes. These subgraphs may correspond to communities in social networks, correlated assets in a market,
or mutually influential proteins in a biological network. Interestingly, Gaussian boson sampling can be used to
solve that problem. By encoding a graph into a Gaussian boson sampling device, it samples dense subgraphs
with high probability [462]. Maximum clique, another well-known problem can be seen as a specific instance of
dense subgraph problem and can therefore be also addressed using a Gaussian boson sampler [463]. Up to today,
there is no study of the potential quantum advantage that these applications could bring, neither a benchmark
against classical solvers.

5.Hardware platform specific performance overview

Values for individual performance metrics of quantum computers often vary significantly for different hardware
platforms. Furthermore, there are metrics that may be relevant only for specific hardware platforms. Therefore,
in this section we give a brief overview of some of the widely used hardware platforms, outlining the type of
operation and the potential hardware specific aspects of performance metrics.

5.1. Superconducting qubits

Superconducting quantum circuits are one of the widely used platforms for the development of scalable quan-
tum computers. The main advantage of this technology is that it is based on electrical circuits that can be
fabricated with well-established methods used in the semiconductor industry. Since the properties of the qubit

28
5. Hardware platform specific performance overview

are mostly determined by the design, there are many ways to implement a quantum processor, thus giving
the designer a great amount of flexibility [5]. A large variety of different architectures, qubits and auxiliary
building blocks have therefore been proposed and evaluated over the years. Currently the transmon qubit is
the most used, but other qubit design such as fluxonium have also shown promise [2]. The flexibility also
means that whilst superconducting qubits are typically used for digital gate-based quantum computing, they
can also be used for quantum annealing or to implement various analogue quantum simulators. Another advan-
tage of superconducting quantum processors is that they are controlled and read-out using microwave signals
in the gigahertz frequency range. This means that the platform has greatly benefited from readily available
high-performance systems, instruments and components originally developed for the telecommunications sector.
Several manufacturers also offer products specifically developed to control superconducting quantum processors.
Since superconducting processors are microfabricated, the technology is, in principle, easy to scale, and
systems with hundreds of qubits have been demonstrated [167]. However, this freedom does come at a cost: for
example, qubits based on trapped ions or other atomic platforms are inherently identical, but this is not the case
for superconducting qubits. Small differences in the electrical parameters due to variability in the fabrication
process of devices translates into differences between qubits, for example, the qubit frequency. The lack of precise
reproducibility therefore is a major issue for superconducting qubits. Another problem is that the coherence
properties of superconducting qubits are very sensitive to defects and impurities in the material [464]. Over the
last twenty years, much progress has been made, with qubit relaxation times improving by over three orders of
magnitude, going from hundreds of nanoseconds to almost a millisecond [465]. Despite the vast improvement,
the coherence times still limit the number of operations that can be performed within the qubit lifetime, since
a qubit operation typically takes on the order of 100 ns. This limits the use of superconducting qubits to
rather shallow quantum circuits in the current noisy intermediate-scale quantum (NISQ) era. The defects and
impurities are usually collectively referred to as two-level systems (TLSs). As well as causing limitations to
coherence times, they also cause the qubit parameters to change over timescales ranging from minutes to hours
or even days, which means that frequent system re-calibration is often needed.
A further challenge is that superconducting qubits must be operated at millikelvin temperatures. This means
that the quantum processing unit must be situated at the bottom of a dilution refrigerator. Modern cryogenics
can reliable cool down even very large systems, and it is believed that systems with a thousand qubits should be
feasible using current methods [466]. However, transferring data to and from the processor is still a challenge,
since it can require hundreds of microwave lines that all need to be cooled. Future systems will require methods
for solving this bottleneck, possibly in combination with placing specially developed cryogenic control electronics
inside the cryostat near the processor itself.
There are many ways to connect superconducting qubits together into a processor. The most straightforward
method is to connect each qubit to its nearest neighbour via a fixed or tunable coupler [1, 2]. This method
restricts connectivity and can slow down certain algorithms [129]; recently major efforts have therefore gone
into enabling three-dimensional connectivity to facilitate communication between distant qubits [467–471].
Superconducting quantum computing has been very successful in the current NISQ era, with a demonstrated
one-qubit Clifford randomized benchmarking gate error of 0.0008 and a two-qubit interleaved Clifford ran-
domized benchmarking gate error of 0.024 [472]. Further improvements will be needed to reliably meet the
performance threshold for error corrected logical qubits using QEC codes such as the surface code. Recently,
much effort has therefore gone into developing and demonstrating new types of architectures and methods,
which allow for error mitigation or error detection or both [473]. This can for example be done by encoding the
information not in individual qubits, but in larger system comprising multiple qubits or microwave resonators.
These methods aim to enable the creation of logical qubits with much less overhead than would be possible by
directly using individual physical qubits.

5.2. Trapped ions

Systems of trapped ions have long been seen as a leading contender for the building of viable quantum computers
due to the high degree of precise control over both their internal and external states that has been possible
for decades at this point [474]. The electronic structure of ions allows for qubits that are well isolated from
the environment, as well as a simple readout mechanism based on laser induced fluorescence. Since ions are
charged, when trapped in the same potential well they naturally form crystals allowing registers of spatially
distinguishable but otherwise identical ions to be easily formed. The different vibrational normal modes of these
crystals then provides a shared property that allows controlled interaction between normally isolated qubits,
allowing for the implementation of multi-qubit gates.
Trapped ion quantum computers are almost always based on the radio-frequency (RF) Paul trap [474],
although Penning traps have been used to very successfully demonstrate quantum simulations in large 2D
ion crystals [475], and there is ongoing work in using arrays of Penning traps for quantum computation and
simulation [476]. The simplest RF traps for quantum computation consist of macroscopic electrodes, which

29
5. Hardware platform specific performance overview

allow a single linear crystal of ions to be formed. However, there are limits to how many ions can be held in a
single potential while still having the precise control required for quantum computation.
To be able to scale to large numbers of ions, ions need to be distributed among a number of trapping potentials,
in a so-called quantum charged-coupled device (QCCD). Microfabrication techniques allow traps to be built with
the large numbers of electrodes required for creating and controlling multiple potential wells, as well as features
such as junctions which allows ions to be re-ordered and re-positioned arbitrarily. Such microfabricated trapping
devices can be divided into two categories: two- or three-dimensional devices. Three-dimensional devices [477,
478] are similar to the macroscopic trap described in terms of their configuration of RF electrodes, and allow
strong confinement of the ions. Two-dimensional surface-electrode traps move the trapping electrodes onto a
two dimensional plane to produce trapping potentials above the surface of the device [479]. Such an electrode
configuration produces weaker confinement compared to a three-dimensional device of similar size. However,
fabrication is more straightforward, and it offers potentially easier integration of optics, detection elements, and
control electronics into the device [9]. Depending on the height of the ions above the ion-trap device’s surface,
cryogenic cooling may be required to reduce motional heating rates to an acceptable rate.
Ions used in trapped ion quantum computers tend to be alkaline earth metals, or other elements that have
a single outer-shell electron when singly-ionized. Such ions have a strong dipole-allowed transition permitting
easy laser cooling, and often possess long-lived low-lying D states. In addition, isotopes with an odd number of
nucleons possess hyperfine structure. There are many possible ways to embody a qubit in an ion, but the two
most common are hyperfine qubits, where a pair of states in the ground-state hyperfine manifold are picked, and
optical qubits, where one ground-state level is combined with a level in the D state. Hyperfine qubits formed
from states both with mF = 0 are extremely insensitive to magnetic fields, leading to extremely stable qubits
with coherence times of seconds or longer.
Optical qubits require extremely narrow linewidth lasers to drive quantum gates, with the coherence time
of the light providing an upper limit to the coherence time of the qubit. For hyperfine qubits, stimulated
Raman transitions can be used to drive two-qubit gates, removing the requirement for highly coherent laser
light. Single qubit rotations can be directly driven between hyperfine states by microwave pulses, allowing gates
to be performed without using lasers. However, free-space microwave photons have insufficient momentum to
drive two-qubit gates in isolation. By using near-field, oscillating magnetic fields generated in the structure of
a surface trap, a sufficiently strong mechanical effect can be obtained, allowing laser-free two-qubit gates to be
performed [260]. Microwave-field driven two-qubit gates can also be obtained if a static magnetic-field gradient
is applied at the ions’ positions [480].
In addition to the ions used for qubits, a second species of ion may be co-trapped to form mixed-species
crystals. These allow processes such as cooling and readout to be performed with sufficient spectral separation
as to leave the qubits unaffected by the laser light used to excite this auxiliary species. It is possible to mimic
this effect with a single species in the so-called ‘omg’ scheme, where optical and ground-state hyperfine qubits
are supplemented by qubits embodied entirely in the metastate D states, which can be used to again spectrally
hide the qubit state from laser light during cooling operations [481].
While the original Cirac-Zoller two-qubit gate proposal [12] kick-started the development of ion trap quantum
computing, current trapped ion quantum computers generally make use of variants of the Mølmer-Sørensen
gate [482], or of the closely related dynamic phase gate [483]. These two gates differ from the Cirac-Zoller gate
in that they do not require the motional mode to be cooled to the ground-state, instead working by applying
state-dependent classical forces to the ions to produce a state-dependent phase, and thus the two-qubit gate.
Such classical forces do not require the ions to be in the ground motional state, relaxing the cooling requirements.
All the required quantum computing primitives have been demonstrated with high-fidelities. Single and two-
qubit gate fidelities of 99.9999% [237] and 99.9% [218, 484] respectively have been demonstrated, as well as
single-qubit detection fidelities of 99.99% [313]. Note that the two-qubit gate fidelities are obtained using the
Bell-state preparation error, which acts as an estimate of the process fidelity [218, 484]. Performance in existing
trapped ion quantum processors is still impressive, with Quantinuum for instance reporting typical two-qubit
fidelities of approximately 99.8% in a system with 12 physical qubits [485].
The values of metrics directly related to quantum computing primitives such as gate fidelities depend on
lower level properties of the device. Hardware specific metrics may include: heating rates, including heating
due to shuttling or the splitting and combining of ion crystals; fidelities of conversion between different internal
qubit representations; and fidelities of photon-mediated remote ion-ion entanglement. In addition, the degree of
crosstalk experienced by spectator ions during the application of native gates is also important to be measured
and minimized.
The key challenge for trapped ion quantum computing is scaling to the large numbers of physical qubits
that are required to obtain even relatively small numbers of logical qubits, once the overhead of quantum
error correction is taken into account. While a QCCD approach can be used to go beyond the limitations of
a single chain of ions, as the number of ions increases, it is increasingly difficult to individually address ions
with light delivered externally from the chip. This is the motivator behind increased integration of optical
delivery into the chip, as well as a driver towards laser-free gates, again driven by electrodes integrated into

30
5. Hardware platform specific performance overview

the QCCD. To go beyond the number of ions that can be accommodated on a single QCCD device a number
of different approaches have been proposed. One is a modular approach of individual QCCD devices, with ion-
photon interconnects used to distributed entangled states between modules[486]. Another approach suggests
tiling individual QCCD devices and shuttling ions across the gaps between individual devices to produce a quasi-
monolithic system [401]. These approaches are not mutually exclusive - it is possible to imagine increasingly large
and powerful individual systems linked by photons, or the use of photonic interconnects to provide connectivity
between physically distant ions in a single monolithic device.

5.3. Neutral atoms

Neutral atom quantum computers have emerged as an exciting candidate for the development of scalable quan-
tum hardware by offering the advantages of large numbers of identical qubits, implementing parallel two-qubit
and multi-qubit gate operations, and the option to perform both digital and analogue quantum algorithms [378,
379].
Using individual atoms trapped in tightly focused optical tweezers, it is possible to create large qubit registers
in one [487], two [488] or three [489] dimensions, with arrays of over 1000 sites demonstrated [490]. Similar
to trapped ions, qubit properties are defined by atomic physics, with qubits encoded in the hyperfine-ground
states for the alkali atoms, or on optical clock transitions with alkaline-earth species, offering long coherence
times of up to 40s [491]. Single qubit gates can be implemented using microwave or optical fields, with Clifford
randomized benchmarking average gate errors below 10−4 demonstrated for large-scale systems [492].
To couple qubits together, atoms are excited to high principal quantum number Rydberg states, which
experience strong, long-range dipole-dipole interactions. For atoms sufficiently close together, typically less
than about 5 − 10 µm, the interactions prevent more than a single Rydberg excitation being created, leading to
an effect known as Rydberg blockade. This blockade can be exploited to implement high-fidelity two or three
qubit gate operations [493–495], with the state-of-the-art Bell-state fidelities exceeding 99.5% for two-qubit
controlled-Z gates [496].
Experimental progress advancing qubit number and gate fidelity has enabled first demonstrations of small-
scale algorithms [22], however combining these interactions with the ability to dynamically reconfigure qubits
using mobile tweezers, it is possible to implement arbitrary connectivity’s for creating cluster states or topological
couplings [19]. This provides a route to future implementation of constant-overhead fault-tolerance [497], further
enabled by recent demonstrations of mid-circuit state readout [498, 499].
Two limitations of the neutral atom hardware come from the susceptibility to loss during the qubit operations,
and leakage from the computational basis. Optical traps are typically on the order of millikelvin deep (compared
to Kelvin for ion traps) and the major source of atom loss is from collisions with residual background gases.
Typical vacuum lifetimes at room temperature are around 10 s, however this loss can be suppressed by moving
to cryogenic platforms offering lifetimes exceeding 10 minutes [383]. Leakage from the qubit basis is caused
by light-scattering during qubit operations or errors from finite blockade strength. For the alkali atoms, this
can be reduced by performing bias-preserving gate operations with the ability to repump atoms into the qubit
basis [500]. However, for the alkaline-earth atoms using clock-state qubit encodings, it is possible to implement
erasure conversion to eject atoms outside of the qubit basis and replace with new qubits from a reservoir
register [501].
A further challenge for neutral atom systems is achieving fast, high-fidelity readout. Typically this is per-
formed using fluorescence imaging over 1-10 ms timescales limited by the finite scattering rate, with state-
selection achieved by first ejecting atoms in |1⟩ from the trap. An alternative approach is to perform non-
destructive state-selection, demonstrated on 10 µs timescales for few atom systems coupled to optical cavities
or detected using single photon counters, recently implemented for 49 atoms using an sCMOS camera showing
routes to scalable readout [492]. Recent proposals for parallel readout using ancilla qubits provide a route to
reduce this to microsecond timescales [502, 503].
Beyond the use for gate-base quantum computing, the same neutral atom hardware provides a powerful tool
for quantum simulation [380], enabling studies of quantum magnetism and topological spin liquids [381], as well
as analogue quantum computing. The native Rydberg interactions can be used to encode classical optimization
problems such as the maximum independent set [504], as recently demonstrated on hundreds of atoms [505], with
further theoretical work extending the application to a broader range of problems including such as weighted
graph optimisation [506–508], QUBO problems [506], and integer factorisation [506, 509].

5.4. Photonic devices

The development of a photonic quantum computer is an attractive alternative to other approaches as it offers
the potential of room temperature operation and the promise of scalability. The advent of miniaturized linear

31
5. Hardware platform specific performance overview

optic circuits on photonic integrated circuits (PICs) has increased the interest in developing fault-tolerant pho-
tonic quantum computing based on the approach developed in the 2001 article of Knill, Laflamme, and Milburn
(KLM) [510], and also the Koashi, Yamamoto, and Imoto (KYI) 2001 article [511], which demonstrated polar-
ization entangled qubits. PICs have the advantage that they are easier to couple directly to fiber communication
networks and could form an integral part of quantum networks under development, where photonic qubits can
be exchanged between remote locations. This makes the existence of a distributed quantum computing resource
a viable possibility. Instead of PICs one may also use optical fibers [512], which build on commercially available
telecommunication components. This does not result in as compact a form factor as a PIC but the architecture
can be simpler, as the detectors do not have to share the same space as the linear optics and most of the
systems can be operated at room temperature. The utility of this approach is based on a high speed (GHz)
noise free optical memory [513], where two-photon off-resonant cascaded absorption (ORCA) can be used as
optical memory inside a hollow core fiber to allow compatibility with telecom components.
An alternative to the KLM universal quantum computation with single-photons and linear optics proposal
or the KYI polarization entangled qubit approach is based on measurement-based quantum computation using
a Gaussian cluster states and photon-counting detectors [450]. This is a non-gate-based quantum computing
approach. A further alternative to deterministic unitary entangling gates as the base unit of quantum comput-
ing, which are difficult to implement in photonic systems, is fusion based photonic quantum computing [514].
This model uses entangling measurements referred to as “fusions”, which are enacted on the qubits produced
by entangled resource states, and which are comprized of low numbers of photons. It employs destructive
measurements of entanglement to enable fault-tolerant computation. This differentiates fusion based photonic
quantum computing from one-way photonic quantum computing and measurement-based quantum computing,
as fusion based can be considered to occupy the space between these other forms. This structure is scalable and
requires less classical processing support than other photonic quantum computing implementations.
There are several companies attempting to produce a universal quantum computing platform using photons,
each one championing their own variants of the methodologies. Some approaches to this are to incrementally
develop linear optics platforms realized on PICs. Recent articles have claimed fidelities as high as 99.69% [40,
515, 516]. A commercial system has claimed quantum computational advantage [40] using both Gaussian boson
Sampling (GBS) and squeezed states of light for differing modes to produce Gottesman, Kitaev and Preskill
(GKP) qubits. This hybrid methodology aims to produce a fault-tolerant, scalable architecture which seeks to
solve the probabilistic problems of successfully making a GKP qubit available on demand rather than having
to use a multiplexing approach with many GBS devices to produce a GKP qubit probabilistically [517].
As photons are not affected by the heat potential of room temperature, scalable quantum computing is offered
by this platform if the current cryogenic requirement for efficient single photon detection can be overcome. The
lack of interaction between photons removes crosstalk issues, but conversely means that there is no conditional
interaction as used by other platforms that can be used for a gate model. Instead, photonic approaches to
quantum computing must use measurement and quantum interference. This approach is by definition proba-
bilistic, hence requiring indistinguishability of the photons used for measurement. Currently, the single photon
detectors required for these systems must be operated at cryogenic temperatures, but these temperatures are
much higher than those needed for other hardware platforms.
The scalability of quantum computers is an essential attribute in the argument for their eventual utility.
A network to integrate remotely located quantum computers which can have different physical manifestations
of qubits, for example trapped ion qubits or superconducting qubits, allows the separate computing units to
operate as a single computing entity with a commensurate increase in performance. Photonic interconnects are
the main platform considered to achieve such networked quantum computers. The quantum networks currently
emerging use entangled photons, superposition and quantum measurement to allow qubits from one system to be
transmitted to another, either in free space [518] or through optical fiber [519, 520]. These systems can be very
simple and use a single prepare-and-measure state to transmit a qubit, or can be complex systems capable of
exchanging multiple qubits at high speed. The current issues with the low efficiency of transduction from matter
qubits, for quantum computing other than photonic based systems, to flying qubits to be transmitted though
the quantum network are under improvement and theoretical studies show that high efficiency is possible [521].
A quantum network can additionally be used for quantum key distribution (QKD) to transmit encryption
keys in a demonstrably secure manner. These systems also have application for long baseline telescopes [522],
secure cloud based computation [523], quantum enhanced measurement networks [524]. Currently the distance
between nodes of a quantum network is limited by the decoherence of the qubits unless reliable quantum
repeaters can be produced that support the end-to-end generation of quantum entanglement and thus allow
a maximally entangled Bell state to be used to continue transmitting the qubit. A reliable quantum network
will require well-defined single photon detection, entangled photon production, repeater technology and Bell
state measurement to ensure that maximally entangled states are maintained through the network via fiber
connections and other currently used telecom components such as Dense Wavelength Division Multiplexing
(DWDM). Existing fiber networks could be used, but technologies such as hollow core fiber show great promise
for low loss, transmission over a wide spectral range and low latency [525]. The latest iteration of this type

32
5. Hardware platform specific performance overview

of fiber is Double Nested Antiresonant Nodeless Fiber (DNANF) with less than 0.11 dB/km loss, which is the
lowest attenuation reported in an optical fiber [526].

5.5. Spin qubits

Spin-based qubits are gaining ground as an appealing platform for the realization of quantum computers. They
are primarily implemented in semiconductor nanoscale systems, even though the host material (e.g. silicon,
diamond, silicon carbide, germanium), the confinement architecture (e.g. crystal defect, dopant, quantum dot)
and the logical encoding (e.g. electronic spin, hole spin, singlet-triplet, exchange only) can vary significantly [31,
32]. A review of performance metrics for spin qubits in semiconductor nanostructures is given in Ref. [527].
Among the different types, those which have attracted most recent attention for their promise of scalability
through compatibility with commercial manufacturing methods, such as the Complementary Metal-Oxide-
Semiconductor (CMOS) technology, are based on electrostatically defined quantum dots (QDs) in silicon [33].
Lately, such qubits have been produced in increasingly large volumes in industrial foundries (e.g. at Intel, IBM,
LETI, IMEC) by adapting the well-established process used for conventional transistor technology nodes [528].
In fact, it has been demonstrated that mass-produced transistors realized in silicon-on-insulator substrates
can host spin-qubits when operated at cryogenic temperature [34, 529]. However, despite compatibility with
standard manufacturing methods, spin qubits are subject to large variability and lack of reproducibility in terms
of operation points, noise susceptibility and performances. This mostly stems from unavoidable atomic-scale
differences among nominally identical qubits due to the presence of random impurities in the material stack
or unintentional variations in the lithographic patterns. In order to improve the understanding and impact of
these reproducibility issues, extensive statistical studies on large numbers of samples are needed. Techniques to
carry this out efficiently at scale are actively being developed for the cryogenic environment [530–533].
Although spin qubits are typically operated at millikelvin temperature in dilution refrigerators, the temper-
ature constraints are much less stringent than for superconductor qubits. In fact, recent results have shown
that it is possible to operate transistor qubits at temperatures above 1K [529, 534]. The relaxed temperature
requirement sets promises for developing chips where the power-hungry control electronics are monolithically
integrated or heterogeneously co-located near the qubit layer thanks to the large cooling power available at the
few-Kelvin stage of pulse tube refrigerators [535, 536]. Having the control and readout electronics physically
close to the quantum processor could prove a step change towards solving the wiring bottleneck issues that have
so far limited scalability [537].
A distinctive advantage of spin qubits is that the host crystal can be designed to minimize sources of magnetic
noise. In fact, hyperfine interactions with nuclear spins in the host crystal do limit the coherence time. However,
for silicon systems the most abundant isotope (28Si: 92.23%) carries no nuclear spin, whilst the second most
abundant isotope (29Si: 4.67%) does. Through epitaxial growth of isotopically purified crystals, the relative
28Si content can be enhanced (>99.99%). This has led to orders of magnitude improvements in the coherence
times of qubit gates, achieving several milliseconds [538].
The highest average gate fidelities obtained by Clifford randomized benchmarking are up to 99.96% for single
qubit [366] and up to 99.81% for two qubits [539]. Gate times as fast as 2.5ns have been obtained for the
singlet-triplet encoding [540]. The fidelities for initialization and measurement can also be as high as 99% [539,
541].
Multi-qubit connectivity has proved challenging. Spin qubits in silicon have been primarily coupled via
nearest neighbour techniques. To this end, the exchange interaction can be controlled via dedicated electrodes
and is used to execute two-qubit gates between spins in adjacent QDs. The current state of the art is a linear
array of six QDs [542]. While it is important to go beyond nearest-neighbour connectivity, this is difficult
to achieve since the exponential decay of the exchange interaction is determined by that of the wavefunction.
One possibility is physical shuttling of the electron to a remote location [543]. Another approach is to couple
distant spins via an on-chip microwave cavity, employing an intrinsic or induced spin-orbit interaction [544].
However, these demonstrations have limited scaling prospects because they require control lines for every qubit,
eventually leading to impractically large footprints or interconnect bottlenecks. To solve these issues, one could
leverage the commonalities between silicon quantum devices and conventional integrated circuits. For example,
proposals currently under scrutiny suggest using a crossbar architecture [545, 546] based on combination of
row and column lines used to address qubit nodes on a grid, a solution similar to that adopted in classical
random-access memory technology.

33
6. Concepts and methodologies used for multiple metrics and benchmarks

6.Concepts and methodologies used for multiple metrics and

benchmarks
This section presents all the mathematical descriptions of the concepts introduced in the earlier sections that
are used for the definitions and methodologies of the metrics and benchmarks presented in Sec. IV. It first
presents the used mathematical definitions of qubit operations and noise descriptions. It then outlines volumetric
benchmarking, a general approach that is common across a number of benchmarks. This is followed by a
technical description of gate set tomography, which is used for a number of metrics in Sec. IV. It is noted
that the metrics in Sec. IV are presented in a self-contained approach, so that it is not necessary to read this
technical section before reading the metrics in Sec. IV. It rather provides all the technical and mathematical
background as reference. At the end of this section a list of symbols used throughout the document is given as
additional lookup reference.

6.1. Qubit states and operations in presence of noise

Noiseless qubit states and operations
In Sec. 4.2 an overview of gate-based quantum computing is given. Here the mathematical descriptions of
the gate operations is provided. Following on from the relations introduced in Sec. 4.2, ignoring a physically
irrelevant global phase one may write a general single qubit pure state as |ψ⟩ = cos (θ/2) |0⟩ + eiϕ sin (θ/2) |1⟩,
where θ and ϕ are real valued rotation angles. One may represent such a general qubit state as a vector on a
unit sphere, named the Bloch sphere, as depicted in Fig. 6.1.1(a). Some commonly used superposition states
are denoted as:
• |+⟩ = √1 (|0⟩ + |1⟩),
2

• |−⟩ = √1 (|0⟩ − |1⟩),

2

• |R⟩ = √1 (|0⟩ + i |1⟩),

2

• |L⟩ = √1 (|0⟩ − i |1⟩).

2

Of these four example states two are real state vectors (|+⟩ and |−⟩, as they only have real-valued
√ coefficients)
and two are complex state vectors (|R⟩ and |L⟩, as they have imaginary-valued coefficients i/ 2). In general, all
qubit states live in a complex vector space that is equipped with an inner product and is known as the Hilbert
space.

Figure 6.1.1.: Bloch sphere visualizations of (a) a general qubit state |ψ⟩ = cos (θ) |0⟩ + eiϕ sin (θ) |1⟩ as green arrow,
(b) an X gate applied to this |ψ⟩, leading to |ψ1 ⟩ = X |ψ⟩, and (c) an Ry (−π/2) gate applied to |ψ1 ⟩, leading to
|ψ2 ⟩ = Ry (−π/2) |ψ1 ⟩.

Quantum gate operations on qubits are described by unitary operators. Denoting a unitary operator by U ,
it evolves a state vector ψ into U |ψ⟩. Unitary operators preserve the length of a complex vector, so that single
qubit gates can be interpreted as rotations in the Bloch sphere representation. A widely used set of gates are
the so called Pauli gates X, Y and Z. These correspond to Pauli matrices σx , σy and σz , respectively, acting
on a qubit:
     
0 1 0 −i 1 0
X = σx = , Y = σy = , Z = σz = . (6.1.1)
1 0 i 0 0 −1

34
6. Concepts and methodologies used for multiple metrics and benchmarks

In the Bloch sphere representation, these gates correspond to a rotation of π around the x, y or z axes,
respectively. In particular, the X gate is also a quantum generalization of the classical logical NOT gate, since
it transforms as |0⟩ state to a |1⟩ state and vice versa. Arbitrary rotations by an angle θ around the three axes
are generated by matrix exponentiation of the corresponding Pauli matrices. For example, an arbitrary rotation
around the x-axis is given by Rx (θ) = exp[−iXθ/2] = cos(θ/2)I − i sin(θ/2)X, where I is the identity matrix.
Example Bloch sphere visualizations of the X and Ry (π/4) gates are shown in Fig. 6.1.1(b, c), respectively.
Other commonly used single qubit gates include the Hadamard gate, H, and the phase gate, S:
√
   
1 1 1 1 0
H=√ , S= Z= . (6.1.2)
2 1 −1 0 i

As described in Sec. 4.2, for a general qubit state |ψ⟩ = α |0⟩ + β |1⟩, with α and β complex numbers and
|α|2 + |β|2 = 1, a measurement of the qubit state in the |0⟩ and |1⟩ basis causes the qubit’s wavefunction to
collapse, outputting |0⟩ with probability |α|2 , or outputting |1⟩ with probability |β|2 . While measurements in
different bases of the qubit state are possible, the |0⟩ and |1⟩ basis is the most commonly used, and is referred
to as the computational basis.
Mathematically, a measurement involves projecting the state of the qubit to a particular basis by applying
a complete set ofP measurement projectors {Mj = |j⟩ ⟨j|} to the qubit state, so that the projectors sum to the
identity matrix, j Mj = I. An example is the measurement in the computational basis, where the set of
projectors is given by {M0 = |0⟩ ⟨0| , M1 = |1⟩ ⟨1|}. Because |0⟩ and |1⟩ are eigenstates of the Pauli-Z operator,
this measurement is also called a Z-basis measurement. It is sometimes useful to measure the qubit in different
bases. This can be done by adding a basis change operation before the computational basis measurement. For
example, to perform a measurement in the X basis, a Hadamard gate H is used before the measurement. To
measure in the Y basis, an S † gate followed by a H gate is applied before measurement [547]. These basis
change operations can also be used to prepare the qubit in the different bases. For example, the H gate can be
applied to the |0⟩ state to obtain the |+⟩ state.
The measurements of the qubit can be used to calculate expectation values of the different Pauli operators.
For example, the expectation value of the Pauli-Z operator, ⟨σz ⟩, is given by ⟨σz ⟩ = 1 − 2p1 , where p1 is the
probability of the measurement outcome being 1, corresponding to the eigenstate of the Pauli-Z operator with
eigenvalue −1. The values of ⟨σx ⟩ or ⟨σy ⟩ can be obtained using the appropriate basis change operations before
measurement.
For systems consisting of more than one qubit, states can be represented by tensor products. For example,
in a two qubit system, the two qubit state can be obtained from the tensor product of single qubit states:
 
    α1 α2
α α  α1 β2 
|ψ⟩ = |ψ1 ⟩ ⊗ |ψ2 ⟩ = 1 ⊗ 2 =  β1 α2 .
 (6.1.3)
β1 β2
β1 β2

For Nq qubits the dimension of the state vector becomes 2Nq , and hence grows rapidly with number of qubits.
Commonly used two-qubit gates include the swap gate, SWAP, the controlled-NOT gate, CX , also known as
the CNOT gate, and the general controlled-unitary gate CU :
     
1 0 0 0 1 0 0 0 1 0 0 0
0 0 1 0 0 1 0 0 0 1 0 0 
SWAP =  0 1 0 0 , CX = 0 0 0 1 , CU = 0 0 U00 U01  , (6.1.4)
    

0 0 0 1 0 0 1 0 0 0 U10 U11
 U00 U01 
where the block matrix U = U 10 U11
is a single-qubit unitary that acts on the target qubit only if the control
qubit is in the |1⟩ state. The SWAP gate exchanges the states of two qubits, and the C-U or CNOT gates
are often used to entangle two qubits. When qubits are entangled, their compound state can no longer be
represented by tensor products, and measurements performed on one qubit restricts possible measurement
outcomes on other entangled qubits. For example, applying a CX gate, with the second qubit as √target qubit,
to a two-qubit non-entangled state √12 (|0⟩ + |1⟩) ⊗ |0⟩ results in the entangled state (|00⟩ + |11⟩)/ 2. This state
cannot be written as a tensor product of single-qubit states, and measuring one of the qubit guarantees that the
other qubit must be measured in the same state, i.e., both are |0⟩ or both are |1⟩. The capability of quantum
computers to generate entangled states is one of the requirements for quantum advantage.
Typically, quantum computing hardware can only directly implement one- and two-qubit operations. How-
ever, for Nq qubits, any Nq -qubit operation can be decomposed into several one- and two-qubit gates by
mathematically decomposing the unitary matrix into products of smaller ones, enabling the execution of arbi-
trary Nq -qubit gates. Operations that apply to a subset of qubits are written as tensor products with identities

35
6. Concepts and methodologies used for multiple metrics and benchmarks

on remaining qubits. For example, in a four qubit system, an operation that applies an X gate on the first qubit
and a Y gate on the third qubit is written as X ⊗ I ⊗ Y ⊗ I, following a convention of qubit indices ordered
from left to right.
Often, random unitary gates are used as part of a quantum circuit [109, 288, 548]. When selecting unitary
gates at random, it is important to ensure that all possible unitary gates have an equal probability of being
selected. In order to ensure that this is the case, random unitary gates are selected from a probability distribution
over unitary operations, known as the Haar measure [549, 550].

Effects of noise
When there is some classical randomness in the state of a qubit, such as when the qubit can be in different
quantum states |ψi ⟩ with classical probabilities pi , then its state can
P be described using the so called density
matrix formalism. In this formalism the state is represented as ρ = i pi |ψi ⟩ ⟨ψi |, where ρ is called the density
matrix or the density operator of the qubit, and pi is the probability of state |ψi ⟩ to occur. If no classical
randomness is involved, a state is said to be a pure state with its density matrix ρ = |ψ⟩ ⟨ψ|, and it can be fully
described using its state vector |ψ⟩ alone. On the other hand, if classical randomness is present, then the state
is said to be a mixed state. In practice, mixed states often arise as a result of noise in the quantum system,
where the classical probabilities in ρ reflect loss of quantum information regarding the exact quantum state the
qubit is in. The density matrix formalism is therefore often used in quantum computer benchmarking methods,
since a number of these characterize the effects of noise on the quantum computation.
Given an arbitrary quantum state described by a density matrix ρ ∈ H, where H is an arbitrary Hilbert
space, ρ transforms as follows [163]:
ρ′ = Φ(ρ), (6.1.5)
where Φ is a quantum operation, also known as quantum process, that maps ρ to a new density matrix ρ′ .
Quantum operations can be used to describe both unitary gates and non-unitary noise processes in quantum
computers. A unitary gate operation is a special example of a possible quantum operation, and evolves a density
matrix ρ into U ρU † .
The quantum operation Φ must meet the following requirements in order to represent a physical process. For
any initial state of the system and environment, the transformed density matrix after applying the quantum
operation must be positive semidefinite. This means that the final density matrix must have non-negative
eigenvalues. This requirement is known as complete positivity (CP). Another common requirement of quantum
operations is that they must also be trace preserving, such that Tr[Φ(ρ)] = Tr[ρ]. This can be understood
as the conservation of probability [163]. When considering quantum operations on qubits, this requirement
may be relaxed in case of transitions outside the qubit subspace; in this scenario, the total probability cannot
increase [551].
By considering only the system and not the environment, one can write a CP quantum operation as
M
Ki ρKi† ,
X
Φ(ρ) = (6.1.6)
i

where M ≤ d2 , where d = 2Nq is the Hilbert space dimension and Nq is the number of qubits. This is called the
operator-sum or Kraus representation. The operators Ki are called operator elements or Kraus operators, and
they have the same dimension as ρ. These Kraus operators do not need to be unitary, Hermitian or invertible.
Additionally, the Kraus operators for a quantum operation are not necessarily unique [163]. If the quantum
operation is trace preserving, then i Ki† Ki = I, where I is the identity.
P
Quantum operations can also be represented in other ways. One widely used representation is the Pauli
transfer matrix form [103, 104]. It is calculated by expanding Φ in the N -qubit Pauli basis Pi ∈ {I, X, Y, Z}⊗N
such that [104]
Tr[Pi Φ(Pj )]
(SΦ )ij = . (6.1.7)
2Nq
Here, SΦ is the Pauli transfer matrix (PTM) of Φ.
Noise in quantum computers can be described as coherent or incoherent, or as a mixture of both [552]. Noise
can be described as coherent when only a single Kraus operator is required to describe the noise. Incoherent
noise is stochastic and requires more than one Kraus operator to describe the noise process [163]. Coherent
noise is noise that is defined by a quantum operation that is described by a single Kraus operator[552]. For
θ
example, an over-rotation error around the x-axis of angle θ has a Kraus operator of ei 2 X where X is a Pauli
operator. In the case of a noiseless ideal quantum gate, ΦIdeal , there is only a single Kraus operator, which is
given by the unitary matrix of the target gate, U , so that

ΦIdeal (ρ) = U ρU † . (6.1.8)

36
6. Concepts and methodologies used for multiple metrics and benchmarks

The Kraus operator representation of the quantum operation assumes that the noise is Markovian, which
corresponds to a memoryless environment. This means that the noise is assumed to be independent of time
during the application of the quantum operation. In physical qubits a number of sources of noise cause memory
effects. When such effects are present, one needs to consider non-Markovian noise descriptions, which can
account for memory effects arising due to the environment [223]. Coupling to coherent two level systems and
slow fluctuations of qubit frequency are examples of mechanisms that can lead to such memory effects. It is
therefore important to also consider benchmarks that characterize the amount of non-Markovianity of the noise.
In what follows we present a number of commonly occurring noise contributions with their associated Kraus
operators.
Amplitude and Phase Damping Amplitude damping describes energy dissipation from the qubit. For example,
in trapped ion qubits, amplitude damping occurs due to spontaneous emission of photons from the ion [553].
The Kraus operators for the amplitude damping channel are given by
   p 
AD 1 p 0 AD 0 γ AD
K0 = , K 1 = . (6.1.9)
0 1 − γ AD 0 0

where γ AD is the probability of decay from excited state to ground state. Here K1AD acts on the qubit and
induces a transition from |1⟩ to |0⟩. The operator K0AD describes how the state changes if there is no transition.
Phase damping noise, also called dephasing noise, describes the decay of off-diagonal elements in the qubit
density matrix. For example, in superconducting qubits, dephasing noise arises due to interaction with charge
[2, 223]. The Kraus operators for dephasing are represented by a phase-flip channel [163] and are given by
r r
PD γ PD PD γ PD
K0 = 1 − I, K1 = Z, (6.1.10)
2 2
where I and Z are the identity and Pauli Z operators. Here K1PD projects the qubit state onto the z-axis in the
Bloch sphere with probability γ PD , and K0PD does nothing with probability 1 − γ PD .
The Kraus operators for the combined amplitude and phase damping channel are given by [554],
 
AP 1 p 0
K0 = ,
0 1 − γ AD − (1 − γ AD )γ PD
 p 
0 γ AD
K1AP = ,
0 0
 
0 p 0
K2AP = (6.1.11)
0 (1 − γ AD )γ PD

One can also represent the amplitude and phase damping parameters in terms of the qubit relaxation time,
T1 (metric M2.1), and of the qubit dephasing time, T2 (metric M2.2), with the following relation [554]:

1 − γ AD = e−t/T1
q
(1 − γ AD )(1 − γ PD ) = e−t/T2 (6.1.12)

These qubit relaxation times are commonly used metrics to characterize the quality of a single qubit.
Depolarizing noise Depolarizing noise describes the noise process, where the qubit transforms to the maximally
mixed state with probability γ D , and does nothing with probability 1 − p [163]. This describes a noise process
of information in the qubit being lost. The Kraus operators for depolarizing noise are given by
r p p p
D 3γ D γDX γDY γDZ
K0 = 1 − I, K1D = , K2D = , K3D = , (6.1.13)
4 2 2 2
where I is the identity, and X, Y, Z are the the Pauli operators.

Noise model used in quantum computing emulator runs of metrics

In a number of of the metrics in Sec. IV results are given using an emulator of a quantum computer including
the effects of noise. In this section the details of the emulator runs and of the used parameters and noise model
are provided.
The quantum computer is emulated with all-to-all connectivity (see metric M1.2). The gates used in the
emulator gate set are {I, Rx (π/2), Rz (θ), CX }, where Rz (θ) is a parameterized rotation gate with rotation angle
θ. To this aim the emulator within the Qiskit open-source quantum computing software library is used [161].
The noise model in Qiskit allows the user to specify the exact type of noise applied when executing a quantum

37
6. Concepts and methodologies used for multiple metrics and benchmarks

circuit. The noise model used in this documents includes the noise contributions specified in Sec. 6.1, where
for amplitude damping and depolarizing noise Eqs. 6.1.11 are used. In order to obtain meaningful amplitude
damping and phase damping parameters, Eq. 6.1.12 was used after specifying the T1 and T2 times of the qubits.
The T1 and T2 times are selected from a random normal distribution with a mean of 50µs and 70µs respectively,
and a standard deviation of 1µs for both. To ensure repeatability, a random seed is set such that the T1 times
selected for each qubit remain the same. In order to apply amplitude and phase damping noise when executing
a circuit, the gate times must also be known. The following gate times are used: time for idle gate (I) is 50ns,
time for Rx (π/2) gate is 50ns, time for CX gate is 300ns, and time for measurement is 1000ns. These values are
motivated by typical gate times found in superconducting qubits (see Sec. 5.1). There is no noise applied on
the Rz (θ) as it is modelled to be a virtual gate that is applied by adding a phase to the following gates [233].
For all of the Rx (π/2) gates applied, after the ideal Rx (π/2) gate the following noise contributions are added:
• an over-rotation around the x-axis of π/100 to simulate coherent calibration errors,
• a rotation about the z-axis of π/120 to simulate the coherent phase error occurring due to the applied
pulse being detuned from the qubit frequency,
• a depolarizing noise channel to approximate effectively averaged noise in a large quantum circuit [555].
The depolarizing parameter used in Eq. 6.1.13 for this gate is γ D = 0.0005.
. For all of the 2-qubi CX gates applied, after the ideal CX gate the following noise contributions are added:
−i −i
• an e 2 σz ⊗σx θzx operation and an e 2 σz ⊗σz θzz operation on the 2-qubit subspace the CX gate acts on. The
parameters θzx and θzz are both set to π/100. The zx- and zz- rotation axes are chosen to reproduce
some of the dominant sources of coherent error when applying a cross-resonance gate in superconducting
qubits [556, 557],
• a depolarizing noise channel, with depolarizing parameter in Eq. 6.1.13 set to γ D = 0.005. It is larger
than the value used for single qubit gates, as two-qubit gates typically have larger average errors.

6.2. Volumetric benchmarking

Volumetric benchmarking is a method to probe the overall performance of a quantum computer. The method
is based on the execution of quantum circuits with varying number of qubits, Nq , and with varying depth,
dc [109, 118, 320]. In the context of volumetric benchmarking, the number of qubits is often also denoted as
circuit width, w. It allows one to evaluate how noise affects the results of quantum circuits when progressively
increasing w and dc . For each pair (w, dc ) a number of test circuits are specified, along with a criterion for when
the execution of the quantum circuit is considered to be successful. By executing the sets of circuits on quantum
hardware, and comparing results with those expected for noiseless quantum computers, one can quantify how
successful a given device is in executing the specified tasks. The results are usually depicted via a graphical
summary that illustrates the performance of the device for different values of w and d [118, 320].
There are multiple metrics that use the volumetric benchmarking framework, which include:
• quantum volume (metric M4.1),
• mirrored circuits average polarization (metric M4.2),
• algorithmic qubits (metric M4.3).
A volumetric benchmarking process has the following components:
1. A selection of a family of quantum circuits C(w, dc ) that are to be tested for each w and dc , along with a
rule of how to sample from this family.
2. An experimental design of how the circuits are to be run, including for example setting the number of
shots, ns .
3. A rule restricting how the specified circuits are to be compiled to the native gates of the hardware tested.
4. A rule to measure how successful a run of the circuit is. It takes the measurement outcomes of a w-qubit
circuit with ns shots as input, and returns either a single bit {0, 1} that represents fail or pass, respectively,
or a real number ∈ [0, 1] that quantifies how well the hardware scored in the test.
There are many different ways of measuring the success of running a circuit. For example, one measure
can be that a quantity needs to be greater than a given threshold, which is an approach used within
the quantum volume metric presented in Sec. M4.1. Alternatively, one may compare the measurement
outcome probability distribution with the ideal distribution, which is an approach used for the algorithmic
qubits metric in Sec. M4.3.

38
6. Concepts and methodologies used for multiple metrics and benchmarks

5. A measure of overall success over the ensemble of circuits that are tested. Similar to measuring the
success of a circuit run, there are different ways to measure if an ensemble of circuits, C(w, dc ), is executed
successfully.
One measure is to require that a specified fraction of the total number of circuits in C(w, dc ) runs success-
fully in order for C(w, dc ) to pass. Alternatively, a stricter requirement can be that all circuits in C(w, dc )
need to pass in order for C(w, dc ) to pass.
6. A specified approach for the analysis of the results, which can be the extraction of a single number for a
metric, or a plot of all the results for a more general analysis.

Measurement procedure
Since this is a framework, the specific measurement procedure depends on the chosen algorithms to be used as
test circuits. Here a general procedure is presented:
1. Select the dimensions (w, dc ) and the family of circuits C(w, dc ) to be tested. Define a method to sample
from C(w, dc ), for example by specifying a distribution. Additionally, define the success criteria for each
circuit, and define the success criteria for the circuit ensembles C(w, dc ).
2. Select one circuit randomly according to the defined sampling method, and compile it according to the
native gates of the hardware.
3. Implement the circuit on the hardware and store the measurement outcome.
4. Choose a number of shots to be used, ns . Using the ns measurement outcomes, compute the success value
of the hardware for the circuit performed using the success criteria defined in step 1.
5. Repeat steps 2-4 and collect the success values for each circuit in C(w, dc ). A success value for the overall
performance of the hardware at the dimension tested can be computed using the success criteria for the
circuit ensemble defined in step 1.
6. Collect the results for different dimensions; these are usually plotted in a 2D figure expressing how the
hardware performs for different widths and depths.
7. Optionally, based on the collected results, an overall single metric number is extracted based on a specified
criterion.

Assumptions and limitations

• One common assumption is that the performance with respect to the property tested does not vary
significantly over time, especially during the testing of different circuits and dimensions.
• This framework relies on a meaningful choice of the success criteria for circuits, as well as of the circuit
ensembles. This choice is largely arbitrary, and hence leads to the potential problem that the chosen
success criteria may be biased, and may favour one hardware platform over another. Since there is no
specific set of rules that define whether a success criterion is a relevant one, it has the potential to be
biased. Any results using volumetric benchmarking should precisely explain and motivate the success
criteria used.
• Whether or not this framework is scalable depends on the success criterion used. Typically it requires
comparison to the ideal measurement outcomes, in which case it is only scalable if these outcomes can
be classically computed efficiently also for larger qubit numbers and circuit depths. For systems where
the circuits selected for the volumetric benchmarking are generated randomly, the number of possible
quantum circuits to sample from may increase steeply with the number of qubits. For these cases the
results can depend on the used sampling strategy.

6.3. Gate set tomography

Gate set tomography (GST) gives a matrix representation for each gate in a user defined set of gates, which is
denoted as the process matrix, and also provides estimates for the initial state preparation and for measurement
accuracy in a quantum computer [103, 104]. Typically GST is used only for very few qubits, often just one or
two, since the parameter space needed to define larger systems grows exponentially [103].
GST is a methodology to characterize operations on qubits, and quality metrics obtained using GST are
various measures for the closenesses of the process matrices obtained on hardware to the ideal targets on a
perfect noiseless quantum computer. Rather than comparing the process matrices themselves, their differences
are typically quantified with more intuitive single-valued metrics, which include the following:

39
6. Concepts and methodologies used for multiple metrics and benchmarks

• process fidelity (metric M3.1),

• diamond norm of a quantum gate (metric M3.2),
• state preparation and measurement fidelity (metric M3.7).

Quantum tomography is a general term used to encompass procedures for completely reconstructing a quan-
tum state or process. This is commonly done by running many different circuits to get complete information
of the state or process. Quantum state tomography is used to reconstruct a quantum state [558]. It works by
preparing many identical states, and then measuring them in different bases. For example, for a single qubit
state, one measures in the X, Y , and Z bases to obtain all elements of the density matrix of the state being
characterized, where the bases are constituted by the eigenstates of the corresponding Pauli gate.
In order to characterize the gates in a quantum computer, quantum process tomography is used [67]. In an
ideal noiseless quantum computer, a quantum gate is a unitary operation. However, when physically realized,
these gates are noisy, and may no longer be unitary operations. Instead, they are described by quantum
processes. These are completely positive linear maps, which map a density matrix to another density matrix,
whilst also ensuring that the defining properties of the density matrix are preserved, such as the fact that it
is positive semi-definite and that its trace is equal to one. Positivity of a density matrix means that all its
eigenvalues are greater than or equal to 0. Quantum process tomography amounts to reconstructing the matrix
corresponding to the action of the quantum gate being applied to the qubit. In order to do this, the quantum
gate is applied to different initial states, and then measured in different measurement bases. This then allows
for the quantum gate to be completely characterized if the set of states, and measurement bases, form a basis
on the Hilbert space of the system.
An important drawback of quantum state and process tomography is that they assume that the state prepa-
ration and also the measurement processes are error-free, while in practice these errors can be significant [101].
GST mitigates this issue by taking these errors into account, and by also providing an estimate for the magnitude
of such errors.
A commonly used implementation of GST is long-sequence GST (LSGST), where estimations of gate set
parameters are possible to a precision of O(1/dcmax ), where dcmax is the maximum depth of the set of circuits
generated by the protocol [103]. LSGST treats the qubit as a black box with operation buttons, one for initial-
ization, one for measurement, and the rest for gate operations. It then self-consistently determines estimates
for: the initial state of the qubits, denoted as |ρ⟩⟩ [103]; the set of native measurement projectors, denoted
as {⟨⟨Mi |}, where i is an integer that spans all the native measurement projectors; the set of gates of the
quantum computer, denoted as {Gk }, where k is an integer that spans all gates in the basis. The notation
|ρ⟩⟩ for the initial density matrix ρ is used to indicate that it is represented in a vectorized form, where the
vector entries are in the Pauli tensor product basis, and represented by a real column vector |ρ⟩⟩, called a
superket. Similarly, each measurement projector Mi is vectorized in the Pauli basis and represented by a real
row vector ⟨⟨Mi |, called a superbra. Consequently, quantum gates are represented with 4Nq × 4Nq matrices,
denoted as superoperators, where Nq is the number of qubits that act on superkets. Thus, for a noisy quantum
gate, defined by the quantum process Φ, the superket |ρ⟩⟩ is mapped to SΦ |ρ⟩⟩, where SΦ is the superoperator
representation of Φ. Using this superoperator formalism simplifies the mathematical representation of maps,
since they can be written as a matrix-vector multiplication. This simplification is the main reason for the use
of the superoperator formalism in in the GST framework. Since the Pauli basis is used to vectorize the states
and measurement projectors, the superoperator that characterizes a quantum operation is also defined by the
Pauli basis, and is thus referred to as the Pauli transfer matrix (PTM) of Φ [559].
GST requires the choice of a gate set, G. This includes the target gates to be characterized, and also the
state preparation and measurement gates, as well as the null gate, which performs the identity operation on an
ideal qubit. All the information required for GST is therefore included in the gate set, which can be written as
G = {|ρ⟩⟩, {⟨⟨Mi |}, {Gk }}.
In order to estimate the experimental process matrices for the gate set, the LSGST protocol creates a set
of circuits that amplify errors in the gate set. It does this by generating a list of so-called germ circuits and
fiducial circuits. The germ circuits are chosen such that they amplify all the different possible errors from the
target gate set. The fiducial circuits are chosen such that they create a set of states that can be used to fully
characterize the germ circuits. They are used to prepare the states and to rotate the states before measurement.
As a result, they effectively sandwich the germ circuits between fiducial circuits, with fiducial circuits being
applied before and after a germ circuit. This then ensures that all possible errors in the gate set can be fully
characterized [103].
The term germ refers to the fact that these circuits are like seeds, which is used as a base that is repeated p
times, where p is referred to as the germ power. They are usually repeated in exponentially increasing numbers,
for example, p ∈ {1, 2, 4, 8, 16, 32, ...} [103, 280]. The larger the power, the more the errors are amplified, and,
as a result, more precise estimates are possible. Given the germs, fiducials, and the germ powers, within LSGST
a list of experiments is generated to obtain the process matrices. As shown in Fig. 6.3.1a, in each experiment,

40
6. Concepts and methodologies used for multiple metrics and benchmarks

first a fiducial circuit is performed. Then the germ circuit is applied p times in series. Finally another fiducial
circuit is applied, and then a native measurement takes place. Each experiment is then a circuit built from
native gates, as shown for a representative example in Fig. 6.3.1b. Using this circuit one can calculate the
depth dc of each circuit by counting the maximum number of native gates applied to a qubit in the circuit. The
maximum depth dcmax is the maximum depth of any circuit in the LSGST list of experiments. The value of
dcmax determines the maximum precision for the GST estimate. Note that this is different from p, since a germ
circuit may involve multiple operations on a qubit.

Figure 6.3.1.: Circuit diagrams showing the circuit structure of all long-sequence gate set tomography (LSGST) circuits
for an arbitrary number of qubits. For both circuits, each fiducial circuit is colored orange, and the germ circuit is colored
green. (a) First the fiducial circuit is applied to the qubits, then the germ circuit is applied p times, and then another
fiducial circuit is applied before measurement. (b) An example circuit, where each fiducial and germ circuit is constructed
from the native gates G1 , G2 , and G3 in the gate set G. Each fiducial and germ circuit in the list of germ and fiducial
circuits is constructed from a unique sequence of native gates. The depth of the LSGST circuit is then calculated by
counting the number of native gates in the circuit.

Once the list of experiments is generated, they are run on the quantum computer, and the outcomes for each
experiment are saved to a data set. For a given an initial state ρ, measurement projector Mi and noisy gate Φ,
the experimentally measured probability of outcome i is given by [103]

pi = ⟨⟨Mi |SΦ |ρ⟩⟩ = Tr(Mi SΦ ρ). (6.3.1)

Using the so obtained data set, a gate set estimate is found, denoted as G̃, by using maximum likelihood
estimation (MLE), where the probability that the experimental data was generated given G̃ is maximized by
varying over different gate set estimates. The MLE works by parameterizing the PTM representation for each
element in G to generate a gate set estimate, G̃. Using G̃, the LSGST circuits are simulated, and outcomes
from the simulations are then used to evaluate the likelihood function. An optimizer is used to maximize this
likelihood by adjusting the parameters that define G̃. This then gives an optimal G̃, the estimated gate set with
the optimized gates from the MLE.
The gate sets estimated in this way are determined up to an arbitrary unitary transformation. For example,
given a unitary transformation determined by a matrix B, the estimate of the gate set can be transformed by
′ ′ ′
⟨⟨Mi | → ⟨⟨Mi | = ⟨⟨Mi |B, and |ρ⟩⟩ → |ρ ⟩⟩ = B −1 |ρ⟩⟩, and SΦ → SΦ = B −1 SΦ B. Upon such a transformation
′ ′ ′
the expectation values in Eq. 6.3.1 remain unchanged, ⟨⟨Mi |SΦ |ρ ⟩⟩ = ⟨⟨Mi |SΦ |ρ⟩⟩. The transformation matrix
B therefore does not change any observable quantity that can be obtained with the gate set, and thus cannot
be measured. The matrix B is often called a gauge transformation, and the fact that the gate set can be
transformed by B without altering expectation values is referred to as the gate set having a gauge freedom.
When comparing the PTMs obtained with GST with their ideal targets, changing the gauge can significantly
alter the results [103]. However, changing the gauge does not alter any predicted circuit outcome probabilities.
Therefore, the gauge needs be set such that gate set estimates are as similar as possible to their ideal target
operations, while still describing the experimental observations. This then allows interpreting the remaining
differences between the PTM of the ideal target gate and the PTM obtained with GST as measures for the
accuracy of implementation of the gate on the hardware. They can then be used to estimate the process fidelity
between the ideal target gate and its practical hardware implementation [103] (Sec. M3.1). To find this optimal
gauge, an optimization is performed, which adjusts the parameters of B to minimize the trace norm between
the the estimated gate set and the ideal target gate set. The gauge optimization is performed after a gate set
compatible with the observed data is estimated with MLE. Once it is found, the final PTM obtained with GST
is the one for this optimized gauge.

41
6. Concepts and methodologies used for multiple metrics and benchmarks

Measurement procedure
The general procedure for implementing LSGST is as follows:
1. Select the gate-set to characterize, find fiducial and germ circuits that allow the full characterization of
all errors possible for the target gate set, define the list of the germ power p to generate circuits for, and
then generate the complete experiment list for the LSGST protocol.
2. Run the generated experiments on the quantum computing hardware.
3. Run an optimizer to find the gate set that can produce a data set most similar to experimental data set
using maximum likelihood optimization.
4. Run gauge optimization to find gate set estimate that is closest to the target gate set by adjusting the
parameters of B.
The output provides the PTMs of the gates in the gate set. The difference between the PTMs obtained by GST
and the target ones obtained for the ideal noiseless quantum computer is the metric that allows quantifying the
quality of the hardware operations. As mentioned earlier, the PTMs are typically further processed to calculate
a set of more intuitive single-valued metrics, which are listed in the first part of this section and described in
the following sections.
Example PTMs of an ideal CX gate and a noisy CX gate obtained by GST are shown in Fig. 6.3.2.

Figure 6.3.2.: Example PTMs of (a) an ideal CX gate, and (b) a noisy CX gate, obtained by GST. The noisy CX has
many small non-zero elements in the PTM that arise due to noise. The color represents the value of the PTM element.
For increased clarity, the size of the squares is set to be proportional to the absolute value of the PTM element.

Assumptions and limitations

• The GST model assumes that all noise is Markovian [103]. Markovian noise corresponds to a memory-
less environment. In contrast, non-Markovian noise can account for memory effects arising due to the
environment [223]. Coupling to coherent two level systems and slow fluctuations of qubit frequency are
examples of mechanisms that can lead to such memory effects. When non-Markovian noise is present in
the experimental hardware, it generally leads to a worse agreement between the GST estimates and the
experimental data. Hence, if fitting experimental data with GST leads to a fit with a large mean squared
error, it is usually attributed to the presence of non-Markovian noise [103].
• It is a very time consuming process to run the GST protocol, and currently to run 2-qubit LSGST requires
on the order of 105 circuits. The development of more efficient ways to perform GST is an active area of
research.

Source code
A python notebook demonstrating how to run gate set tomography can be found at the QCMet software repos-
itory under:
gate_execution_quality_metrics/gst_based_gate_execution_quality_metrics. It is based on the PyGSTi
software package [282].

42
6. Concepts and methodologies used for multiple metrics and benchmarks

6.4. Symbols and notation used for the metrics and benchmarks
In this section a list of symbols used throughout the document is given as lookup reference.

Quantum operations and circuits

H Hilbert space
ρ density matrix or density operator
{Mi } set of measurement projectors
I the identity operator
σx ≡ X Pauli X operator
σy ≡ Y Pauli Y operator
σz ≡ Z Pauli Z operator
Φ(ρ) quantum operation, also known as a quantum process or channel
|ρ⟩⟩ density matrix vectorized in Pauli basis
{⟨⟨Mi |} set of native measurement projectors vectorized in Pauli basis
SΦ Pauli transfer matrix of Φ
ρΦ Choi matrix of quantum operation Φ
Nq number of qubits
d = 2Nq Hilbert space dimension
Ki Kraus operator with integer index i
U unitary operation
ψ state vector
Ru (θ) single qubit rotation by angle θ around arbitrary rotation axis u
ρinit initial state of the qubits in quantum computer
U1q (θ, ϕ, λ) general single-qubit unitary with parameters θ, ϕ, and λ
G Clifford gate
C quantum circuit
SWAP SWAP gate
PERM qubit permutation gate
CU controlled unitary gate
CX controlled X gate, also known as CNOT gate
ns the number of shots for the circuit
w width of circuit, typically equal to Nq
dc depth of circuit
C(w, dc ) a family of circuits with width w and depth dc

Probabilities
The notation here uses p to denote a single probability, while the notation of Q denotes a probability distribution.

pi probability of measuring the qubit in state |i⟩

p1 (t) probability of measuring 1 state in T1 experiment at time t
pR
1 (t) probability of measuring 1 state in Ramsey T2∗ experiment at time t
pE
0 (t) probability of measuring 0 state in Hahn echo T2 experiment at time t
psurvival survival probability in randomized benchmarking
p0 (m) probability of measuring the qubit in the |0⟩ state after application of m pseudo-identities
psuccess success probability
px,ideal probability of obtaining output bitstring x in measurement of ideal circuit
Qideal probability distribution of output bitstrings from measuring ideal circuit
px,output probability of obtaining output bitstring x in measurement of experimental circuit
Qoutput probability distribution of output bitstrings from measuring experimental circuit
p(HYP) (sk ) the probability of sample sk under hypothesis HYP in boson sampling
p(HYP) (Nc ) the grouped probability of obtaining Nc clicks in total under the hypothesis HYP in boson
sampling

Fidelities
All fidelities use F as the symbol, and can be distinguished by their subscript.

43
6. Concepts and methodologies used for multiple metrics and benchmarks

F (ρ, σ) state fidelity between two density matrices

Fpro (ΦA , ΦB ) process fidelity between two quantum processes
Fρinit the fidelity of the initial state
FM0 fidelity of the |0⟩ measurement projector for a single qubit
FM1 fidelity of the |1⟩ measurement projector for a single qubit
Fr readout fidelity
FCB composite process fidelity obtained using cycle benchmarking
Fc fidelity of output probability distributions
Fnorm normalized fidelity between two probability distributions

Times
All symbols with the units of time are denoted with the symbol t. The sub and superscripts are used to clarify
their meanings.

tU N q time taken to execute a general Nq -qubit unitary

tRx time taken to execute a single Rx gate
tO total execution time that are independent of the number of gates, for example state
preparation and measurement time
ttot total computation time
to overhead time which occurs independently of the circuit, which is different to tO
tC actual time to run a circuit C
tm maximum of the measurements times of all the qubits
treset time to reset device

Metrics
These are the symbols used for the metrics presented in Sec. IV. Note that some performance indicators
presented in that section may not have a single value as their output and instead may describe the hardware
architecture properties of the quantum computer.

Nmax number of usable qubits

T1 qubit relaxation time
T2 qubit dephasing time from Hahn echo experiment
T2∗ qubit dephasing time from Ramsey experiment
ωmax idle qubit purity oscillation frequency
Fpro (ΦA , ΦB ) process fidelity between two quantum processes

∥ΦA − ΦB ∥⋄ diamond norm between two quantum operations

r Clifford randomized benchmarking average gate error
rGtarget interleaved Clifford randomized benchmarking gate error for target gate
θerr amount of over- or under- rotation
Fρinit the state fidelity of the initial state
FMi fidelity of the |i⟩ measurement projector
FCB composite process fidelity obtained using cycle benchmarking
VQ quantum volume
Jave mirrored circuits average polarization
#AQ the number of algorithmic qubits
b upper bound on the variation distance
Ediff difference between the exact energy of wavefunction and VQE computed energy
Q-score the largest size of graph for which the hardware can solve the MaxCut problem using
QAOA sufficiently accurately on average
Fnorm (Qideal , Qoutput ) normalized fidelity between two probability distributions
tU N q time taken to execute a general single- or multi-qubit gate
treset time to reset qubits

44
IV. Metrics and Benchmarks: definitions, methodology and software

IV.Metrics and Benchmarks: definitions, methodology

and software
In this part, the metrics are presented following the categorization introduced in Sec. II. As the first section
in this part, we also include a description of benchmarking and characterization methodologies used for the
computation of a number of the metrics. The listed metrics include both well defined and widely used metrics,
as well as emerging performance measures. Most metrics are characterized by a single number, for example
the various fidelities. Some metrics, especially architectural metrics such as the connectivity or the available
gate-set, are specified by lists or graphs.
For each metric we present a short set of assumptions and limitations. There are also implicit assumptions
and limitations that apply more generally to most metrics. These general assumptions and limitations include
but are not limited to the following:
• For many benchmarking approaches it is implicitly assumed that there is no leakage outside the qubit
subspace to higher-order levels in the hardware.
• For all qubit quality metrics and gate execution quality metrics, with the exception of the idle qubit purity
oscillation frequency (metric M2.3), Markovian noise is assumed. Note that non-Markovianity does not
negate the usefulness of the metrics, however, more care needs to be taken when interpreting the metrics
and using them for comparisons if there is significant non-Markovianity.
• Most metrics require running circuits repeatedly to obtain the probability distributions of the outcomes.
The number of repetitions, also called the number of shots, should be chosen based on the desired precision
of the metric, and is assumed to be reported with the metric value together with other relevant circuit
execution and analysis parameters.
• It is assumed that the circuits are run on hardware as specified, and that where circuit recompilations or
classical emulations of circuits to obtain results are performed, these are reported with the metrics values.
We will not explicitly refer to these general assumptions and limitations in each metric description.
For most of the metrics, we develop tutorial code on how to run and calculate the metric, and make it publicly
available in an online open-source repository, with links provided. All tutorial code is written using open-source
Python packages with permissive licenses. We have developed a generic framework for submitting quantum
circuits to a user-specified backend, which can be emulators or real quantum hardware. We have run most
metrics for gate-based quantum computers on an emulator with a consistent noise model, which is described
in Sec. 6.1, and the results are given as examples in the description of each metric. Users can both adapt the
code relevant to the metrics independently of the backend, and change the targeted backend independently of
the metrics.To execute circuit on a specific quantum hardware the users need to include the hardware-specific
backend interfaces to enable running the circuits generated by the metrics code and also fetching the results. We
have included example backend interfaces for quantum computers that were accessible through the the Amazon
Braket interface via Amazon Web Services (AWS) at the time of writing. Standardization of such low-level
hardware interfaces may progress independently as hardware platforms mature.
Below, we provide the template that each metric in our database follows. Each metric includes a short and
then a more extended description, followed by the measurement procedure. This is followed by an outline of
the main assumptions and limitations for the measurement and use of the metric. Finally, a link to open source
code that implements the measurement procedure is provided. The references used for the metric description
are provided at the end of each metric section. The format for the metric description is designed to provide an
overview of the metric, and for a more in-depth discussion the provided literature and references therein can
be consulted. Note that metric descriptions in Sec. M9 sway from this template due to the relative nascency of
the metrics presented in them.

Metric title

Short definition of the metric.

Description
More detailed description of the metric.

45
IV. Metrics and Benchmarks: definitions, methodology and software

Measurement procedure
Description of the methodology to measure the metric.

Assumptions and limitations

Where relevant, a list of assumptions underpinning the metric is provided, as well as practical limitations.

Source code
Where available, this section provides links to free open source libraries suitable for measuring the metric.

Metric references
At the end of each entry, a short list of relevant literature is provided.

46
M1. Hardware architecture properties

M1.Hardware architecture properties

M1.1. Number of usable qubits

This metric gives the total number of qubits available to use in a quantum computer.

Description
The number of usable qubits, Nmax , in a quantum computer refers to the maximum number of qubits that can
be used in a single quantum circuit, so that an Nmax -qubit quantum state can be represented on the hardware.
A device may have more than Nmax qubits in total, but some of these may not be available for use in a quantum
circuit. Qubits may be disabled for various reasons, for example when individual qubits are defective due to
problems in the fabrication process. The aim of this metric is not to determine the quality of the qubits, only
that they exist, and they can be used together to run quantum circuits.

Measurement procedure
The number of qubits available to use needs to be reported by the hardware manufacturer. The procedure
below is a way for the end-user to verify that the number provided by the hardware manufacturer is indeed
usable in a single quantum circuit:

1. Initialize Nq qubits and perform a single-qubit gate, such as the X gate, on each of the Nq qubits. Then
measure all qubits at the end.
2. If the circuit is executed and an Nq -bit output corresponding to the measurement results is received
without an error being raised by the hardware or by its software interface, one can infer that the hardware
contains at least Nq usable qubits.

3. Increase Nq until an error is raised by the hardware, or generally if the measurement over all qubits returns
fewer than Nq values.
4. The maximum number of usable qubits Nmax is given by the largest value of Nq , for which the circuit was
executed and resulted in an Nq -bit output without raising an error.

In this procedure it is assumed that when a user tries to run a circuit with more qubits than those that are
available, the hardware either raises an error that explains to the user that the number of qubits available is
lower than the requested one, or that the number of bits in the output is lower than the requested one.

Assumptions and limitations

• The connectivity of the qubits is not taken into account. Therefore, a device with Nmax qubits, but which
does not allow two-qubit gates, has Nmax as the number of usable qubits, even though entangled states
cannot be generated.

• The number of usable qubits in a device by itself may not be representative of how many qubits can be
used concurrently in a useful manner due to the noise in the device.
• If a subset of qubits has significantly higher error rates than the rest of the qubits, then an application
may not be able to usefully include these large-error qubits in a quantum circuit, even though they are
counted in this metric.

• For some hardware platforms, the number of available qubits may be probabilistic, depending on the state
of the device at a given time [20, 560]. For example, this may be the case for neutral atom systems (see
Sec. 5.3).

Metric references
[20] J. Wurtz, A. Bylinskii, B. Braverman, J. Amato-Grill, S. H. Cantu, F. Huber, A. Lukin, F. Liu, P. Weinberg,
J. Long, et al., “Aquila: QuEra’s 256-qubit neutral-atom quantum computer”, arXiv:2306.11727 (2023).
[560] K.-N. Schymik, V. Lienhard, D. Barredo, P. Scholl, H. Williams, A. Browaeys, and T. Lahaye, “Enhanced atom-
by-atom assembly of arbitrary tweezer arrays”, Phys. Rev. A 102, 063107 (2020).

47
M1. Hardware architecture properties

M1.2. Pairwise connectivity

This is an architectural property that specifies which pairs of qubits can physically perform two-qubit gates
between each other.

Description
The connectivity outlines which pairs of qubits are directly connected, and is typically expressed as a graph or
list of pairs. Two qubits are considered to be directly connected if a two-qubit gate between these two qubits can
be executed directly on the hardware without the need to resort to SWAP gates. For some hardware platforms,
the two qubits may only be connected in a unidirectional manner, where only one of the two qubits can be
the target qubit. For example, Fig. M1.2.1a shows the connectivity of the Oxford Quantum Circuits (OQC)
quantum computer “Lucy”, where the arrows between qubits represent the direction of the native two-qubit
gate. The arrow starts at the control qubit and is pointed towards the target qubit. Applying two-qubit gates
between qubits that are not directly connected may be achieved by applying a chain of SWAP gates or by
physically moving qubits to different locations in architectures such as trapped ions or neutral atoms. Such
operations increase both the execution time as well as the induced noise error [176].
A better connectivity can reduce algorithmic complexity [176, 179]. Error correction codes typically require
specific types of connectivity between physical qubits [141, 561].
For example, Fig. M1.2.1 shows the connectivity of the OQC device “Lucy” and the IonQ device “Aria 1”.

Figure M1.2.1.: The connectivity of (a) the OQC superconducting circuit quantum computer “Lucy” and (b) the IonQ
trapped-ion quantum computer “Harmony”. The qubits are indexed with numbers starting from 0. (a) There are 8
qubits in OQC Lucy with circular connectivity, where an arrow between a pair of qubits indicates the control to target
direction of applicable two-qubit controlled gates. (b) There are 11 qubits in IonQ Harmony with all-to-all connectivity.

Measurement procedure
As connectivity is hardware specific, it needs to be reported by the hardware manufacturer. To verify the
correctness of the reported connectivity, as user one can execute a set of quantum circuits that apply two-qubit
gates between all pairs of reportedly connected qubits. Circuit transpilation needs to be disabled for these runs,
since it would implement two-qubit gates also between non-connected qubits by including SWAP gates.

Assumptions and limitations

• The information on the connectivity needs to be provided by the manufacturer. Not all manufacturers
may be willing to provide this information.
• Whilst connectivity determines which qubits are physically connected to each other, the connectivity map
may not be a good representation of the physical layout of the device. The physical layout is important
in problems such as analysing crosstalk errors, where qubits that are physically closer may create more
mutual crosstalk noise [562].

48
M1. Hardware architecture properties

• While a device may claim all-to-all connectivity, this does not necessarily mean that the qubits are always
connected. For example, in the trapped ion architecture (see section 5.2), a two-qubit gate between
arbitrary qubits is implemented by physically moving the two selected qubits together for a two-qubit
gate [187, 237].

• Some hardware platforms, such as neutral atoms or trapped ion devices, also have native multi-qubit gates
that operate simultaneously on more than two qubits.

Metric references
[141] J. Roffe, “Quantum error correction: an introductory guide”, Contemp. Phys. 60, 226 (2019).
[176] A. Holmes, S. Johri, G. G. Guerreschi, J. S. Clarke, and A. Y. Matsuura, “Impact of qubit connectivity on
quantum algorithm performance”, Quantum Sci. Technol. 5, 025009 (2020).
[179] A. Cowtan, S. Dilkes, R. Duncan, A. Krajenbrink, W. Simmons, and S. Sivarajah, “On the qubit routing problem”,
arXiv:1902.08091 (2019).
[187] S. Crain, C. Cahall, G. Vrijsen, E. E. Wollman, M. D. Shaw, V. B. Verma, S. W. Nam, and J. Kim, “High-Speed
Low-Crosstalk Detection of a 171Yb+ Qubit Using Superconducting Nanowire Single Photon Detectors”, Nat.
Commun. 2, 1 (2019).
[237] T. P. Harty, D. T. C. Allcock, C. J. Ballance, L. Guidoni, H. A. Janacek, N. M. Linke, D. N. Stacey, and D. M.
Lucas, “High-Fidelity Preparation, Gates, Memory, and Readout of a Trapped-Ion Quantum Bit”, Phys. Rev.
Lett. 113, 220501 (2014).
[561] L. Z. Cohen, I. H. Kim, S. D. Bartlett, and B. J. Brown, “Low-overhead fault-tolerant quantum computing using
long-range connectivity”, Sci. Adv. 8, eabn1717 (2022).
[562] P. Murali, D. C. Mckay, M. Martonosi, and A. Javadi-Abhari, “Software Mitigation of Crosstalk on Noisy
Intermediate-Scale Quantum Computers”, Proc 25th Int Conf Arch. Support Progra Lang Oper. Syst, ASPLOS
’20 (2020), 1001.

M1.3. Native gate set

This architectural metric corresponds to a list of the native gates that are available on the quantum processor.
The native gates are those that are physically executed on the quantum processor. Quantum circuits not
specified using native gates need to be compiled to circuits using the native gates in order to be executed, which
is also referred to as circuit transpilation or decomposition. An important information provided by the metric
is whether the native gate set allows for a universal quantum computation.

Description
The native gate set that a quantum processing unit (QPU) uses can vary between different hardware platforms.
Furthermore, individual algorithms can benefit from the availability of a specific native gate set, as transpilation
into the required gates may typically lead to slower performance. The native gates need to correspond to
physical operations on the qubit hardware, and therefore different types of quantum processors have different
native gate sets [125]. For example, the single-qubit native gates may only consist of calibrated Rx (π/2) pulses,
which combined with virtual Rz -gates can give arbitrary rotations [233]; alternatively, the Z-gates can also be
implemented in hardware rather than executed virtually [232]. For two-qubit gates, a superconducting QPU
may have controlled-Z gates as its native two qubit gate [2], whereas an ion trap QPU may use an XX(θ)
gate [9], which is defined by e−iθX⊗X/2 , and referred to as Mølmer-Sørenson gate [482].

Measurement procedure
The information on the native gate set needs to be provided by the hardware manufacturer. Here we list a
number of possibilities, with which it can be extracted from the device:
1. The hardware provider documentation can directly provide the details on the native gate set.
2. Performing only the transpilation step before executing a quantum circuit allows extracting the native
gates that the circuit uses to run on hardware.
3. A list of gates can be sent to the hardware for execution without transpilation: if a circuit runs successfully
for a given gate, then that gate is a native gate. If an error is reported by the system, then that gate is
not part of the native gate set.

49
M1. Hardware architecture properties

Assumptions and limitations

• Hardware manufacturers may not make the information required to determine the native gate set publicly
available.

Source code
A tutorial showing how to fetch the set of native gates of a device is provided in the QCMet software repository
under:
hardware_architecture_properties/native_gates.

Metric references
[2] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A Quantum Engineer’s
Guide to Superconducting Qubits”, Appl. Phys. Rev. 6, 021318 (2019).
[9] C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, “Trapped-ion quantum computing: Progress and
challenges”, Appl. Phys. Rev. 6, 021314 (2019).
[125] P. Murali, N. M. Linke, M. Martonosi, A. J. Abhari, N. H. Nguyen, and C. H. Alderete, “Full-stack, real-system
quantum computer studies: architectural comparisons and design insights”, Proc 46th Int Symp Comput Arch.
ISCA ’19 (2019), 527.
[232] E. Lucero et al., “Reduced phase error through optimized control of a superconducting qubit”, Phys. Rev. A 82,
042339 (2010).
[233] D. C. McKay, C. J. Wood, S. Sheldon, J. M. Chow, and J. M. Gambetta, “Efficient Z gates for quantum
computing”, Phys. Rev. A 96, 022330 (2017).
[482] A. Sørensen and K. Mølmer, “Entanglement and quantum computation with ions in thermal motion”, Phys. Rev.
A 62, 022311 (2000).

M1.4. Capability to perform mid-circuit measurements

This architectural metric states whether or not a quantum computer has the capability to perform mid-circuit
measurements, where one or more qubits can be measured multiple times with the results of the measurements
recorded, while the circuit is still being executed without measurement on other qubits, so that, in absence of
noise in the device, these preserve their quantum coherence during the full circuit execution.

Description
Mid-circuit measurements are required for many quantum error correction protocols [148, 485]. They can be
used to reduce the number of qubits in other algorithms, such as Shor’s algorithm [328], where the total number
of qubits is one-third of that required in the protocol without mid-circuit measurements [563]. They can also
be used to reduce the number of physical qubits required to run quantum algorithms such as the quantum
approximate optimization algorithm (see metric M5.2) by measuring qubits as early as possible, and reusing
them elsewhere in the circuit [201]. Additionally, mid-circuit measurements can be used to drive conditional
operations later in the quantum circuit which can enable measurement-based quantum computation [564]. This
binary metric provides the information on whether such mid-circuit measurements can be performed.

Measurement procedure
The hardware vendor is to provide the information as to whether mid-circuit measurements are possible. If
they are possible, also the software commands to execute them are to be provided. The user can then execute
such commands to verify the functionality of the mid-circuit measurements. To verify and test the information
provided by the hardware provider, a user can apply the following quantum circuit:
1. Initialize two qubits to the |00⟩ state.

2. Apply a Hadamard gate to the first qubit, then apply a CX gate between the two qubits, with the first
qubit being the control and the second qubit being the target.
3. Perform a mid-circuit measurement on the second qubit, and record the result.
4. Apply a second CX gate between the two qubits, again with the first qubit being the control and the
second qubit being the target.

50
M1. Hardware architecture properties

5. Measure both qubits and record the results.

6. Repeat steps 1-5 for a set number of shots.
If an error is returned by the sequence of operations, then it means the device does not accept mid-circuit
measurements. Alternatively, if there is no error, one must check that there are no measurement inconsistencies
to ensure that the mid-circuit measurements are working as expected.
We note that this metric only evaluates the capability to perform a mid-circuit measurement, and does not
evaluate its fidelity. Nevertheless, the results obtained with the methodology outlined above may also form
the basis for the evaluation of the fidelity of the mid-circuit measurement. In an ideal noiseless case, the first
measurement in the quantum circuit gives states |0⟩ or |1⟩ with equal probabilities, and the second measurement
gives |00⟩ if the first measurement gives |0⟩, or it gives |10⟩ if the first measurement gives |1⟩. The difference
between the measured probabilities of the three measurements to these ideal values give an estimate of the
fidelity of the whole circuit including the mid-circuit measurements. Deviations of the results obtained on
hardware from the ideal result are also induced by noise in state preparation and measurement, as well as in
circuit execution. One may separate out the errors induced by the mid-circuit measurements from the other
noise-induced errors by running the same circuit, but without performing the mid-circuit measurement. The
final output for a noiseless device in this case is also either |00⟩ or |10⟩ with equal probability. By comparing the
results with and without mid-circuit measurements, one can then infer the infidelity in the final probabilities
induced by the mid-circuit measurement.

Assumptions and limitations

• Beyond the ability to perform mid-circuit measurements, it is also important to quantify the errors that
such measurement cause, both on the measured qubit as well as on other qubits, which are a benchmark
for the quality of mid-circuit measurements [97, 565]. Information on the quality of the mid-circuit
measurement can be provided through further metrics, such as circuit based metrics that include mid-
circuit measurements [565].
• Further useful information that the hardware manufacturer should provide is if control on qubits can be
performed based on the outcome of mid-circuit measurements during circuit execution.

Source code
A tutorial for testing if a device supports mid-circuit measurements is provided in the QCMet software repository
under:
hardware_architecture_properties/mid-circuit_measurements.

Metric references
[97] L. Govia, P. Jurcevic, C. Wood, N. Kanazawa, S. Merkel, and D. McKay, “A randomized benchmarking suite for
mid-circuit measurements”, New J. Phys. 25, 123016 (2023).
[148] Google Quantum AI, “Suppressing Quantum Errors by Scaling a Surface Code Logical Qubit”, Nature 614, 676
(2023).
[201] M. DeCross, E. Chertkov, M. Kohagen, and M. Foss-Feig, “Qubit-Reuse Compilation with Mid-Circuit Measure-
ment and Reset”, Phys. Rev. X 13, 041057 (2023).
[328] P. W. Shor, “Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum
Computer”, SIAM J. Comput. 26, 1484 (1997).
[485] C. Ryan-Anderson et al., “Implementing fault-tolerant entangling gates on the five-qubit code and the color code”,
arXiv:2208.01863 (2022).
[563] E. Martin-Lopez, A. Laing, T. Lawson, R. Alvarez, X.-Q. Zhou, and J. L. O’brien, “Experimental realization of
Shor’s quantum factoring algorithm using qubit recycling”, Nat. Photonics 6, 773 (2012).
[564] D. E. Browne, E. Kashefi, M. Mhalla, and S. Perdrix, “Generalized flow and determinism in measurement-based
quantum computation”, New J. Phys. 9, 250 (2007).
[565] K. Rudinger, G. J. Ribeill, L. C. Govia, M. Ware, E. Nielsen, K. Young, T. A. Ohki, R. Blume-Kohout, and T.
Proctor, “Characterizing Midcircuit Measurements on a Superconducting Qubit Using Gate Set Tomography”,
Phys. Rev. Appl. 17, 014014 (2022).

51
M2. Qubit quality metrics

M2.Qubit quality metrics

M2.1. Qubit relaxation time (T1 )

The qubit relaxation time, usually referred to as T1 time, is a metric for the timescale at which a qubit decays
from its excited state, |1⟩, to the ground state, |0⟩ [2].

Description
Energy exchange with the environment can lead to the qubit spontaneously going from the excited state, |1⟩,
to the ground state, |0⟩, or vice versa. The steady-state population of the states depends on the temperature
and can be derived using Boltzmann statistics [2]. Typically, qubits are operated at low temperatures, such
that the rate of excitation is significantly suppressed compared to the rate of decay [566]. In this regime, the
steady-state of the qubit is the |0⟩ state. The probability that a qubit prepared in the |1⟩ state at time zero has
decayed to the |0⟩ at time t, denoted as p1 (t), is given by

p1 (t) = e−t/T1 , (M2.1.1)

where T1 is the qubit relaxation time. This is also sometimes referred to as the longitudinal relaxation time [2].

Measurement procedure
1. Prepare the qubit in the |1⟩ state.
2. Leave the qubit idle for a set time, t.

3. Measure the qubit in the {|0⟩ , |1⟩} computational basis and store the outcome.
4. Repeat steps 1-3 for a certain number of shots to calculate the probability of the qubit being in the |1⟩
state. The number of shots should be chosen based on the desired benchmarking precision.
5. Repeat steps 1-4 for different values of t in step 2.

6. Fit the observed probabilities to Eq. M2.1.1 to obtain T1 .

An example of the results obtained on an emulator is shown in Fig. M2.1.1.

Figure M2.1.1.: Results of an emulator run with the methodology outlined in this section to obtain the T1 time, where
the noise model specified in Sec. 6.1 is used. The black crosses indicate the measured probabilities of the |1⟩ state, p1 ,
at each qubit idle time, t, where the time is in units of the single-qubit gate time, t1qgate . An exponential decay curve
following Eq. M2.1.1 is fitted to obtain T1 = 1035 t1qgate . This value is in good agreement with the parameters set in the
used noise model, which give a theoretical value of T1 = 1035 t1qgate . If t1qgate is known, then one can obtain T1 in units
of seconds.

52
M2. Qubit quality metrics

Assumptions and limitations

• It is assumed that the |1⟩ state is the higher-energy state, and that the operating temperature of the qubit
is low enough that the steady state of the qubit is mostly the |0⟩ state.

• At the beginning of the measurement procedure the qubit is typically in the steady state |0⟩, and the |1⟩
state is then set by applying a gate to rotate the qubit from the |0⟩ state to the |1⟩ state as part of the
initialization. The result depends on the accuracy of the calibration of this gate. A badly calibrated gate
only approximately rotates the |0⟩ state to the |1⟩ state in the first step completely, which adds to the
overall measurement uncertainty of the T1 estimation.
• It is assumed that one either has precise knowledge of, or has precise control over, the timings of the
gates applied to the qubits. When this information is not provided by the hardware vendor, it may be
challenging to estimate the durations and timings of the gates.
• The T1 time of single qubits is also meaningful in an annealing setting, but may be misleading. In
particular, this quantity does not take into account that low-temperature dissipation can actually restore
coherence in an annealing setting, and as a result quantum annealing can operate successfully even when
the annealing time far exceeds the single qubit T1 .

Source code
A tutorial for measuring T1 time is provided in the QCMet software repository under:
qubit_quality_metrics/t1.

Metric references
[2] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A Quantum Engineer’s
Guide to Superconducting Qubits”, Appl. Phys. Rev. 6, 021318 (2019).
[566] D. M. Berns, W. D. Oliver, S. O. Valenzuela, A. V. Shytov, K. K. Berggren, L. S. Levitov, and T. P. Orlando,
“Coherent Quasiclassical Dynamics of a Persistent Current Qubit”, Phys. Rev. Lett. 97, 150502 (2006).

M2.2. Qubit dephasing time (T2 )

The qubit dephasing time, usually referred to as T2 , is a metric for the timescale at which the phase information
of a state in superposition is lost [2, 567].

Description
The qubit dephasing time, also sometimes referred to as the transverse relaxation time [2], can be reported
through both the so-called T2 and the T2∗ values. Both metrics quantify the rate of phase information loss,
but are different in that T2∗ is highly sensitive to low-frequency fluctuations in qubit frequency, while the
measurement procedure of T2 makes it less sensitive to such fluctuations [2, 567]. Note that the qubit dephasing
times include the effects of both amplitude damping and pure dephasing (see Sec. 6.1).

Measurement procedure
First, the procedure for measuring the T2∗ time, denoted as the Ramsey experiment [2], is described. This
experiment is carried out with intentional detuning of the qubit reference frame, which leads to the qubit
precessing about the Z-axis even when no quantum gates are being applied. The physical implementation of
this detuning depends on the hardware platform being used. Slow oscillations of the qubit due to unintentional
detuning caused by a lack of accuracy in the frequency calibration can be misinterpreted as a decay, thus a
large intentional detuning, which leads to clearly visible fast oscillations, is used to avoid this.
1. With the qubit starting in the |0⟩ state, apply an Rx (π/2) rotation gate to rotate the qubit into the
equator of the Bloch sphere.

2. Leave the qubit idle for a certain amount of time, t.

3. Apply a Rx (π/2) gate, which rotates the qubit back from the equator of the Bloch sphere into the Z-axis.
4. Measure the qubit in the computational basis.

53
M2. Qubit quality metrics

5. Repeat steps 1-4 for a specified number of shots to calculate the probability of the qubit being in the |1⟩
state. The number of shots should be chosen based on the desired benchmarking precision.
6. Repeat steps 1-5 for different idle time durations t.
7. By estimating the probability for different t, the oscillations between the |0⟩ and |1⟩ can be fitted to the
equation
−t/T2∗
pR
1 (t) = a e cos(2πf t + ϕ) + b, (M2.2.1)
where pR 1 (t) is the probability of measuring the |1⟩ state at a given delay time t, a is the amplitude of
the decaying cosine, f is the frequency of the oscillations, which depends on the detuning of the qubit
reference frame, ϕ is the offset of the cosine, and b is the baseline offset of the cosine. This allows one to
find T2∗ .
To measure the T2 time, the same procedure as the Ramsey experiment is followed, with the difference that
in step 2, the qubit is left idle for time t/2 , then an Rx (π) gate is applied, and then the qubit is left idle again
for time t/2. This is denoted as a Hahn echo experiment [2, 567]. The Hahn echo experiment is fitted to the
following equation:
−t/T2
pE
0 (t) = A e + B, (M2.2.2)
where pE 0 (t) is the probability of measuring the |0⟩ state versus the idle time t, A sets the amplitude, and B is
the baseline offset. This allows one to find T2 .
The naming for these experiments has been adopted from nuclear magnetic resonance (NMR) spectroscopy [567,
568]. Since the T2 time is measured using an echo experiment, it is often labeled T2E . Similarly, the T2∗ time is
often labeled T2R , as it is obtained using a Ramsey experiment.
The choice of the idle durations t can either be made by using prior knowledge of the expected T2∗ time, or
by trial-and-error. To do the latter, one chooses the longest t, denoted as tmax , by repeating the experiment
and evaluating the time required for the probability to measure |1⟩ to be approximately 12 . Then, the number
of idle durations between 0 and tmax must be chosen to allow for a sufficiently good fit of the resulting decay
envelope.
An example of the results one might obtain when measuring the T2∗ and the T2 times is shown in Fig. M2.2.1a
and Fig. M2.2.1b, respectively.

Figure M2.2.1.: Results of emulator runs with the methodology outlined in this section to obtain the T2∗ and T2 times,
where the noise model specified in Sec. 6.1 is used. Here the crosses represent the measured results on the emulator,
and the dashed curves are fits to the data. The time is in units of the single-qubit gate time, t1qgate . (a) Results of
the Ramsey experiment: probability of the |1⟩ state, pR 1 , at each qubit idle time, t, where the time is in units of t1qgate ;
Eq. M2.2.1 is fitted to the data to obtain a T2∗ = 1444.3 t1qgate . (b) Results of the Hahn echo experiment: probability
of the |0⟩ state, pE0 as function of time; here Eq. M2.2.2 is fitted to the data to obtain a T2 = 1554.6 t1qgate . This
value is in approximate agreement with the parameters set in the used noise model, which give a theoretical value of
T2 = 1437 t1qgate . If t1qgate is known, then one can obtain the times in units of seconds.

Assumptions and limitations

• It is assumed that one either has precise knowledge of, or has precise control over, the timings of the
gates applied to the qubits. When this information is not provided by the hardware vendor, it may be
challenging to estimate the durations and timings of the gates.

54
M2. Qubit quality metrics

• The T2 time of single qubits is also meaningful in an annealing setting, but may be misleading. In
particular this quantity does not take into account that low-temperature dissipation can actually restore
coherence in an annealing setting, and as a result quantum annealing can operate successfully even when
the annealing time far exceeds the single qubit T2 .

• Inaccuracies in gate calibrations can add to the overall measurement uncertainty of the T2 estimation.

Source code
A tutorial for measuring the T2∗ time is provided in the QCMet software repository under:
qubit_quality_metrics/t2.

Metric references
[2] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A Quantum Engineer’s
Guide to Superconducting Qubits”, Appl. Phys. Rev. 6, 021318 (2019).
[567] G. Ithier et al., “Decoherence in a superconducting quantum bit circuit”, Phys. Rev. B 72, 134519 (2005).
[568] E. L. Hahn, “Spin Echoes”, Phys. Rev. 80, 580 (1950).

M2.3. Idle qubit purity oscillation frequency

An idle qubit periodically losing and regaining coherence can be a signature of non-Markovian noise [223, 224].
The oscillation frequency of the qubit purity can quantify the effect of non-Markovian noise induced coherence
revivals, where a higher frequency indicates larger non-Markovianity of the noise.

Description
Non-Markovian noise can lead to the qubit purity oscillating as a function of time, even for an idle qubit[223,
224]. This noise is challenging to correct with quantum error correction because it leads to time-correlated
 
error [569]. The qubit purity, ζ, is a measure of the coherence of a qubit, and is defined as ζ = Tr ρ2 , where ρ
is the qubit density matrix. Measuring the qubit observables corresponding to the expectation values of σx , σy ,
and σz , allows for the calculation of the qubit purity using the equation
1 2 2 2

ζ= 1 + ⟨σx ⟩ + ⟨σy ⟩ + ⟨σz ⟩ , (M2.3.1)
2
where ⟨σi ⟩ = Tr{σi ρ}. The values of ⟨σi ⟩ can be measured by applying a change of basis operation before the
qubit measurement.
In this metric the single-qubit non-Markovian noise induced purity oscillation frequency is characterized for
the idle qubit. Since purity oscillations in general depend on the initial state of the qubit, it is √ necessary to
prepare the qubit
√ in the x, y, and z bases by initializing the qubit in the |0⟩, |+⟩ = (|0⟩ + |1⟩)/ 2, and |R⟩ =
(|0⟩ + i |1⟩)/ 2 states, respectively, and then perform the calculations in each basis. The native measurement
occurs in the z basis, whilst measuring in the x basis requires a Hadamard gate before the native measurement
operation, and measuring in the y basis requires applying the conjugate transpose of the S gate followed by a
Hadamard gate before native measurement.
When measuring the qubit purity for different initial states, oscillations in purity as a function of idle time
appear. These can be fitted with the following decaying oscillation function

ζ(t) = a + b e−λ t cos(ω t), (M2.3.2)

where fitting parameters a, b include the effects of readout and state preparation error, respectively. The fitting
parameter λ includes the effect of decoherent errors, and the fitting parameter ω corresponds to purity oscillation
frequency. The result of this procedure gives a value for the qubit purity oscillation frequency for each of the
different initial states of the qubit. To estimate the worst-case effect on the qubit performance, the maximum
of the three fitted frequencies, ωmax , is taken as a metric for the amount of single-qubit non-Markovian noise
induced purity oscillations. Markovian qubit dynamics correspond to ωmax = 0.

Measurement procedure
1. Select the time step ∆t and maximum time tmax to leave the qubit idle to get the set of idle times
T = {0, ∆t, 2∆t, ..., tmax } for which to run the circuits.

55
M2. Qubit quality metrics

2. Then, for each initial state |i⟩ ∈ {|0⟩ , |+⟩ , |R⟩}, perform the following operations:
a) For each idle time t ∈ T :
i. For each measurement basis b ∈ {x, y, z}:
A. Prepare the qubit in the state |i⟩.
B. Leave the qubit idle for time t.
C. Measure the qubit in basis b by applying the corresponding basis change gates and then
measuring in the computational basis.
ii. Calculate the purity ζ using Eq. M2.3.1.
b) For each initial state |i⟩, fit the function in Eq. M2.3.2 to a decaying oscillation and store the estimated
oscillation frequency ωi .
3. The largest oscillation frequency from all three initial states gives the single-qubit non-Markovian noise
induced purity oscillation frequency, ωmax .
The value of ∆t must be chosen to be small enough to avoid aliasing effects in oscillations, and tmax must be
chosen to be large enough to either see oscillations, or to ensure ζ(t) has reached its steady state value in the
absence of oscillations.
An example of the results one might obtain is shown in Fig. M2.3.1.

Figure M2.3.1.: Results of emulator runs using the methodology outlined in this section to obtain the idle qubit
purity oscillation frequency, ωmax , where the noise model used is specified in Sec. 6.1. Since that noise model does not
include non-Markovian noise, in order to also model the non-Markovian noise, a second qubit is added to the emulator
to act as the environment. Then, a two qubit RZZ (θ) gate is applied at each time-step between the two qubits, with
θ = 0.15 to emulate ZZ-crosstalk noise. Tracing out the additional qubit, which is acting as the environment, leads
to non-Markovian dynamics of the qubit being measured [223]. The theoretical value is then ωmax = 0.15/t1qgate . The
three plots show the qubit purity ζ(t) calculated using Eq. M2.3.1 for each initial state, (a) |0⟩, (b) |+⟩ and (c) |R⟩,
respectively. The black crosses are the calculated ζ. The gray dashed curves are the fits to Eq. M2.3.2. The estimated
oscillation frequencies are (a) 0, (b) 0.1490 and (c) 0.1496, respectively, hence the largest oscillation frequency gives the
single-qubit non-Markovian noise induced purity oscillation frequency, ωmax = 0.1496/t1qgate . This is in good agreement
with the theoretical value obtained directly from the noise model parameters.

Assumptions and limitations

• This method only measures the effect of non-Markovian noise which leads to purity oscillations. There
may be other kinds of non-Markovian noise effects not accounted for by this procedure [221, 570].
• This metric can indicate the presence of effects that cannot be describe by a purely Markovian single qubit
model, however, it cannot distinguish between environment induced sources of non-Markovian noise and
effects such as crosstalk without further measurements.
• In the fitting procedure it is assumed that the purity has at most one oscillation frequency. If multiple
sources of non-Markovian noise are present, the purity may exhibit multi-frequency oscillations as well.
• State preparation and measurement (SPAM) errors can lead to significant errors in the estimation of the
qubit purity, p, but they will not affect the purity oscillation frequency. However, if the SPAM errors
are large, they can make the fitting procedure to calculate oscillation frequency challenging. In this case,
precise experiments with a large number of shots might be needed.

56
M3. Gate execution quality metrics

Source code
A tutorial showing how to obtain the amount of single-qubit non-Markovian noise induced purity oscillations is
provided in the QCMet software repository under:
qubit_quality_metrics/idle_qubit_purity_oscillation.

Metric references
[221] Á. Rivas, S. F. Huelga, and M. B. Plenio, “Quantum non-Markovianity: characterization, quantification and
detection”, Rep. Prog. Phys. 77, 094001 (2014).
[223] A. Agarwal, L. P. Lindoy, D. Lall, F. Jamet, and I. Rungger, “Modelling non-markovian noise in driven super-
conducting qubits”, Quantum Sci. Technol. 9, 035017 (2024).
[224] B. Gulácsi and G. Burkard, “Signatures of non-Markovianity of a superconducting qubit”, Phys. Rev. B 107,
174511 (2023).
[569] K. Young, S. Bartlett, R. J. Blume-Kohout, J. K. Gamble, D. Lobser, P. Maunz, E. Nielsen, T. J. Proctor,
M. Revelle, and K. M. Rudinger, “Diagnosing and Destroying Non-Markovian Noise”, Tech Rep Sandia Natl.
LabSNL-CA Livermore CA U. S. (2020).
[570] M. J. W. Hall, J. D. Cresser, L. Li, and E. Andersson, “Canonical form of master equations and characterization
of non-markovianity”, Phys. Rev. A 89, 042120 (2014).

M3.Gate execution quality metrics

M3.1. Gate set tomography based process fidelity

The process fidelity is a measure of how close one quantum process is to another quantum process. When
referring to the process fidelity of a quantum process, this means that it compares the closeness of the quantum
process implemented on a noisy quantum computer to its ideal noiseless and perfectly calibrated target pro-
cess [71, 78, 80]. Often, the resulting process infidelity is given rather than the fidelity, which is given by one
minus the fidelity. For this specific metric, gate set tomography (GST) is used to compute the fidelity (section
6.3), but also different process tomography approaches may be used.

Description
The evaluation of the fidelity between two processes is based on the methodology to obtain the fidelity of two
states [80]. The fidelity between two states, represented by their density matrices ρ and σ, is given by [72]
 q 2
√ √
F (ρ, σ) = Tr ρσ ρ . (M3.1.1)

This is a measure for the closeness between the two generally mixed states ρ and σ. If one of the two states is
a pure state, then the equation for the fidelity is simplified to

F (ρ, σ) = Tr [ρσ] . (M3.1.2)

To determine the fidelity of a quantum process, denoted as Φ, one can use these equations as well, where the
process is represented as a valid density matrix, ρΦ . For a given SΦ , the Pauli transfer matrix (PTM) of Φ as
described in section 6.3, one can find ρΦ using the following equation [101]
2
d
1 X
ρΦ = SΦ P † ⊗ Pi , (M3.1.3)
d2 i,j=1 ij j

where Pi and Pj are tensor products of Pauli matrices, and i, j iterate through all possible P ∈ {I, X, Y, Z}⊗Nq ,
where Nq is the number of qubits and d = 2Nq . The matrix ρΦ is denoted as the Choi matrix of a quantum
channel. Using this representation of a quantum process, one can calculate the process fidelity between two
quantum processes, ΦA and ΦB , denoted as Fpro (ΦA , ΦB ), using Eq. M3.1.3 and Eq. M3.1.1 to get [80]

Fpro (ΦA , ΦB ) = F (ρΦA , ρΦB ). (M3.1.4)

57
M3. Gate execution quality metrics

This equation is valid for all quantum processes. In the case where one wants to calculate the process fidelity
where the target process ΦB is unitary, then one can use Eq. M3.1.2 to obtain a simpler equation [77]:
1 h
†
i
Fpro (ΦA , ΦB ) = Tr SΦA
SΦB . (M3.1.5)
d2
To obtain the process fidelity, the PTMs for the processes are needed, so that one has to run process tomogra-
phy to obtain the PTMs. In order to mitigate for state preparation and measurement (SPAM) errors, typically
gate set tomography (GST) is run, which is a special type of process tomography that can correct for SPAM
errors at an approximate level (see section 6.3).

Measurement procedure
The procedure to calculate the process fidelity of a single experimental gate is as follows:

1. Select a gate to characterize, Φexp , with the ideal quantum process matrix given by Φideal , which typically
is a unitary operation for a noiseless quantum computer.
2. Find SΦexp , the PTM of the gate Φexp , using GST, as described in section 6.3. Also determine SΦideal , the
PTM of the ideal gate.

3. Using Eq. M3.1.5 find the process fidelity Fpro (Φexp , Φideal ), where it is assumed that the ideal target
process is unitary. If it is not, then the more general Eq. M3.1.4 needs to be used.

Assumptions and limitations

• Since the measurement procedure is based on GST, the assumptions and limitations for GST in section 6.3
also apply here.

Source code
A tutorial for calculating the single qubit process fidelity for a given gate, also shortened as gate fidelity, is
provided in: the QCMet software repository under:
gate_execution_quality_metrics/gst_based_gate_execution_quality_metrics.

Metric references
[71] B. Schumacher, “Sending Entanglement through Noisy Quantum Channels”, Phys. Rev. A 54, 2614 (1996).
[72] R. Jozsa, “Fidelity for mixed quantum states”, J. Mod. Opt. 41, 2315 (1994).
[77] M. A. Nielsen, “A Simple Formula for the Average Gate Fidelity of a Quantum Dynamical Operation”, Phys.
Lett. A 303, 249 (2002).
[78] M. Horodecki, P. Horodecki, and R. Horodecki, “General teleportation channel, singlet fraction, and quasidistil-
lation”, Phys. Rev. A 60, 1888 (1999).
[80] A. Gilchrist, N. K. Langford, and M. A. Nielsen, “Distance measures to compare real and ideal quantum processes”,
Phys. Rev. A 71, 062310 (2005).
[101] S. T. Merkel, J. M. Gambetta, J. A. Smolin, S. Poletto, A. D. Córcoles, B. R. Johnson, C. A. Ryan, and M.
Steffen, “Self-Consistent Quantum Process Tomography”, Phys. Rev. A 87, 062119 (2013).

M3.2. Diamond norm of a quantum gate

The diamond norm of a quantum gate gives the probability that, given only one application of the gate, the
implemented quantum gate can be distinguished from its ideal target gate [74]. The diamond norm, also known
as completely bounded trace norm, is a norm on the space of quantum operations on density matrices, and is
the metric in which fault-tolerant thresholds are typically presented [100].

Description
Given two different quantum processes, if a quantum state is randomly passed through one of the two processes
and then measured, the maximum probability of successfully determining which of the two quantum processes
was applied is determined by the diamond norm of the difference between the two quantum processes [74].
A quantum gate corresponds to a quantum operation, which is also referred to as a quantum process. The
diamond norm of a quantum gate is calculated by taking the ideal gate and its physically realized noisy quantum

58
M3. Gate execution quality metrics

gate as quantum operations, ΦA and ΦB , respectively. A quantum operation is any operation that takes one
density matrix to another one, and does not need to be unitary. The resulting diamond norm gives the
maximum probability of distinguishing between the ideal gate and its physically realized noisy version in a
single measurement. This is sometimes referred to as obtaining the worst case error of the noisy gate [140, 295].
When comparing to metrics such as the average gate fidelity, the relation with the diamond norm is generally
only loose [571]. Hence computing the diamond norm provides additional independent information from the
fidelity based metrics.
The diamond norm between two quantum operations, ΦA , ΦB ∈ H, is defined as

∥ΦA − ΦB ∥⋄ = max ∥(ΦA ⊗ I)[ρ] − (ΦB ⊗ I)[ρ]∥1 , (M3.2.1)

ρ∈H⊗H
h√ i
where the norm ||X||1 = Tr X ∗ X for some matrix X, I ∈ H is the identity operator, and ρ ∈ H ⊗ H is a
density matrix. The quantum gate operates on the entangled state ρ, which can make it easier to distinguish
between ΦA and ΦB [572]. The density matrix ρ is parameterized, and each element is optimized to maximize
the distance ∥ΦA − ΦB ∥⋄ , whilst also ensuring ρ remains a valid density matrix.
It is possible to compute the diamond norm by first obtaining a full characterization of a quantum hardware
gate using gate set tomography (GST), and then using a classical computer to perform the maximization task
in Eq. (M3.2.1) via a semi-definite program (SDP) [573]. In general, a SDP solves an optimization problem
that is defined by a linear objective function, while also satisfying a set of constraints defined by the problem.
For example, the density matrices need to satisfy the constraint that they are positive semi-definite Hermitian
matrices with trace one. SDP also requires that the solution matrix is positive semi-definite, meaning that its
eigenvalues are greater than or equal to zero.

Measurement procedure
Calculating the diamond norm of a quantum gate requires a complete characterization of the gate. Typically,
this means that the gate estimate is given using gate set tomography (GST), where the output is the Pauli
transfer matrix (PTM). Details on GST and the PTM are given in section 6.3.
To calculate the diamond norm of a quantum gate, the procedure is as follows:
1. Characterize the gate of interest Φexp using GST to obtain the PTM, SΦexp , for the noisy gate. See
section 6.3 for information on how to run GST. Additionally, obtain the PTM for the ideal gate using a
classical computer, SΦideal .
2. Find the Choi matrix, ρdiff , of Sdiff = SΦexp − SΦideal using Eq. M3.1.3 in metric M3.1; ρdiff is a density
matrix in the space H ⊗ H,
3. Run the following SDP, as outlined in section 3.2 in Ref. [573]:
 
1 h † i  †
 I ⊗ ρ0 X
Maximize Tr ρdiff X + Tr ρdiff X , subject to ≥ 0, (M3.2.2)
2 X† I ⊗ ρ1

where ρ0 , ρ1 are density matrices acting on the space H, and X is a linear operator acting on the space
H ⊗ H. Here X, ρ0 , ρ1 are variables that are optimized in the SDP. The SDP also ensures that ρ0 and
ρ1 are valid density matrices, meaning that they must have Tr[ρ] = 1 and that they must be positive
semi-definite Hermitian.
4. The final output of the SDP optimization gives the diamond norm of the quantum gate.

Assumptions and limitations

• For high quality gates the diamond norm of a single gate can be very small, and hence difficult to measure
accurately. In this case it is often more useful to compute ∥ΦnA −ΦnB ∥⋄ , which corresponds to the application
of the gate n times, which amplifies the noise induced errors.
• For larger number of qubits the resources needed for the computation of the diamond norm scale exponen-
tially in the number of qubits. This is due to the scaling of GST which is required to fully characterize the
PTM of the operation of interest and not the classical optimization time using semi-definite programming
which can handle matrices with 1014 entries [574].

59
M3. Gate execution quality metrics

Source code
A tutorial showing how to use gate set tomography to calculate the diamond norm is provided in the QCMet
software repository under:
gate_execution_quality_metrics/gst_based_gate_execution_quality_metrics.
It is based on the open-source PyGSTi software package [282].

Metric references
[74] D. Aharonov, A. Kitaev, and N. Nisan, “Quantum Circuits with Mixed States”, Proc. Thirtieth Annu. ACM
Symp. Theory Comput. Proc. Thirtieth Annu. ACM Symp. Theory Comput. (1998), 20.
[100] R. Kueng, D. M. Long, A. C. Doherty, and S. T. Flammia, “Comparing Experiments to the Fault-Tolerance
Threshold”, Phys. Rev. Lett. 117, 170502 (2016).
[140] A. Y. Kitaev, “Quantum computations: algorithms and error correction”, Russ. Math. Surv. 52, 1191 (1997).
[282] E. Nielsen, K. Rudinger, T. Proctor, A. Russo, K. Young, and R. Blume-Kohout, “Probing quantum processor
performance with pyGSTi”, Quantum Sci. Technol. 5, 044002 (2020).
[295] A. Hashim, S. Seritan, T. Proctor, K. Rudinger, N. Goss, R. K. Naik, J. M. Kreikebaum, D. I. Santiago, and
I. Siddiqi, “Benchmarking quantum logic operations relative to thresholds for fault tolerance”, Npj Quantum Inf.
9, 109 (2023).
[571] Y. R. Sanders, J. J. Wallman, and B. C. Sanders, “Bounding quantum gate error rate based on reported average
fidelity”, New J. Phys. 18, 012002 (2015).
[572] M. M. Wilde, “From classical to quantum Shannon theory”, arXiv:1106.1445 (2011).
[573] J. Watrous, “Simpler semidefinite programs for completely bounded norms”, arXiv:1207.5726 (2012).
[574] A. Yurtsever, J. A. Tropp, O. Fercoq, M. Udell, and V. Cevher, “Scalable Semidefinite Programming”, SIAM J.
Math. Data Sci. 3, 171 (2021).

M3.3. Clifford randomized benchmarking average gate error

The Clifford randomized benchmarking (RB) average gate error metric provides an estimate of the average gate
error of a set of single- and multi-qubit Clifford gates in a quantum computer [81, 82].

Description
RB estimates the average gate error of a set of gates in a way that is robust to state preparation and measurement
(SPAM) errors, giving an estimate on their average performance in a quantum computer. RB runs random
Clifford circuits at increasing depths, where the final Clifford gate is the inverse of all previous operations. A
Clifford gate, denoted as G, is a gate that transforms any Pauli matrix, P , into another Pauli matrix, P ′ , by
GP G† = P ′ . Note that P and P ′ may not necessarily be different. The set of Clifford gates is called the Clifford
group.
The random Clifford gate sequence used within RB is expressed as

G1 G2 G3 . . . Gm−1 Gm Ginverse ,
−1 −1 −1 −1
where each Gi is a Clifford gate, and Ginverse inverts all the previous gates, so that Ginverse = G−1
m Gm−1 . . . G3 G2 G1 ;
m corresponds to the number of Clifford gates applied, also called sequence length or depth. Since all gates
are Clifford gates, Ginverse can be computed efficiently on a classical computer. In the ideal noiseless case,
the RB operations reduce to the identity, and the measurement of the circuit gives the initial state that is set
at the start of the circuit. In a realistic noisy case, the probability of the state to remain in its initial state
decreases with increasing number of operations due to noise in the hardware. This probability is called the
survival probability, psurvival . For example, if after running the RB circuit with an initial state of |0⟩, 80% of
the outcomes are measured to be in the |0⟩ state, then psurvival = 0.8.
Long sequences of random Clifford gates for Nq qubits uniformly sampled from the Clifford group result in
an exponential decay of the survival probability [81]. The decay can then be written in the form of

psurvival = A0 αm + B0 , (M3.3.1)

where A0 , α, and B0 are the fitting parameters corresponding to amplitude, decay, and baseline, respectively.
Here α represents the error per Clifford gate. Using α, the estimate for the Clifford RB average gate error, r,
can be computed as [288]
(1 − α)
r =1−α− , (M3.3.2)
d

60
M3. Gate execution quality metrics

where d = 2Nq is the dimension of the Clifford gates, and Nq is the number of qubits. For the single-qubit case
Nq = 1 and thus d = 2. For the multi-qubit case the random Clifford gates are always of dimension 2Nq , and
are then transpiled into the native gate set to obtain a circuit that can be executed on the hardware.
For multi-qubit RB the set of possible Clifford gates that one needs to sample grows exponentially [93] with
the number of qubits in the multi-qubit operation, which limits scalability. Furthermore, to obtain accurate
results one needs to run a large number of circuits, making it difficult to implement RB on more than a few
qubits, with the largest published result being for three qubits [93]. For a given number of qubits, Nq , the Nq -
qubit Clifford gates need to be decomposed into the native gate-set available on the hardware platform, which
typically consists of single- and two-qubit gates. The resulting circuit depth of an individual Nq -qubit Clifford
gate become very large as Nq increases. Hence, the survival probability can eventually become negligible even
for a single Nq -qubit Clifford gate, so that the decay of the survival probability with increasing number of
Clifford gates cannot be obtained [575].
As a way to run RB on large devices with many qubits, one can run RB on subsets of qubits on the device
in parallel. Note that in this case the number of circuits needed still grows exponentially with the size of the
multi-qubit operations, but does not scale with the total number of qubits in the device. Running RB on
multiple sets of qubits in parallel may introduce additional crosstalk errors due to the qubits being operated in
parallel. This can be used to evaluate the amount of crosstalk errors [88] in the device.

Measurement procedure
1. Select a set of sequence lengths over which the expected survival probability decays significantly.
2. Choose a number ncirc . For each sequence length, m, construct ncirc circuits in the following way. First
generate a sequence of Clifford gates, uniformly randomly selected from the Clifford group. Then add
a last gate to the circuit, which should be the inverse of all previous gates before it, so that the circuit
applied performs an identity operation for an ideal noiseless quantum computer. Note that when running
RB on Nq qubits, then Nq -qubit Clifford gates are required. The Clifford gates are decomposed into the
native gates of the quantum computer at time of execution, which are typically single- and two-qubit
gates.
3. Run each of these RB circuits on hardware with a specified number of shots. The number of shots should
be chosen based on the desired benchmarking precision.
4. Calculate psurvival for each sequence length m.
5. Fit the exponential decay with Eq. M3.3.1 and obtain the error per Clifford gate, α.
6. Determine the Clifford RB average gate error, r, using Eq. M3.3.2.
An example of the results one might obtain is shown in Fig. M3.3.1.

Assumptions and limitations

• RB for Clifford gates only spans a subset of all possible quantum gates. Clifford gates can be simulated
efficiently using a classical computer. This may not be fully representative of the quantum computer
performance, as gates from the Clifford group do not form a universal gate set.
• When implementing RB experimentally, the Clifford gates need to be decomposed into the native gates
of the device. There can be differences in reported errors depending on the decomposition of the Clifford
gates. To estimate the average gate error of native gates after obtaining the RB average gate error r, one
needs to calculate the average number of native gates required to decompose Clifford gates.
• Clifford RB itself has limited scalability. As discussed above, modifications with improved scalability are
required for large numbers of qubits.
• When benchmarking larger systems, decomposing the Nq -qubit Clifford requires increasingly more two-
qubit gates. The error per Clifford increases, so that the survival probability decays more rapidly and is
only non-zero for a small number of layers, leading to larger uncertainties in fitting the decay parameters.
• This protocol assumes that all noise is Markovian and gate independent [82, 231]. Under this assumption,
the Clifford RB gate error is linearly related to the process fidelity averaged over the Nq -qubit Clifford
group [98]. In practice, the noise from a gate application can depend on gates applied before it or have
some time-dependent noise and the relationship between the Clifford RB gate error and the process fidelity
averaged over the Nq -qubit Clifford group becomes less clear [98]. Additionally, if the survival probability
deviates from the exponentially decay described in the description, this may indicate the presence of
non-Markovian noise [98, 576].

61
M3. Gate execution quality metrics

Figure M3.3.1.: Results of emulator runs of 1-qubit Clifford randomized benchmarking (RB) using the methodology
outlined in this section and the noise model specified in Sec. 6.1. The results are shown as a violin plot. The gray regions
use kernel density estimation to show the distribution of survival probabilities obtained from the ncirc circuits run for
each sequence length of Clifford gates, m, where the average survival probability for each sequence length is marked with
a cross. Here, ncirc = 10. The black dashed curve shows the fitted Eq. M3.3.1, from which the Clifford RB average gate
error, r, is calculated using Eq. M3.3.2 to obtain r = 0.0018.

• The RB protocol assumes that the average error converges to the average over all possible Clifford se-
quence for small sample sizes because only a small fraction of possible sequence lengths are able to be
implemented [89].
• The protocol assumes there is not much variation in errors so that most errors are close to the average [89].

Source code
A tutorial demonstrating how to calculate the Clifford RB average gate error for a single qubit is provided in
the QCMet software repository under:
gate_execution_quality_metrics/randomized_benchmarking/clifford_randomized_benchmarking.

Metric references
[81] J. Emerson, R. Alicki, and K. Życzkowski, “Scalable Noise Estimation with Random Unitary Operators”, J. Opt.
B Quantum Semiclassical Opt. 7, S347 (2005).
[82] E. Knill, D. Leibfried, R. Reichle, J. Britton, R. B. Blakestad, J. D. Jost, C. Langer, R. Ozeri, S. Seidelin, and
D. J. Wineland, “Randomized Benchmarking of Quantum Gates”, Phys. Rev. A 77, 012307 (2008).
[88] J. M. Gambetta et al., “Characterization of Addressability by Simultaneous Randomized Benchmarking”, Phys.
Rev. Lett. 109, 240504 (2012).
[89] J. M. Epstein, A. W. Cross, E. Magesan, and J. M. Gambetta, “Investigating the limits of randomized bench-
marking protocols”, Phys. Rev. A 89, 062321 (2014).
[93] D. C. McKay, S. Sheldon, J. A. Smolin, J. M. Chow, and J. M. Gambetta, “Three-Qubit Randomized Bench-
marking”, Phys. Rev. Lett. 122, 200502 (2019).
[98] J. Helsen, I. Roth, E. Onorati, A. H. Werner, and J. Eisert, “General Framework for Randomized Benchmarking”,
PRX Quantum 3, 020357 (2022).
[231] P. Figueroa-Romero, K. Modi, R. J. Harris, T. M. Stace, and M.-H. Hsieh, “Randomized Benchmarking for
Non-Markovian Noise”, PRX Quantum 2, 040351 (2021).
[288] E. Magesan, J. M. Gambetta, and J. Emerson, “Scalable and Robust Randomized Benchmarking of Quantum
Processes”, Phys. Rev. Lett. 106, 180504 (2011).
[575] C. H. Baldwin, B. J. Bjork, J. P. Gaebler, D. Hayes, and D. Stack, “Subspace benchmarking high-fidelity entangling
operations with trapped ions”, Phys. Rev. Res. 2, 013317 (2020).
[576] J. J. Wallman, “Randomized benchmarking with gate-dependent noise”, Quantum 2, 47 (2018).

62
M3. Gate execution quality metrics

M3.4. Interleaved Clifford randomized benchmarking gate error

The interleaved Clifford randomized benchmarking (RB) gate error provides an estimate for the average error
of a target Clifford gate in a gate set [90, 91].

Description
As described in metric M3.3, Clifford RB provides the average gate error over all of the gates in the gate set
used to decompose the Clifford gates. In contrast, interleaved Clifford RB characterizes the average error of a
specific Clifford gate, which is interleaved through the random Clifford sequences used in RB. If the RB random
Clifford gate sequence is expressed as (see section M3.3)

G1 G2 G3 . . . Gm−1 Gm Ginverse ,

where each G is a Clifford gate, then the interleaved RB sequence can be written as
′
GCinverse , G1 Gtarget G2 Gtarget G3 Gtarget . . . Gm−1 Gtarget Gm Gtarget G′inverse ,

where Gtarget is the gate to characterize and must also be part of the Clifford group. Additionally, the final
inverting gate must be updated to invert the full gate sequence including Gtarget . The specific gate error is
obtained by comparing the decay parameters in Eq. M3.3.1 for both the non-interleaved and interleaved RB
results.
The fitted decay parameters from the non-interleaved RB and the interleaved RB are denoted by α and αG ,
respectively. Then, the interleaved RB gate error for Gtarget is found using [90, 91]

(d − 1) 1 − ααG


rGtarget = , (M3.4.1)
d
where d = 2Nq , and Nq is the number of qubits on which the randomized benchmarking is performed. For the
single-qubit case Nq = 1 and thus d = 2.

Measurement procedure
1. Select the target gate, Gtarget , for which the average gate error needs to be estimated. Then, select a set
of sequence lengths over which the expected survival probability decays significantly.

2. Generate the circuits for the varying depths used to obtain the non-interleaved Clifford RB average gate
error, as outlined in metric M3.3.
3. For each sequence length generate another sequence of Clifford gates, uniformly randomly selected from
the Clifford group, but this time with the target gate interleaved in each sequence. The last gate should
be the inverse of all previous gates before it, so that the circuit applied acts as an identity operation for
a noiseless quantum computer.
4. Execute each of these non-interleaved and interleaved RB sequences on hardware.
5. Calculate the survival probability for each sequence length for both.
6. Fit the exponential decay to model the data for the non-interleaved and for the interleaved RB gate
sequences using Eq. M3.3.1.
7. Convert decay parameters extracted from the fits into Clifford RB average gate error, rGtarget , using
Eq M3.4.1.

Assumptions and limitations

• Interleaved Clifford RB has the same assumptions and limitations as for the Clifford RB average gate
error estimation (see metric M3.3).
• The protocol may underestimate the interleaved gate error if the error on the interleaved gate partially
inverts the error on the previous Clifford gate [89].

63
M3. Gate execution quality metrics

Source code
A tutorial demonstrating how to calculate the interleaved Clifford randomized benchmarking gate error is pro-
vided in the QCMet software repository under:
gate_execution_quality_metrics/randomized_benchmarking/interleaved_clifford_randomised_benchmarking.

Metric references
[89] J. M. Epstein, A. W. Cross, E. Magesan, and J. M. Gambetta, “Investigating the limits of randomized bench-
marking protocols”, Phys. Rev. A 89, 062321 (2014).
[90] J. P. Gaebler et al., “Randomized Benchmarking of Multiqubit Gates”, Phys. Rev. Lett. 108, 260503 (2012).
[91] E. Magesan et al., “Efficient Measurement of Quantum Gate Error by Interleaved Randomized Benchmarking”,
Phys. Rev. Lett. 109, 080505 (2012).

M3.5. Cycle-benchmarking composite process fidelity

The cycle-benchmarking (CB) composite process fidelity quantifies the total effect of errors when a full clock
cycle of specified gates is applied on a quantum computer [293]. A clock cycle refers to the application of a
specified sequence of gates acting on a given number of qubits, and may consist of single and multi-qubit gates.

Description
CB evaluates the performance of a quantum hardware in performing full cycles of operations. The specific
sequence of the gates in such a cycle is specified by the user, and must be reported by the user together with
the obtained composite process fidelity. A given quantum process performed with a cycle in CB, denoted as G,
consists of Clifford operations, and has the property that when repeated sufficient times it returns the identity
for an ideal noiseless quantum computer. Therefore, if the cycle for an ideal operation is denoted as Ḡ, one has
Ḡ m = I, where m is an integer number specifying the number of repetitions of the cycle. The CB composite
process fidelity can then be written as Fpro (G, Ḡ). The process fidelity is presented in detail in Sec. M3.1, and
given in mathematical form in Eq. M3.1.5. In order to obtain Fpro (G, Ḡ) the Pauli transfer matrix (PTM) of
G needs to be estimated. The procedure to obtain the PTM using gate set tomography (GST) is presented in
Sec. 6.3. Since the evaluation of the PTM using GST is very time-consuming, especially for larger number of
qubits, within CB a number of approximations are performed, which allow for improved efficiency and increased
scalability. To emphasize that the CB composite process fidelity is based on a defined set of approximations, it
is denoted as FCB (G, Ḡ) to distinguish it from the exact fidelity.
The main component that CB uses to increase the efficiency and scalability is the inclusion of randomized
compiling (RC). Within RC one applies random single qubit Pauli gates to all qubits before each application
of a target gate sequence, in this case each individual G, and then applies single qubit Pauli gates afterwards,
such that the overall unitary remains unchanged for the noiseless case [298]. When using RC one generates a
set of L randomized circuits. It can be shown that when evaluating expectation values averaging over all the L
randomized gate sequences, RC converts noise with arbitrary coherence and spatial correlations into stochastic
Pauli noise [295]. RC therefore ensures that off-diagonal noise in the PTM is transformed to diagonal noise.
This is referred to as noise tailoring. Since the noiseless cycle satisfies Ḡ m = I, its PTM is purely diagonal.
Importantly, the addition of stochastic Pauli noise to such a diagonal PTM keeps the PTM diagonal. The larger
the value of L, the better the noise tailoring performs, and hence the closer the PTM is to a diagonal form.
The key advantage of using RC within CB therefore is that it makes the PTM of the noisy quantum process
of G m diagonal, so that only the diagonal elements need to be computed. Using Eq. M3.1.5, and the fact that
Ḡ m = I, one obtains the fidelity of G m as

FCB (G m , Ḡ m = I) = Tr [SG m ] . (M3.5.1)

where SG m is the PTM of G m when using RC.

Within CB one therefore only needs to compute the diagonal elements of the PTM of G m . To this aim, a loop
is performed over all initial states that span to eigenbasis of the Nq -qubit tensor products of Pauli matrices.
These Nq -qubit Pauli strings are composed of a single arbitrary Pauli gate on each qubit, including the identity,
so that the set of all Pauli strings can be written as P = {X, Y, Z, I}⊗Nq . For a given Pauli string, Pi , the
initial state is set so that on each qubit it corresponds to the eigenstate with +1 eigenvalue of the Pauli matrix
for that specific qubit. This is prepared by applying an appropriate Clifford gate to the initial zero state on
that qubit. Once the appropriate initial state is prepared on the quantum computer for a given Pi , a RC
based implementation of G m is applied in the circuit, which requires running L different randomized sequences

64
M3. Gate execution quality metrics

of gates as described above. For each of these circuits, the inverse of the state preparation is applied before
measurement in the Z basis on all qubits. For each randomized RC circuit with index l ∈ [1, L], a number of
shots is performed to evaluate the expectation value of these measurements. The diagonal element of the PTM
of G m for the given Pi , denoted as fPi ,m,l , is then equal to the parity of each of the measurement outcomes
averaged over the number of shots. The parity is calculated by first assigning a value of +1 (−1) to each qubit
measurement outcome of zero (one), with the exception for those qubits that have an identity operation as part
of the Pi Pauli string, for which the assigned value is always +1. These assigned values are then multiplied over
all qubits to obtain the overall parity. By averaging the obtained parities over all the L randomized circuits one
obtains an estimate of fPi ,m as
L
1X
fPi ,m = fPi ,m,l , (M3.5.2)
L
l=1

so one obtains the composite process fidelity of m applications of G using

1 X
FCB (G m , Ḡ m = I) = fPi ,m . (M3.5.3)
NP
Pi ∈P

The fPi ,m computed in this way include the effects of state preparation and measurement (SPAM) errors,
which is undesirable. To filter out such effects at an approximate level, one can compute these elements for two
different values of m, and use the fact that the fidelity for such cycles is expected to decay exponentially with
increasing m. The robustness to SPAM errors is therefore achieved by running the cycle for different number of
repetitions, and using the fact that the reduction in fidelity increases for increasing m, while the SPAM induced
infidelity remains approximately constant. One can therefore approximate the diagonal element of the PTM
for a single application of G as
  1
fPi ,m2 m2 −m1
fPi = , (M3.5.4)
fPi ,m1
where m2 > m1 , and the values are chosen in a way that Ḡ m1 = Ḡ m2 = I. Note that one can also use multiple
values of m and then fit the results as function of m to an exponential decay:

fPi (m) = ae−bfPi ,m + c, (M3.5.5)

where a, b and c are fitting parameters, and they must be chosen such that when m = 0, fPi (m = 0) = 1. Then
the diagonal element of the PTM for a single application of G is given by:

fPi = fPi (m = 1). (M3.5.6)

The overall CB composite process fidelity of a single application of G is obtained by averaging over all diagonal
elements of the PTM:
1 X
FCB (G, Ḡ) = fPi . (M3.5.7)
NP
Pi ∈P
Nq
Since the number of Pauli strings, NP , scales as NP ≤ 4 , it becomes infeasible to include all Pauli strings in
the sum as the number of qubits becomes large. As an approximation one therefore uses a uniform sampling of
Pauli strings at random to produce a set with a user defined number of elements. This generally gives a good
approximation of a sum over all Pauli strings, provided a sufficiently large number of elements is sampled at
random [293].

Measurement procedure
1. Select a set of Nq -qubit Pauli strings, P, with the number of elements denoted by NP . The elements of P
determine which diagonal elements of the PTM are included in the sum to obtain the total CB combined
process fidelity.

2. Select the cycle of Clifford gates G to be tested. Either select two lengths, m1 and m2 , such that m2 > m1 ,
or select a number of lengths {m1 , m2 , · · · , mk }, where mj+1 > mj ∀j ∈ {1, 2, · · · , k − 1}. For all selected
lengths m, Ḡ m = I must be satisfied.
3. Select the number of randomizations, L, to apply for each Pauli string.

4. For each Pi ∈ P, and each length m ∈ {m1 , m2 } or {m1 , m2 , · · · , mk }, repeat the following steps for
l ∈ {1, · · · , L}:

65
M3. Gate execution quality metrics

a) Select m + 1 random Nq -qubit Pauli strings R0 , R1 , · · · , Rm , to define the randomized circuit, so

that Ri′ ḠRi = Ḡ, where Ri′ is defined through Ri and Ḡ. This then defines the randomized circuit
CP,m,l (G) as
CP,m,l (G) = B(Pi )† Rm
′ ′
GRm Rm−1 GRm−1 · · · R1 R0′ GR0 B(Pi ), (M3.5.8)
where B(Pi ) is the operation to transform the initial zero state into the +1 eigenstates on each qubit
of the Pauli string Pi .
b) Run the circuit on the quantum computer for a specified number of shots, and use the circuit outcomes
to calculate the Pauli expectation fP,m,l . For each shot, the parity of the measurement outcomes
bitstrings over all the qubits of CP,m,l (G) is computed, and the results averaged over all shots.
5. Calculate fPi ,m using Eq. M3.5.2 for all Pi and m.
6. If two lengths, {m1 , m2 }, were used, then use Eq. M3.5.4 to calculate fPi . If more than two lengths were
used, then use Eqs. M3.5.5 and M3.5.6 to calculate fPi .
7. Calculate the composite process fidelity using Eq. M3.5.7.

An example of the results one might obtain is shown in Fig. M3.5.1.

Figure M3.5.1.: Results of an emulator run with the methodology outlined in this section to obtain the cycle-
benchmarking composite process fidelity, where the noise model specified in Sec. 6.1 is used. Here the cycle consists of
a single CNOT gate such that G = CX . The cross symbols indicate the calculated diagonal element of the PTM, fPi ,m ,
from the emulator results, for each number of cycles m. The gray dashed line is the fit to the data using Eq. M3.5.5.
Then Eqs. M3.5.6 and M3.5.7 are used to obtain the composite process fidelity FCB = 0.9781 for the cycle consisting of
a single CX gate.

Assumptions and limitations

• The cycle tested needs to be composed of Clifford gates, and furthermore needs to obey the condition
G m = I for some m. Generalizations that lift these restrictions may be possible by selecting random
gates before the cycle and applying the corresponding gates that leave the cycle unitary unchanged using
randomized compiling [298].
• The noise is assumed to be Markovian (see metric M2.3), meaning that it is not dependent on time; it is
also assumed that the noise on Pauli gates is independent of the cycle.

Source code
A tutorial showing how to create and run the CB circuits and estimate the CB composite process fidelity is
provided in the QCMet software repository under:
gate_execution_quality_metrics/cycle_benchmarking_composite_process_fidelity.

66
M3. Gate execution quality metrics

Metric references
[293] A. Erhard, J. J. Wallman, L. Postler, M. Meth, R. Stricker, E. A. Martinez, P. Schindler, T. Monz, J. Emerson,
and R. Blatt, “Characterizing Large-Scale Quantum Computers via Cycle Benchmarking”, Nat. Commun. 10,
5347 (2019).
[295] A. Hashim, S. Seritan, T. Proctor, K. Rudinger, N. Goss, R. K. Naik, J. M. Kreikebaum, D. I. Santiago, and
I. Siddiqi, “Benchmarking quantum logic operations relative to thresholds for fault tolerance”, Npj Quantum Inf.
9, 109 (2023).
[298] J. J. Wallman and J. Emerson, “Noise Tailoring for Scalable Quantum Computation via Randomized Compiling”,
Phys. Rev. A 94, 052325 (2016).

M3.6. Over- or under-rotation angle

For gates corresponding to single qubit rotations, this metric quantifies the amount of over- or under-rotation
compared to the target rotation angle.

Description
Single qubit unitary gates correspond to rotations on the Bloch sphere, Ru (θ), where u is the rotation axis, and
θ is the rotation angle. Due to gate mis-calibration, drifts present in the device, and limitations in the precision
of gate calibration, the actual rotation angle, θ′ , of the gate is usually different from the target rotation angle, θ,
such that θ′ = θ +θerr , where θerr is the amount of over or under rotation. The value of θerr may be small, so that
repeated gates are necessary to amplify the rotation errors in order to reliably quantify them [92]. To quantify
the amount of over- or under-rotation one can consider angles θ = 2π n , so that n repeated applications of Ru ,
corresponding to Run , lead to a pseudo-identity operation for the ideal noiseless operation, since it corresponds
to a total ideal rotation amount of 2π.

Measurement procedure
In order to extract the rotation error, a number of different circuits are run, where each applies a different
number of pseudo-identities. The number of pseudo-identities applied, m, starts from 0 and increases with step
size ∆m, up to the maximum number of pseudo-identities to apply, mmax . It must also be ensured to prepare
the qubit in a state that is orthogonal to the rotation axis of Ru . For example, one can prepare the √ qubit in
the |0⟩ state for gates corresponding to rotations about the x or y axes, and in the |+⟩ = (|0⟩ + |1⟩)/ 2 state
for rotations about the z axis. This maximises the amplitude of the oscillations that will occur due to the over
or under rotations, allowing for more precise estimation of the error.
Each circuit is executed for a chosen number of shots in order to find the probability of measuring the qubit
in the |0⟩ state after application of m pseudo-identities, which is denoted as p0 (m). An equation for p0 (m)
in the presence of over/under-rotation can be derived [92], and then generalized to include the effects of state
preparation and measurement errors, as well as decoherent errors. This equation is given by
1 1
p0 (m) = + a + (1 − b)e−λ m cos(m θerr ). (M3.6.1)
2 2
Here, the fitting parameters a, b include the effects of state preparation and measurement errors, since these
errors affect the vertical offset and the maximum amplitude of the oscillation decay envelope. The fitting
parameter λ includes the effect of decoherent errors, and the fitting parameter θerr corresponds to the over- or
under-rotation error amount. All parameters a, b, λ, and θerr are 0 in the absence of noise. Since θerr effectively is
determined by an oscillation frequency, its characterization from experimental data is affected little by the exact
values of the other parameters. State preparation and measurement errors, as well as decoherent errors, can in
principle also be extracted using this equation, although one typically uses dedicated metrics and measurement
procedures that are aimed to quantify them (see metrics M2.1, M2.2, M3.7).
The measurement procedure then is as follows:
1. To characterize the rotation error for a single-qubit rotation gate, Ru , select ∆m, mmax , and n. This
defines the set of circuits to run, where each pseudo-identity is given by Run , and the set of circuits to run
with differing pseudo-identities is given by M = {0, ∆m, 2∆m, ..., mmax }. Additionally, select a state |ψ⟩
that is orthogonal to the eigenstates of Ru , and execute the operation Uψ that prepares this state such
that |ψ⟩ = Uψ |0⟩.
2. For each m ∈ M :
a) Prepare the qubit in the state |ψ⟩ by applying Uψ to a qubit initially in state |0⟩.

67
M3. Gate execution quality metrics

b) Apply gate Ru a total of n × m times.

c) Invert the state preparation operation by applying Uψ† and measure the qubit in the computational
basis.
d) Repeat steps 2a-2c for a specific number of shots in order to estimate the probability of measuring
the qubit in the 0 state, p0 (m).
3. Fit the results to the function in Eq. M3.6.1 to extract the over or under rotation error amount θerr .
4. To determine whether the rotation error is an under- or over-rotation, proceed as follows:
a) Using the fit in the previous step, select meq , the number of pseudoidentities at which p0 first reaches
the mean or equilibrium point of the first full oscillation.
b) Run the circuit as before, preparing the qubit in the state |ψ⟩, and then applying Ru a total of
n × meq times.
⃗ be
c) Let ⃗u be the vector in the Bloch sphere corresponding to the positive eigenstate of Ru , and let ψ
the vector corresponding to |ψ⟩. Evaluate the state |ϕ⟩ which is defined as the state corresponding
to the Bloch vector ϕ ⃗ Apply the operation U † , where Uϕ is defined such that Uϕ |0⟩ = |ϕ⟩.
⃗ = ⃗u × ψ.
ϕ
Finally, measure the qubit in the computational basis.
d) Repeat the above and define pϕ as the probability of obtaining the |0⟩ state upon measurement. A
value of pϕ > 0.5 corresponds to an over-rotation error while pϕ < 0.5 corresponds to an under-
rotation error.
In practice, ∆m must be chosen to be small enough to avoid aliasing effects in oscillations, and mmax must be
large enough to either see at least half an oscillation, or to ensure P0 (m) has reached its steady state value.
An example of the results one might obtain is shown in Fig. M3.6.1.

Figure M3.6.1.: Results of emulator runs with the methodology outlined in this section to obtain the amount of
over- or under-rotation, using the noise model specified in Sec. 6.1. Here the pseudo-identity gates are implemented by
applying an Rx (π/4) four times. The crosses indicate the measured probabilities of the |0⟩ state, p0 , at each number of
pseudo-identities applied, m. The oscillation frequency is found by fitting Eq. M3.6.1 to the data. Then using step 4
in the measurement procedure, the rotation error is determined to be an over-rotation. The calculated over-rotation is
found to be θerr = 0.01π. This is in agreement with the theoretical value as specified in the noise model.

Assumptions and limitations

• In the fitting equation it is assumed that p0 (m) has at most a single frequency oscillation. In the presence
of large amounts of non-Markovian noise (see metric M2.3), this assumption is often violated [223].
• The coherence time of the qubit sets a bound on the smallest rotation error that can be detected. If
no oscillations are detected, and only a decay of p0 (m) to its steady state is observed, it suggests that
rotation error is smaller than this bound.
• The rotation axis u is typically chosen such that the rotation is around the x, y, or z axis. This means
that the single-qubit rotation gates typically characterized are the Rx , Ry , and the Rz gates.

68
M3. Gate execution quality metrics

Source code
A tutorial showing how to calculate the over- and under-rotation errors for an Rx rotation gate is provided in
the QCMet software repository under:
gate_execution_quality_metrics/over_under_rotations.

Metric references
[92] S. Sheldon, L. S. Bishop, E. Magesan, S. Filipp, J. M. Chow, and J. M. Gambetta, “Characterizing Errors on
Qubit Operations via Iterative Randomized Benchmarking”, Phys. Rev. A 93, 012301 (2016).
[223] A. Agarwal, L. P. Lindoy, D. Lall, F. Jamet, and I. Rungger, “Modelling non-markovian noise in driven super-
conducting qubits”, Quantum Sci. Technol. 9, 035017 (2024).

M3.7. State preparation and measurement fidelity

The state preparation and measurement (SPAM) fidelity measures how well qubits can be initialized and
measured [103]. The SPAM operations are represented by the density matrix of the initial state, ρinit , and by
the measurement projectors, {Mi }. The SPAM fidelity measures how close the SPAM operations, {ρinit , {Mi }},
are to the ideal initial states and measurement projectors.

Description
SPAM, which makes up two of the five criteria for the physical realization of a quantum computer [172], directly
impacts the performance of quantum computers. The method to determine the SPAM fidelity is based on the
methodology to obtain general process and state fidelities, outlined in section M3.1.
The state preparation errors occur due to the non-ideality of the physical realization of quantum computers.
For example, the state of a qubit after initial preparation, usually its equilibrium state, may not fully be the
generally targeted |0⟩ state, but instead may have a small chance of being in the |1⟩ state. Similarly, when the
measurement errors occur, it means that the wrong outcome is reported by the measurement. For example, the
measurement may report a |0⟩ even when the qubit is in the |1⟩ state. The SPAM fidelity is a measure of how
close the SPAM operations are to their ideal operations.
For a single qubit, the SPAM components are ρinit , M0 , and M1 , where ρinit is the initial density matrix,
and M0 is the measurement projector to measure the qubit in the |0⟩ state, and M1 is the projector to measure
the qubit in the |1⟩ state. The ideal SPAM components for the single qubit case are denoted by ρ̄init , M̄0 ,
M̄1 . Typically the initial state of the qubit is set to the |0⟩ state, and, in this case, the ideal noiseless SPAM
operations are given by the following [163]:
     
1 0 1 0 0 0
ρ̄init = , M̄0 = , M̄1 = , (M3.7.1)
0 0 0 0 0 1

The number of measurement projectors is dependent on the number of possible outcomes from measurement.
For a single qubit, there are two outcomes, and hence there are two measurement projectors. As the number
of qubits increases, the number of measurement projectors increase as 2Nq , where Nq is the number of qubits.
Given all of the SPAM components, there are two fidelities that constitute SPAM fidelity: the fidelity of the
initial density matrix, Fρinit , and the measurement fidelity FM , which is calculated using estimates of {Mi } [282].
The SPAM fidelity is different to what is often referred to as readout fidelity, Fr , which gives a fidelity estimate
solely based on mis-classification of readout [577].

Measurement procedure
1. Using gate set tomography (GST) on the device of interest, obtain an estimate of the Pauli transfer ma-
trices of the device gate set (see Sec. 6.3). This GST gate set estimate includes estimates of the state
preparation, characterized by the initial density matrices ρ0 , and the measurement projectors, character-
ized {Mi } . For a singe qubit one has only two measurement projectors, {M0 , M1 }.
2. Using these estimates, the fidelity of the initial density matrix can be calculated using Eq. M3.1.2. The
measurement fidelity is computed by transforming the measurement projectors into a gate-like quan-
tity [282]. Then one can compute the measurement fidelity using Eq. M3.1.5.

Fr , on the other hand, is obtained by preparing all possible non-superposition states in the computational
basis, and then measuring and recording the probability of each outcome. This then gives a matrix, where
each row corresponds to the prepared state, and each column corresponds to the measured state. The readout

69
M4. Circuit execution quality metrics

fidelity is the trace of this matrix. For example, for a single qubit system, one prepares the |0⟩ and the |1⟩
state, and then immediately measures it to find the outcome probabilities. This gives a 2 × 2 matrix, which has
the elements P (0|0), P (1|0), P (0|1), and P (1|1). The readout fidelity is then given by Fr = P (0|0) + P (1|1).
There are limitations to this method, as it assumes the state preparation to be perfect, and as a result, Fr also
includes the state preparation error in the estimate. The SPAM estimate obtained using GST overcomes this
limitation.

Assumptions and limitations

• Since the measurement procedure is based on GST, the assumptions and limitations for GST (section 6.3)
also apply here.

Source code
A tutorial for calculating the SPAM fidelity using gate set tomography is provided in: the QCMet software
repository under:
gate_execution_quality_metrics/gst_based_gate_execution_quality_metrics.

Metric references
[103] E. Nielsen, J. K. Gamble, K. Rudinger, T. Scholten, K. Young, and R. Blume-Kohout, “Gate Set Tomography”,
Quantum 5, 557 (2021).
[163] M. A. Nielsen and I. Chuang, “Quantum Computation and Quantum Information”, Camb. Univ. Press 2010.
[172] D. P. DiVincenzo, “The Physical Implementation of Quantum Computation”, Fortschr. Phys. 48, 771 (2000).
[282] E. Nielsen, K. Rudinger, T. Proctor, A. Russo, K. Young, and R. Blume-Kohout, “Probing quantum processor
performance with pyGSTi”, Quantum Sci. Technol. 5, 044002 (2020).
[577] L. Chen, H.-X. Li, Y. Lu, C. W. Warren, C. J. Križan, S. Kosen, M. Rommel, S. Ahmed, A. Osman, J. Biznárová,
et al., “Transmon qubit readout fidelity at the threshold for quantum error correction without a quantum-limited
amplifier”, Npj Quantum Inf. 9, 26 (2023).

M4.Circuit execution quality metrics

M4.1. Quantum volume

The quantum volume is a metric for the overall capabilities of a noisy quantum computer. It follows the
volumetric benchmarking framework (see section 6.2), where the number of layers in the quantum circuit is
set equal to the number of qubits. Quantum volume is defined as 2npass , where npass is the largest number
of qubits for which a type of randomized circuits can be successfully run. The success criterion is that the
heavy output probability, ph , is greater than 2/3 [109]. The heavy output probability for a circuit is obtained
by first performing noiseless emulator runs for the purpose of finding all the measurement outcomes that have
probabilities greater than the median of the measurement outcomes. These are collected into a set of selected
measurement outcomes. The heavy output probability for a given quantum hardware then is the sum of the
measurement probabilities of this set of outcomes when the circuit is executed on the experimental device.
Evaluation of this quantity requires to run both the experiments on the quantum hardware and the analogous
emulator runs for noiseless quantum circuits on classical computers, and hence scalability is limited by the
maximum system sizes accessible in the emulator runs.

Description
Volumetric benchmarking metrics are defined by a task to solve, which determines the type of quantum circuits
to run, and by a success criterion. For the quantum volume, VQ , the task to solve is motivated by the work in
Ref. [578], which proposes a method to demonstrate quantum advantage through sampling randomized quantum
circuits. The circuit structure is illustrated in Fig. M4.1.1. There are Nq qubits, and Nq layers of gates that
create a square circuit. Each layer consists of an Nq -qubit gate that randomly changes the order of qubits,
followed by a layer of two-qubit gates that apply general two-qubit operations between nearest neighbor qubits.
Both the permutations and two-qubit operations are sampled randomly, where the two-qubit gates are sampled
from a Haar distribution.

70
M4. Circuit execution quality metrics



 |0⟩ ···
SU(4) SU(4) SU(4)






 |0⟩ ···




|0⟩ ···




SU(4) SU(4) SU(4)


Nq |0⟩ PERM PERM ··· PERM




 .. .. .. .. .. .. ..
 . . . . . . .




|0⟩ ···




SU(4) SU(4) SU(4)




|0⟩ ···

| {z }| {z } | {z }
1 2 Nq

Figure M4.1.1.: Circuit diagram indicating the structure of a quantum volume circuit. There are Nq qubits, and Nq
layers of gates, creating a square circuit. Each layer consists of a Nq -qubit gate that randomly changes the order of
qubits, denoted as PERM gate, and a column of Haar-random two-qubit gates, denoted as SU(4).

For each circuit, ph is obtained as follows. For a given Nq -qubit circuit, obtain the ideal probabilities of all
the possible 2Nq output bitstrings using noiseless simulations. Then sort these probabilities, and collect the half
of the bitstrings that have the ideal probability greater than the median ideal probability: these are called the
heavy outputs. The sum of the probabilities of heavy outputs is ph . For the heavy output bitstrings obtained
for the noiseless emulators, also calculate the total probability obtained from the experimental measurements
on the hardware. The process is repeated for each circuit in the set of randomly selected circuits, and the
experimentally measured ph is averaged over all the circuits to obtain ⟨ph ⟩. It is shown in Ref. [578] that for
the types of circuits considered here, ⟨ph ⟩ for a noiseless quantum computer needs to be larger than 2/3. The
success criterion is therefore set to be that ⟨ph ⟩ > 2/3. If the qubits on a noisy quantum computer have fully
decohered due to noise, then the output probabilities of the quantum circuits converge to an approximately
uniform distribution, for which ⟨ph ⟩ = 1/2.
The VQ is then defined as [109]

VQ = 2npass , (M4.1.1)
npass = argmax min(Nq , d(Nq )), (M4.1.2)
Nq

where Nq is the number of qubits, and d(Nq ) is the largest number of circuit layers at which an Nq -qubit circuit
can produce heavy outputs with probability greater than 2/3. Due to taking the minimum of Nq and d(Nq ), it
suffices to test only Nq -qubit square circuits, which are constructed with Nq layers.
This description follows the definition of quantum volume in Ref. [109]. This is a redefinition of the quantum
volume metric introduced in Ref. [579].

Measurement procedure
1. Generate m ≥ 100 circuits for Nq qubits. Each layer consists of a permutation block, and then a block
of Haar-random two-qubit gates, as illustrated in Fig. M4.1.1. The number of layers must be equal to
the number of qubits. The manner in which the random permutations are achieved depends on qubit
connectivity. For example, for a superconducting qubit device, a block of SWAP gates is required for the
qubit order permutations, whereas for a trapped-ion device with all-to-all connectivity, no SWAP gates
are necessary.
2. Run noiseless simulations for these circuits to find the heavy output bitstrings.
3. Run the m circuits, and for each circuit calculate ph .
4. Compute the ⟨ph ⟩ across all m circuits. If it is greater than 2/3 with a confidence of at least 2 standard
deviations, meaning that at least 97.5% of heavy output probabilities from all circuits fall above 2/3, then
the device passes the test, and hence has a quantum volume of at least 2Nq . If it is smaller, then the
quantum volume is less than 2Nq .
5. If the test in the previous point fails, then test for a smaller n until it passes. If the test passes, keep using
successively larger Nq until the test fails. The quantum volume, VQ , is then determined by the largest Nq
that did not fail, npass , so that VQ = 2npass .

71
M4. Circuit execution quality metrics

An example of the results one might obtain is shown in Fig. M4.1.2.

Figure M4.1.2.: Results of emulator runs using the measurement procedure outlined in this section to obtain the
quantum volume using the noise model specified in Sec. 6.1. (a) A histogram is plotted horizontally, showing the
distribution of the heavy output probabilities, ph , for Nq = 4 and m = 200. The black line represents ⟨ph ⟩, the magenta
dashed line shows the confidence bound of the mean with 2 standard deviations, and the black dashed line shows the
threshold of 2/3. The device fails the test as ⟨ph ⟩ with a confidence of 2 standard deviations is less than the required
value of 2/3, that is, the magenta dashed line is below the black dashed line. (b) This violin plot shows the distribution of
heavy output probabilities for the m circuits for increasing Nq . The distribution of results is plotted using kernel density
estimation. The black dashed line indicates the passing criteria of 2/3. The black cross represents ⟨ph ⟩ output and the
magenta star symbol represents ⟨ph ⟩ with a confidence of 2 standard deviations. This shows that for the emulator with
the specified noise model in Sec. 6.1, when Nq ≥ 4 it fails the quantum volume test, therefore its quantum volume is
VQ = 23 = 8.

Assumptions and limitations

• Computing the quantum volume requires classical simulation of the circuits, which have a computational
cost that scales exponentially with qubit number; this puts a limit on the size of the systems that can be
run at around 30 qubits [322].

• The randomly generated circuits are typically passed to a compiler to optimize them for each specific
hardware platform. The optimized circuits should be provided together with the original unoptimized
circuits in order to ensure reproducibility.
• The quantum volume protocol is constructed from random circuits, and as such, the effect of coherent
errors in the quantum volume test may not be representative of the effect of coherent errors in certain
quantum algorithms [322].

Source code
A tutorial demonstrating quantum volume can be found at the QCMet software repository under:
circuit_execution_quality_metrics/quantum_volume.

Metric references
[109] A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, “Validating Quantum Computers
Using Randomized Model Circuits”, Phys. Rev. A 100, 032328 (2019).
[322] C. H. Baldwin, K. Mayer, N. C. Brown, C. Ryan-Anderson, and D. Hayes, “Re-Examining the Quantum Volume
Test: Ideal Distributions, Compiler Optimizations, Confidence Intervals, and Scalable Resource Estimations”,
Quantum 6, 707 (2022).

72
M4. Circuit execution quality metrics

[578] S. Aaronson and L. Chen, “Complexity-Theoretic Foundations of Quantum Supremacy Experiments”, Proc 32nd
Comput Complex. Conf (2016), 22:1.
[579] N. Moll et al., “Quantum Optimization Using Variational Algorithms on Near-Term Quantum Devices”, Quantum
Sci. Technol. 3, 030503 (2018).

M4.2. Mirrored circuits average polarization

The mirrored circuits polarization metric measures the capability of a quantum computer to run a specified
type of quantum circuits successfully [113]. Mirrored circuits are based on the application of a circuit C, and
then on the application of its inverse. Randomization layers are applied before, after and between C and its
inverse. The randomization layers map the initial random circuit C to a family of circuits, MC . This reduces
unwanted potential cancellation of systematic errors, which may occur when directly applying a circuit and
its inverse. In the noiseless case, each circuit in MC results in a deterministic output state, which is easy to
compute classically.
The success probability, psuccess , is the probability of measuring the correct output state for the full mirrored
circuit. In the case of a fully depolarized state, each possible outcome has the same probability, and so psuccess =
1/2w , where w is the number of qubits, also denoted as the width of the circuit. To avoid having a non-zero
psuccess for this case, the interpretation of circuit execution quality for a given value of the metric depends on
the number of qubits. The normalized difference between psuccess and this fully depolarized limit, denoted as
the polarization, J, is taken as the success metric. J is given by
1
psuccess − 2w
J= . (M4.2.1)
1 − 21w

A J = 0 typically indicates a fully depolarized state. Note however that a J = 0 can also be due to coherent
errors, and that J can also be negative, for example if the success probability is zero.
If the base circuit C can be generalized to different widths and numbers of layers, then this protocol can be
run following the volumetric benchmarking framework (section 6.2). Since the ideal output can be classically
computed also for large systems, the method is scalable.

Description

|0⟩

|0⟩

|0⟩

|0⟩ L0 C Q0 C̃ −1 L−1
0

.. .. .. ..
. . . .
|0⟩

|0⟩

Figure M4.2.1.: Circuit diagram showing the structure of a mirrored circuit. The first layer L0 is a w-fold tensor
product of randomly selected single qubit Clifford gates, where w is the number of qubits in the circuit. C is the base
circuit, Q0 is the w-fold tensor product of randomly selected Pauli gates. The C̃ −1 block is the quasi inverse of C, though
it is just as valid to be the inverse C −1 . The final block L−1
0 is the inverse of the first layer L0 .

A w-qubit circuit C is selected, which is split into m layers, such that C = Lm Lm−1 ...L2 L1 . C can be a
dedicated test circuit, or also a circuit of used within a specific application, such as a state preparation circuit
within a variational quantum eigensolver. The mirrored circuits protocol transforms C into a set of circuits MC
⊗w
as follows. Given an initial |0⟩ state and measurement in the computational basis, a mirrored circuit in MC ,
as depicted in Fig. M4.2.1, is written in matrix form as L−1 0 C̃
−1
QCL0 , where L0 is a random layer of single-
qubit Clifford gates, Q is a random Pauli string, such that Q ∈ (X, Y, Z, I)⊗w , and C̃ −1 is a quasi-inversion
circuit, C̃ = L̃−1 −1 −1 −1 −1
1 L̃2 ...L̃m−1 L̃m . Each quasi-inversion layer satisfies L̃i Li = Qi , where Qi is a random Pauli
layer. Adding this additional randomization can further reduce systematic error cancellations. The insertion of

73
M4. Circuit execution quality metrics

randomization layers can vary. For example, all the Qi can also be set to the identity, in which case C̃ is the
inverse of C. For each circuit C, a family of circuits MC is generated by selecting different Q0 and L0 . The
layer L0 is selected randomly from Cw 1 , the w-fold tensor product of all single qubit Clifford gates. The layer
Q0 is selected randomly from the w-fold tensor product of the Pauli gates. As part of the protocol, the detailed
distribution of randomization layers needs to be specified for reproducibility of the results.
The protocol is defined for Clifford circuits, as well as for circuits containing alternating layers of Clifford and
Rz (θ) operations [113]. The circuit unitary is L−1 0 C̃
−1
QCL0 = Q′ , where Q′ is another Pauli layer. The value
′
of Q can be easily computed classically. This is due to the fact that if C is a Clifford circuit, one can use the
property that any Clifford multiplied with a Pauli returns another Pauli, so that for an arbitrary Pauli layer,
Q(1) , one obtains L̃−1
i Q
(1)
Li = Q(2) , where Q(2) is another Pauli layer, and which can be efficiently computed.
In the more general case, where Rz (θ) gates are included in C, the construction is more involved, as outlined
in the supplementary material of Ref. [113]. The main reason that Q′ can still be efficiently calculated is that
commuting a Pauli gate through a Rz (θ) gate either does nothing, or just flips the sign of the angle. In the
⊗w
noiseless case, applying the circuit L−1 0 C̃
−1
QCL0 = Q′ on |0⟩ returns a single deterministic output state.
For example, for a single qubit the output state is |0⟩ when Q is I or Z, and |1⟩ when Q′ is X or Y .
′

For each circuit c ∈ MC the circuit is executed on hardware, and the polarization Jc is calculated using
Eq. M4.2.1. The average mirrored circuits polarization is given by
1 X
Jave = Jc . (M4.2.2)
|MC |
c∈MC

When executed as part of a volumetric benchmarking approach, Jave is determined for pairs of integers (w, dc ),
where dc is the full depth of the circuit, such that dc = 2m + 3. The 2m layers come from the base circuit and
its inverse, and the remaining 3 come from L0 , its inverse, and Q0 . Within a depolarising error model, where
the error is the same for all layers, the circuit polarization decays exponentially with depth [113]. Hence, the
volumetric benchmarking success criteria is typically set to the threshold Jave ≥ 1/e ≈ 0.37.

Measurement procedure
In what follows the procedure to measure the mirrored circuits average circuit polarization for a specific circuit
C is presented. This mirrored circuits protocol can be used as a volumetric benchmarking instance if C can be
set for specified w and m. The process then is as follows:
1. Select a base circuit C with width w and with number of layers m.
2. Choose the number of mirrored circuits, k, to generate MC such that |MC | = k. Each element in MC is
generated in the following way:
a) Select the L0 layer at random. This layer consists of a random single qubit Clifford gate on each
qubit.
b) Select the Pauli layer, Q0 , at random. Q0 consists of random single qubit Pauli gates on each qubit.
c) Select a number k of quasi-inverse circuits, as defined in the description.
3. All circuits in MC have the form MC = L−1 −1

0 C̃ Q0 CL0 . Then, for each circuit q ∈ MC :
a) compute the output string classically for the circuit q in the noiseless case.
⊗w
b) execute q on the quantum computer, with initial state |0⟩ .
c) Measure the outcome probabilities and compute J using Eq. M4.2.1.
4. Calculate the average polarization Jave , given in Eq. M4.2.2.

Assumptions and limitations

• The considered gates in the circuit need to be Clifford gates or Rz (θ) gates.

Source code
A tutorial for the mirrored circuits benchmarking is provided in the QCMet software repository under:
circuit_execution_quality_metrics/mirrored_circuits.

Metric references
[113] T. Proctor, K. Rudinger, K. Young, E. Nielsen, and R. Blume-Kohout, “Measuring the Capabilities of Quantum
Computers”, Nat. Phys. 18, 75 (2022).

74
M4. Circuit execution quality metrics

M4.3. Algorithmic qubits

The algorithmic qubits metric, denoted as #AQ, corresponds to the largest number of qubits that can run a
set of selected quantum algorithms successfully according to a specific success criterion [324]. It is built on the
volumetric benchmarking framework (section 6.2).

Description
The #AQ metric, which corresponds to an integer number of qubits, builds on the volumetric benchmarking
framework introduced in section 6.2, using a pre-defined set of circuits that are based on representative quantum
computing tasks [324]. These are six tasks selected from the the benchmarking study performed by the quantum
economic development consortium (QED-C) [118]. The single valued output #AQ corresponds to an integer
number.
For each circuit c within this set, the fidelity Fc of the measured output probability distribution Qoutput against
the ideal probability distribution obtained by a noiseless quantum computing emulator, Qideal , is computed.
Here the following relation is used for the fidelity of the output probabilities,
!2
X√
Fc (Qideal , Qoutput ) = px,output px,ideal , (M4.3.1)
x

where the sum goes over all possible measurement outcomes, x is a measurement output bitstring, and px,ideal
and px,output are the probabilities of obtaining the output bitstring x in the measurements for the ideal and
experimental circuits, respectively. Within the volumetric benchmarking framework the obtained fidelities for
all circuits in the set can be plotted as function of the number of qubits, Nq , and of the circuit depth, dc . For
the algorithmic qubits metric, the depth is defined as the total number of CNOT gates in the circuit [324].
Note that this definition of depth also includes CNOT gates that can, in principle, be executed in parallel at
the same time.
To compute the value of #AQ one defines the success criterion for a circuit c as Fc − ϵc > t, where t is a
constant representing the success threshold,p and is set to t = 1/e ≈ 0.368; ϵc is the standard error of the mean,
ns is the number of shots, given by ϵc = (Fc (1 − Fc ))/ns . Then the value of #AQ is defined as the width of
the largest box that can be drawn on the 2D fidelity plot with width Nq and depth Nq2 , and where all points
inside the box pass the success criterion.

Measurement procedure
In the measurement procedure presented here, the method and choice of circuits follows the definitions of #AQ
by IonQ [324] in its #AQ definition version 1.0. This in turn builds on the benchmarking study from the
quantum economic development consortium (QED-C) [118]. It specifies six tasks developed within the QED-C
benchmarking study to use for the calculation of #AQ. These tasks comprise the quantum Fourier transform,
as described in metric M5.4, the quantum phase estimation, the amplitude estimation, Monte Carlo sampling,
the variation quantum eigensolver, as described in metric M5.1, and Hamiltonian simulation.
Any circuit transpilation applied before running this set of circuits needs to comply with the following rules:
• The number of CNOT gates per circuit is calculated by transpiling the circuits to a gate set of {Rx , Ry ,
Rz , CNOT}, regardless of whether the hardware implements this set of native gates or not.
• Any optimization to the circuits are only allowed if the quantum computer executes the same unitary
operator as the submitted circuits.
• Potentially applied error mitigation techniques, such as randomized compiling, must be reported along
with the value for #AQ.
The detailed rules specified by IonQ are provided in the following folder forked from IonQ repository
circuit_execution_quality_metrics/algorithmic_qubits/code/_doc/AQ.md.
Then the measurement procedure is defined as:
1. All the circuits for the six tasks constitute the set of circuits C. For each circuit c ∈ C, determine the
fidelity Fc of the experimental output versus the ideal results obtained on a quantum computing emulator
for a noiseless quantum computer.
2. Make a 2D plot of all the obtained fidelities, with the axes being the width and depth of circuits.
3. On the plot, draw a box with width Nq , depth Nq2 , and where all fidelities inside the box pass the success
criterion defined in the previous section. Then #AQ is equal to the width of the largest such box.

75
M4. Circuit execution quality metrics

Assumptions and limitations

• The tasks and circuits included in the benchmarking set are chosen such that they span a range of potential
applications. Although the choice is well motivated, ultimately the specific list of tasks included in the
set is rather arbitrary, and can be amended in future.

• Since the #AQ metric requires running the selected applications on noiseless emulators of quantum com-
puters, it can only be evaluated for limited number of qubits[118].

Source Code
A tutorial for measuring #AQ is provided in the QCMet software repository under:
circuit_execution_quality_metrics/algorithmic_qubits.

Metric references
[118] T. Lubinski, S. Johri, P. Varosy, J. Coleman, L. Zhao, J. Necaise, C. H. Baldwin, K. Mayer, and T. Proctor,
“Application-Oriented Performance Benchmarks for Quantum Computing”, IEEE Trans. Quantum Eng. 4, 1
(2023).
[324] IonQ, “Algorithmic Qubits: A Better Single-Number Metric”, IonQ Website, [Accessed: 1-November-2023] (2022).

M4.4. Upper bound on the variation distance

The quantum accreditation protocol provides an upper bound on the variation distance between the probability
distribution of the experimental outputs of a noisy quantum circuit and its noiseless counterparts [116, 117].
It is therefore a measure for the correctness of the outputs of a given computation on the quantum computing
hardware.

Description
To establish the upper bound within an accreditation protocol (AP), one considers the probability distribution
of the experimental outputs of a noisy quantum circuit, {px,output }, and its noiseless counterpart, {px,ideal }.
Here x denotes the bitstrings that may be obtained as output. The quantum AP provides an upper bound,
denoted as b, on the variation distance (VD) [116, 117], given by
1X
VD := px,ideal − px,output , (M4.4.1)
2 x

so that
VD ≤ b. (M4.4.2)
The VD ∈ [0, 1], where the lower limit corresponds to a noiseless, ideal implementation of the quantum circuit.
In general, the VD is the measure of the correctness of a noisy, error-prone computation relative to the noiseless,
error-free ideal. However, measuring {px,output } is exponentially expensive due to requiring exponentially many
measurements, as is computing {px,ideal } classically.
The AP overcomes this challenge by estimating a bound b on VD in an efficient manner. The quantity b
is estimated experimentally with accuracy µ ∈ (0, 1) and confidence η ∈ (0, 1) chosen by the user. Obtaining
the upper bound b does not require exponentially expensive classical simulation of quantum circuits. This
is a significant advantage for larger systems when compared to typically considered volumetric benchmarking
metrics (Sec. 6.2), such as for example the quantum volume (metric M4.1) or algorithmic qubits (metric M4.3),
where such exponentially scaling classical computations are needed.
The AP ascertains the correctness of a noisy quantum circuit rather than the performance of individual
gates or gate sets. Thus, it includes all contributions to the overall noise obtained when the quantum circuit
is executed. The AP is motivated by ideas of quantum verification, whose origins lie in theoretical computer
science. Therein, the verifiability of quantum computations is related to the existence of an interactive proof
system [135]. A primary technique is that of blindness, whereby the computation to be verified called the target,
and whose outputs are hard to obtain classically, is replaced by others, called traps, and whose outputs are easy
to obtain classically. The traps have the same size and structure as the target, and the correctness of several
traps is used to bound the correctness of the target. In this metric the trap circuits are designed to be Clifford
circuits.
To verify the correctness of a computation, some assumption on the trustworthiness of components of the
computation must be made. Trust is thus a central notion in any verification, quantum or classical. In theoretical

76
M4. Circuit execution quality metrics

computer science studies, trust assumptions are made on the preparation or measurement of single qubits [135].
Formally, the AP can be thought of as shifting the assumption to trusting single-qubit gates, in the sense it is
assumed that the error in single qubit gates does not depend strongly on the specific rotation angle applied to
the qubit by the gate. This is motivated by the empirical observation that single-qubit gates are the components
in quantum hardware least affected by noise when compared to multi-qubit gates.
The AP [116, 117] uses trap circuits to obtain the bound b. The AP takes as input a target circuit, and two
numbers, µ and η ∈ (0, 1), which quantify the desired accuracy and confidence on the final bound, respectively.
The target circuit must:
⊗Nq
1. take as input Nq qubits in the state |0⟩ ,

2. contain 2m cycles alternating between a cycle of one-qubit gates and a cycle of two-qubit gates, which
can be efficiently chosen to be CZ or another two-qubit Clifford gate (see Fig. M4.4.1a),
3. end with measurements in the Pauli-Z basis (see Fig. M4.4.1a).
The trap-based AP requires executing v trap circuits sequentially, where v = ⌈2 ln(2/(1 − η))/µ2 ⌉, and ⌈·⌉ is
the ceiling function. Given a target circuit, the trap circuits are obtained as in Fig. M4.4.1b.

|0⟩0 U0,1 U0,2 U0,m−1 U0,m Z

|0⟩1 U1,1 U1,2 U1,m−1 U1,m Z

|0⟩2 U2,1 U2,2 U2,m−1 U2,m Z

|0⟩3 U3,1 U3,2 U3,m−1 U3,m Z

|0⟩4 U4,1 U4,2 U4,m−1 U4,m Z

|0⟩5 U5,1 U5,2 U5,m−1 U5,m Z

(a)

|0⟩0 Ht S S† H H H Ht Z

|0⟩1 Ht S S† S S S† Ht Z

|0⟩2 Ht H H S S S† Ht Z

|0⟩3 Ht S S† S H H Ht Z

|0⟩4 Ht S S† S H H Ht Z

|0⟩5 Ht H H H S S† Ht Z

(b)

Figure M4.4.1.: (a) Example of a target circuit used in the quantum accreditation protocol. The target circuit must
be compiled into m cycles of one-qubit gates Ui,j (on qubit i and cycle j). Each cycle of Ui,j , apart from the last
cycle, is followed by a cycle of controlled-Z (CZ ) gates, denoted as vertically connected dots. This gives a circuit depth
dc = 2m − 1. The qubits are initialized in the state |0⟩ and measurements are in the computational basis. Subfigure (b)
shows an example of a trap circuit for the target circuit in (a). The trap circuit is obtained by replacing the one-qubit
gates in the target circuit with one-qubit Clifford gates, such as the H gate and the S gate (see Sec. 6.1). Neighboring
cycles of one-qubit gates can be recompiled into a single cycle. Thus, the trap circuit has the same circuit depth as the
target.

Measurement procedure
1. Select a target circuit following the restrictions outlined in the previous section. Select two numbers, µ
and η ∈ (0, 1), which quantify the desired accuracy and confidence on the final bound.
2. Generate v trap circuits, where v = ⌈2ln(2/(1 − η))/µ2 ⌉, and ⌈·⌉ is the ceiling function. Each trap circuit
is obtained by replacing the single qubit gates in the target circuit with random one-qubit Clifford gates
as follows (see Fig. M4.4.1b):

77
M5. Well-studied tasks execution quality metrics

a) For all j ∈ {1, . . . , m − 1} and for all i ∈ {0, . . . , n − 1}:

• If the j-th cycle of CZ (controlled-Z) gates connects qubit i to qubit i′ , randomly replace Ui,j
with an S (phase) gate, and Ui′ ,j with an H (Hadamard) gate, or Ui,j with H and Ui′ ,j with
S. For definitions of the Hadamard and phase gates, see section 6.1. After the j-th cycle of CZ
gates, insert a S † gate if S has replaced Ui,j , or insert another H if H has replaced Ui,j . This is
referred as undoing the H and S gates.
• If the j-th cycle of CZ gates does not connect qubit i to any other qubit, randomly replace Ui,j
with H or S. Undo this gate after the cycle of CZ gates.
b) Initialize a random bit t ∈ {0, 1}. If t = 0, do nothing. If t = 1, append a Hadamard gate on every
qubit at the beginning and at the end of the circuit.
3. Append a cycle of random Pauli gates after every cycle of one-qubit gates, followed by a second cycle of
Pauli gates before the next cycle of one-qubit gates that undoes the first.
4. Recompile neighbouring cycles of one-qubit gates into a single cycle.
5. Implement all the circuits on hardware, and count the number Ninc of traps that do not return the output
bitstring (0, 0, · · · , 0).
6. Set b = 2Ninc /v.

Assumptions and limitations

1. The trap-based AP requires that all two-qubit gates in the target circuit are Clifford gates.
2. The noise in state preparation, measurements and cycles can be modelled by completely-positive, trace-
preserving (CPTP) maps acting on all the qubits (see section 6.1).
3. An implementation of a cycle of gates G on a state ρ at circuit depth dc returns ΦG,j G(ρ), where ΦG,j is
a CPTP map that potentially acts on the whole system and depends on both G and the depth dc . This
is a Markovian noise model that encompasses a broad class of noise processes afflicting current platforms,
such as gate-dependent noise and crosstalk between neighbouring qubits.
4. Cycles of one-qubit gates suffer gate-independent noise, i.e. ΦG1q ,j = Φj for all the cycles of one-qubit
gates G1q . However, the AP is robust to noise that depends weakly on the cycles of one-qubit gates [117,
Appendix 2].
An alternate trap-based [116] AP can detect even more complex noise, such as temporally correlated qubit-
environment couplings, albeit at the cost of looser bounds on the VD [117, Appendix 3].

Source code
A tutorial for calculating the upper bound on the VD using the AP protocol is provided in the QCMet software
repository under:
circuit_execution_quality_metrics/upper_bound_on_the_variation_distance.

Metric references
[116] S. Ferracin, T. Kapourniotis, and A. Datta, “Accrediting Outputs of Noisy Intermediate-Scale Quantum Com-
puting Devices”, New J. Phys. 21, 113038 (2019).
[117] S. Ferracin, S. T. Merkel, D. McKay, and A. Datta, “Experimental Accreditation of Outputs of Noisy Quantum
Computers”, Phys. Rev. A 104, 042603 (2021).
[135] A. Gheorghiu, T. Kapourniotis, and E. Kashefi, “Verification of Quantum Computation: An Overview of Existing
Approaches”, Theory Comput. Syst. 63, 715 (2019).

M5.Well-studied tasks execution quality metrics

M5.1. Variational quantum eigensolver (VQE) metric

This metric evaluates how well a quantum computer can execute the variational quantum eigensolver (VQE)
algorithm [330], which is a type of variational quantum algorithm (VQA) [580]. VQAs use parameterized

78
M5. Well-studied tasks execution quality metrics

quantum circuits to prepare wave functions on the quantum computer, and use a classical optimizer to find
the state that minimizes a cost function. The parameters of the quantum circuit typically correspond to
the parameters of single-qubit and two-qubit gates, and the cost function can be the energy in materials or
quantum chemistry simulations. VQAs have some level of resilience to noise [581], and are hence often used
for demonstrations on noisy intermediate scale quantum (NISQ) hardware [330]. The optimization of circuit
parameters can become difficult as system sizes increase, where the effect of noise also becomes significant [330].
The VQE is a widely used VQA, which has been applied extensively to find the ground state of electronic
structure problems. VQE has two components: the first component consists of the use of a quantum computer
to obtain the energy expectation value of a given trial wave function for a specified Hamiltonian, while the
second component is an optimization loop on a classical computer, where the parameters of the quantum circuit
are adjusted to minimize this energy.

Description
For an extensive review of VQE see Ref. [330]. In this section, in order to distinguish between operators
and scalar values, hats on symbols are used to denote operators. For a quantum system of interest with a
Hamiltonian, Ĥ, a common task is to find a wave function that minimizes the energy of the quantum system.
The VQE solves this task with a parameterized quantum circuit that produces a trial wave function, |Ψ(θ)⟩, on
the quantum computer, where θ is a vector of parameters. By minimizing the expectation value of Ĥ, given by

E = ⟨Ψ(θ)|Ĥ|Ψ(θ)⟩ , (M5.1.1)

with respect to the parameters θ, the energy is minimized. Thus, the two components of VQE include the
method to calculate E, and the method for finding the optimal values for θ, such that the optimized |Ψ(θ)⟩
minimizes E.
To calculate E with a quantum computer,P Ĥ needs to be rewritten as a sum of Pauli strings that can be
measured on quantum computers, Ĥ = i wi P̂i , where the range of terms in the sum over i depends on the
Hamiltonian, wi are weights and P̂i are Pauli strings. Calculating E for a given trial wave function can then
be achieved by measuring the expectation value for each of the Pauli strings separately, and then summing the
results weighted by wi . The number of shots, with which the Pauli strings are measured, directly impacts the
accuracy of the obtained E.
The structure of the parameterized quantum circuit is denoted as the ansatz. The circuit ansatz determines
how the trial wave function |Ψ(θ)⟩ is constructed on a quantum computer. When choosing an ansatz, key aspects
to consider are the expressibility, trainability and scalability. The expressibility indicates whether an ansatz
can generate a large or only a restricted class of wave functions in the Hilbert space. Trainability indicates how
easy it is to optimize the parameters θ with classical optimization techniques. As the expressibility increases,
it may generally be more difficult to train the ansatz, hence a trade-off between expressibility and trainability
often needs to be made. Scalability indicates how the depth of the circuit, as well as trainability, changes for
an increasing size of the system, typically related to the number of qubits. If an ansatz performs well for small
systems, but has poor scalability, it may not be practically useful for large systems.
Finding the optimal θ is achieved using a classical optimizer that iteratively updates θ based on the value
of E until it converges. The final solution to the wave-function-finding problem is then given by |Ψ(θoptimal )⟩,
produced by the parameterized circuit with the optimal parameters. The choice of optimizer is important,
as it strongly influences the speed of convergence and whether the final state is optimal. Both these aspects
are relevant metrics for the performance of a VQE run. However, the aim of the metric in this section is to
benchmark the quality of the quantum processor rather than that of the classical optimizer. The quality of
the quantum processor determines how accurately the energy in Eq. M5.1.1 is computed for a given trial wave
function. Therefore, the output of a quantum hardware metric based on the VQE well-studied task requires
the evaluation of the fidelity of the computed energy for one or more wave functions and Hamiltonians, without
optimizing the parameters. The chosen Hamiltonians needs to be representative of the classes of Hamiltonians
appearing in quantum chemistry and materials science problems, and the chosen fixed wave functions for a
given Hamiltonian need to be representative for the types of wave functions appearing during the optimization
loop.

Measurement procedure
VQE performance depends on the choices of the different components outlined in the previous metric [330].
Here a measurement procedure to evaluate the accuracy of the computed energy for a representative system is
presented.
The target system chosen in this metric is the Hubbard model [582]. Specifically, the 1D Fermi-Hubbard
model with Ns sites is used and is the same as described in detail in metric M5.3. Given 2Ns distinct fermionic

79
M5. Well-studied tasks execution quality metrics

sites, where there are Ns physical sites each with 2 spin degrees of freedom, the Hamiltonian is given by
s −1
NX X   Ns
ĉ†iσ ĉi+1σ + ĉ†i+1σ ĉiσ + UH
X
Ĥ = −γ n̂i↑ n̂i↓ , (M5.1.2)
i=1 σ∈{↑,↓} i=1

where ĉ†iσ (ĉiσ ) is the creation (annihilation) operator for a Fermion on site i with spin σ, n̂iσ is the corresponding
number operator, γ is the hopping integral, and UH is the on-site electron-electron interaction strength.
This Hamiltonian maps to Pauli operators using the Jordan-Wigner transformation [583]. In order to map
the system onto qubits, it is first necessary to specify the ordering of the spin sites. This methodology uses the
following ordering of site and spin indices into an up-and-down chain

{1 ↑, 2 ↑, · · · , Ns ↑, 1 ↓, 2 ↓, · · · , Ns ↓} → {1, 2, · · · , Ns , Ns + 1, Ns + 2, · · · , 2Ns }. (M5.1.3)

With this ordering and the use of the Jordan-Wigner mapping, the Hamiltonian maps onto a 2Ns qubit Hamil-
tonian that can be expressed as
2Ns −1 Ns 2N
γ X x x y y
UH X z z UH Xs z Ns UH
Ĥq = − σ̂i σ̂i+1 + σ̂i σ̂i+1 + σ̂ σ̂ − σ̂ + , (M5.1.4)
2 i=1 4 i=1 i i+Ns 4 i=1 i 4
i̸=Ns

where σ̂iα (for α = x, y, z) is the Pauli-α operator acting on qubit i. Here, the ratio of UH /γ is what determines
the regime of the system, such as whether a metallic or insulating behaviour is described. As a result, γ is set
to 1 so the ratio is determined solely by the value of UH .
Next, the choice of ansatz for the VQE is taken to be the Hamiltonian variational ansatz (HVA) [584, 585].
The ansatz consists of parameterized gates which form a layer of circuit, and the circuit layer can be repeated
m times with each repetition having different parameters. Increasing the number of layers m of the ansatz
improves the expressibility of the quantum circuit [586]. In order to apply the HVA, an initial state, |Ψ0 ⟩, needs
to be prepared. The state |Ψ0 ⟩ is chosen such that it is the ground state of the non-interacting problem where
UH = 0, which can be computed efficiently classically [587].
The following measurement procedure largely follows Ref. [586]. However, since this is a metric for a quantum
computer, the classical optimization part of the VQE has been removed. The measurement procedure then is
as follows:
1. Select the number of sites Ns , and choose the onsite energy UH for the Fermi-Hubbard model. Then
convert the Hamiltonian into a weighted sum of Pauli strings using the Jordan-Wigner transformation
following Eq. M5.1.4. The number of qubits Nq = 2Ns .
2. Classically compute |Ψ0 ⟩, the ground state when UH = 0 for the Hamiltonian.

3. Select the number of layers of the Hamiltonian variational ansatz m to apply , and choose the number of
trials, ntrials , for which to calculate the metric.
4. Then, for each trial i ∈ {1, 2, · · · , ntrials }:
a) Prepare the qubits in the initial state |Ψ0 ⟩.
b) Randomly select the angles, θ, for the parameterized gates in all of the m layers of the HVA, and
then apply the m layers of the HVA to the qubits with the selected angles.
c) Compute the numerically exact energy for the wave function given by this ansatz using a classical
(i)
computer, denoted as Eexact .
d) Compute the energy on the quantum computer being tested. For each Pauli string from step 1,
measure the qubits with a chosen number of shots, then compute the energy using the weighted
sum, denoted as E (i) . The number of shots should be large enough so that the uncertainty in E (i) is
(i)
significantly smaller than its difference to Eexact .
5. Calculate Ediff , the average of the difference between the quantum computer estimated energy and exact
energy per-site for all of the trials
nX
trials
1 (i)
Ediff = |E (i) − Eexact |. (M5.1.5)
ntrials Ns i=1

This difference is the metric for the quality of the quantum circuit execution on the quantum computer
within the VQE algorithm. We note that as alternative or additional component of the metric one may also

80
M5. Well-studied tasks execution quality metrics

compute the fidelity between the wave functions obtained on the quantum computer and the numerically
exact classical result, which is the metric used for the Fermi-Hubbard model simulation in metric M5.3.
However, since the quality of the VQE optimization loop depends mainly on the accuracy of the energy,
we use the energy difference as representative metric rather than the fidelity.

Assumptions and limitations

• The VQE performance depends significantly on the type of Hamiltonian, on the choice of ansatz and on
the number of qubits. Hence, evaluating the VQE well-studied task for a specific system and size may
not be representative of the general hardware performance. For example, the obtained performance can
change significantly for increasing number of qubits [330].
• In this metric only the quality of the energy obtained for a trial wave function is considered. The choice
of classical optimizer also affects the overall performance and cost of the full VQE algorithm [330].

Source code
A tutorial for measuring the ability of the quantum computer to execute VQE circuits is provided in the QCMet
software repository under:
well_studied_task_execution_quality_metrics/vqe, with associated source code.

Metric references
[330] J. Tilly et al., “The Variational Quantum Eigensolver: A review of methods and best practices”, Phys. Rep. 986,
1 (2022).
[580] K. Bharti et al., “Noisy intermediate-scale quantum algorithms”, Rev. Mod. Phys. 94, 015004 (2022).
[581] E. Fontana, N. Fitzpatrick, D. M. Ramo, R. Duncan, and I. Rungger, “Evaluating the noise resilience of variational
quantum algorithms”, Phys. Rev. A 104, 022403 (2021).
[582] J. Hubbard and B. H. Flowers, “Electron correlations in narrow energy bands”, Proc R Soc Lond. A 276, 238
(1963).
[583] M. A. Nielsen, “The Fermionic canonical commutation relations and the Jordan-Wigner transform”, Tech Rep
Sch. Phys. Sci. Univ. Qld. 59, 75 (2005).
[584] D. Wecker, M. B. Hastings, and M. Troyer, “Progress towards practical quantum variational algorithms”, Phys.
Rev. A 92, 042303 (2015).
[585] J.-M. Reiner, F. Wilhelm-Mauch, G. Schön, and M. Marthaler, “Finding the ground state of the Hubbard model
by variational methods on a quantum computer with gate errors”, Quantum Sci. Technol. 4, 035005 (2019).
[586] B. Anselme Martin, P. Simon, and M. J. Rančić, “Simulating strongly interacting Hubbard chains with the
variational Hamiltonian ansatz on a quantum computer”, Phys. Rev. Res. 4, 023190 (2022).
[587] Z. Jiang, K. J. Sung, K. Kechedzhi, V. N. Smelyanskiy, and S. Boixo, “Quantum Algorithms to Simulate Many-
Body Physics of Correlated Fermions”, Phys. Rev. Appl. 9, 044036 (2018).

M5.2. Quantum approximate optimization algorithm (QAOA) metric

This metric evaluates how well a quantum computer can execute the quantum approximate optimization algo-
rithm (QAOA) by using it to solve the so called MaxCut optimization problem. MaxCut requires one to assign
the vertices in a given graph into two sets, such that the number of edges between the two sets is maximized.
It is a well defined fundamental problem in combinatorial optimization [588]. QAOA is a type of variational
quantum algorithm (VQA) for solving optimization tasks by re-expressing the problem as a Hamiltonian and
then finding its ground state, which corresponds to the solution of the problem [345]. Note that QAOA can
also be applied to other optimization problems [345, 348].
Solving MaxCut for random graphs is NP-hard for classical computers [588]. There are classical approximation
methods that can find solutions that are close to the optimal solution up to an approximation ratio [589].
MaxCut can be re-expressed as an Ising Hamiltonian [580], so that it can be solved on a quantum computer
using QAOA. One particular implementation of a QAOA based metric to solve the MaxCut problem is the so
called Q-Score [130]. It specifies the specific parameters to be used within the QAOA, and is the one that is
described in the measurement procedure section of this metric [130].

81
M5. Well-studied tasks execution quality metrics

Description
Given a graph G(VG , EG ), where VG is the set of vertices and EG is the set of edges with each edge connecting
a pair of vertices, the MaxCut problem is to partition the vertices into two sets, A and B, such that the number
of edges between A and B is maximized [345].
This problem can be solved on quantum computers using QAOA, outlined in what follows [348, 580]. Given a
graph G(VG , EG ), define the Hamiltonians H0 = − 1≤i≤n Xi and HG = (i,j)∈EG Zi ⊗ Zj − |E2G | , where n =
P P

|VG | is the size of VG , and X, Z are Pauli operators. Then implement the following unitary as a parameterized
circuit on n qubits: Y
U (γ, β) = e−iβq H0 /2 e−iγq HG /2 , (M5.2.1)
1≤q≤l

where l is a parameter determining the number of circuit layers, also known as the depth parameter, q is
the index of the layer, and γ and β are two vectors of parameters with size Q. In each layer, for each edge
(i, j) ∈ EG , the eγZi ⊗Zj terms are decomposed to two CNOT gates on qubits (i, j) and a single Rz rotation on
qubit j. Then the term involving the H0 is implemented with Rx gates on each qubit.
The cost function is defined as L(γ, β) = − ⟨0| U † (γ, β)HG U (γ, β) |0⟩. Using a classical optimizer and the
cost computed above, QAOA seeks to find the parameters γ, β that minimize the cost L(γ, β). For current
NISQ devices, the depth parameter l is restricted to be 1 or 2, since for higher depths the noise induced errors
become too large.
After the parameters are optimized, the solution to MaxCut is determined by the bitstring that has the highest
probability from measuring the circuit. For each digit in the bitstring, if the digit is 0, then the corresponding
vertex in the graph is assigned to the set A, otherwise if the digit is 1, then it is assigned to B.

Measurement procedure
The measurement procedure presented here for benchmarking QAOA on a quantum computer follows from
Ref. [130], where a number of fixed choices for the used parameters are made. For example, this includes the
choice of the number of shots, and of the threshold above which a computation is considered successful. The
metric is named Q-Score, defined as the largest size of graph for which the hardware can solve the MaxCut
problem using QAOA sufficiently accurately on average. The methodology then is as follows:
1. Select the size of the problem, Nq .
2. Generate a random graph G(VG , EG ) with Nq vertices in the following way. For each possible edge between
the vertices, randomly decide whether it should be included in EG with a probability of 1/2.

3. Select Nq qubits from the hardware that perform best and that have good connectivity, and initialize
⊗N
them into |0⟩ q .
4. Select a depth l, usually 1 or 2 for current NISQ devices. Randomly initialize the parameters γ and β,
and apply the parameterized circuit U (γ, β) for l layers of QAOA, defined in the previous section.

5. Using a fixed number of shots, set to 2048, compute the cost L(γ, β).
6. Using the COBYLA optimizer [590], find the values (γ, β) that minimize the cost.
7. Repeat steps 2-6 for 100 different graphs. The average cost of all graphs defines the value L(Nq ), the
L(Nq )−Nq2 /8
average cost that the device can achieve. Then compute the ratio β(Nq ) := 3/2 , where λ is taken
λNq
to be 0.178. This number is obtained from the costs achieved by a classical exact solver [130], so that
optimal solutions give β(Nq ) = 1.
8. Repeat steps 1-7 for different Nq , and find the largest Nq that the ratio β(n) exceeds the value β ∗ = 0.2.
The value of the largest Nq is the Q-score of the device.

Assumptions and limitations

• A number of choices are made within Q-Score, which, while justified, are somewhat arbitrary. This
includes choice of the depth parameter l, the choice of optimizer (COBYLA), the choice of class of graphs
(random graphs with edge probability 1/2), the number of different graphs for each Nq (100 graphs), the
value of the ratio that is considered success (β ∗ = 0.2).

82
M5. Well-studied tasks execution quality metrics

• Because of those choices, it is not straight forward to generalize this score for different types of hardware. In
Ref. [373], a generalization that applies to quantum annealers is introduced. It is conceivable that a digital
quantum device could perform better for the same task using a different algorithm, or supplemented with
quantum error mitigation techniques, hence a comparison with quantum annealing must be performed
with care.

• For a small number of vertices corresponding to a small number of qubits, the Nq2 /8 term in the calculation
of β(Nq ) no longer corresponds to the score that would be achieved with a random solution. For example,
the problem is ill-defined when there is no edge between any of the vertices in the graph, hence no cuts
can be made and the MaxCut problem is not valid. In these cases, the random solution cannot achieve the
expected average score of Nq2 /8. This problem is particularly significant for a small number of vertices,
where there is a greater chance that a random graph contains no edges. Therefore the Q-Score test is
more likely to fail in these cases. This problem is most apparent for Nq = 2, where it is impossible to pass
the Q-Score test because the pass criteria Nq2 /8 = 0.5 is too high. In Ref. [130] the authors evaluate the
results from a problem instance with Nq ≥ 5.

Source Code
A tutorial to evaluate the Q-Score metric is provided in: the QCMet software repository under:
well_studied_task_execution_quality_metrics/qscore.

Metric references
[130] S. Martiel, T. Ayral, and C. Allouche, “Benchmarking Quantum Coprocessors in an Application-Centric, Hardware-
Agnostic, and Scalable Way”, IEEE Trans. Quantum Eng. 2, 1 (2021).
[345] E. Farhi, J. Goldstone, and S. Gutmann, “A quantum approximate optimization algorithm”, arXiv:1411.4028
(2014).
[348] L. Zhou, S.-T. Wang, S. Choi, H. Pichler, and M. D. Lukin, “Quantum Approximate Optimization Algorithm:
Performance, Mechanism, and Implementation on Near-Term Devices”, Phys. Rev. X 10, 021067 (2020).
[373] W. van der Schoot, D. Leermakers, R. Wezeman, N. Neumann, and F. Phillipson, “Evaluating the Q-score of
Quantum Annealers”, 2022 IEEE Int. Conf. Quantum Softw. QSW, 9 (2022).
[580] K. Bharti et al., “Noisy intermediate-scale quantum algorithms”, Rev. Mod. Phys. 94, 015004 (2022).
[588] J. Håstad, “Some optimal inapproximability results”, J. ACM 48, 798 (2001).
[589] K. J. Ahn and S. Guha, “Graph sparsification in the semi-streaming model”, Int Colloq Autom. Lang Program
(2009), 328.
[590] M. J. D. Powell, “A Direct Search Optimization Method That Models the Objective and Constraint Functions by
Linear Interpolation”, Advances in Optimization and Numerical Analysis, edited by S. Gomez and J.-P. Hennart
(Springer Netherlands, Dordrecht, 1994), 51.

M5.3. Fermi-Hubbard model simulation (FHMS) metric

This metric evaluates how well a quantum computer can perform Fermi-Hubbard model simulations (FHMS).
The Fermi-Hubbard model is a prototype many-body Hamiltonian for materials science simulations [591–598].
Depending on the quantities of interest and the method used, the cost of the classical computation scales
exponentially with the number of orbitals in the Hamiltonian, while that for quantum computation can scale
polynomially. It is thus a potential testbed for demonstrating quantum advantage. A number of quantum
methods have been proposed, such as those based on time-evolution of the wavefunction, which requires long
quantum circuits and hence long coherence times, or VQAs, as demonstrated on existing hardware [599–601].
This metric discusses benchmarking methods of the former.

Description
In this section, in order to distinguish between operators and scalar values, hats on symbols are used to denote
operators. The 1D Fermi-Hubbard model Hamiltonian is given by
s −1
NX X   Ns
ĉ†iσ ĉi+1σ ĉ†i+1σ ĉiσ
X
Ĥ = −γ + + UH n̂i↑ n̂i↓ , (M5.3.1)
i=1 σ∈{↑,↓} i=1

83
M5. Well-studied tasks execution quality metrics

where ĉ†iσ (ĉiσ ) is the creation (annihilation) operator for a Fermion in site i with spin σ, n̂iσ is the correspond-
ing number operator, γ is the hopping integral, and UH is the on-site electron-electron interaction strength.
Typically γ is taken to be positive, whereas UH can be positive (repulsive interactions) or negative (attractive
interactions).
In order to obtain a quantum circuit representation of the evolution operator for the Fermi-Hubbard model,
it is first necessary to map the fermionic system onto a system of qubits. For Ns physical sites each with 2 spin
sites, there are a total of 2Ns distinct fermionic sites. Using a Jordan-Wigner mapping, this can be mapped
onto a system of 2Ns qubits. In order to do so it is first necessary to specify the ordering of the spin sites to
qubit index, for example

{1 ↑, 1 ↓, 2 ↓, 2 ↑, . . . , Ns ↑, Ns ↓} → {1, 2, 3, 4, . . . , 2Ns − 1, 2Ns }.

With this ordering and the use of the Jordan-Wigner mapping, the fermionic Hamiltonian Ĥ maps onto a
qubit Hamiltonian of the form
s −1
NX
γ x z z x y z z y

Ĥq = − σ̂2i−1 σ̂2i σ̂2i+1 σ̂2i+2 + σ̂2i−1 σ̂2i σ̂2i+1 σ̂2i+2
2 i=1
i mod 2=1
s −1
NX Ns 2N
γ x x y y
UH X z z UH Xs z Ns UH
− σ̂2i−2 σ̂2i−1 + σ̂2i−2 σ̂2i−1 + σ̂2i−1 σ̂2i − σ̂ + , (M5.3.2)
2 i=1
4 i=1 4 i=1 i 4
i mod 2=0

where σ̂iα (for α = x, y, z) is the Pauli-α operator acting on qubit i. Here, the hopping terms are split into
two sets of terms. The first corresponds to the case where there are two hopping sites with opposite spins
in between (odd i). For example, hopping from 1 ↑ to 2 ↑ has 1 ↓ and 2 ↓ in between, hence σ̂ z terms
arise from the Jordan-Wigner mapping. The second corresponds to the case where the two hopping sites
occur on neighbouring qubits (even i). Note that different choices of the fermion ordering will give rise to
different forms of the spin Hamiltonian. The constant energy shift in the last term simply corresponds to an
arbitrary phase rotation and so can be ignored. The dynamics associated with this model is generated by the
h t
iNs
propagator Û = e−iĤt = e−iĤ Ns , working in units where ℏ = 1. The resulting Trotterized short time
propagator Ûδt can be implemented with quantum circuits. An example for 3 physical sites corresponding to
θ
  θ
6 qubits is shown
 in Fig. M5.3.1,
 where Rxx+yy (θ) = exp −i 4 (σ̂ x ⊗ σ̂ x + σ̂ y ⊗ σ̂ y ) , R zz = exp −i 2 σ̂z ⊗ σ̂ z
1 0 0 0
0 0 1 0 
and Fswap =  0 1 0 0 . These gates are decomposed into Ry , Rz and CX gates for implementation on


0 0 0 −1
quantum hardware.

Rz ( −U2H δt )
Rzz ( UH2 δt ) Fswap Fswap
Rz ( −U2H δt )
Rxx+yy (−2γδt) Rxx+yy (−2γδt)
Rz ( −U2H δt )
Rzz ( UH2 δt ) Fswap Fswap
Rz ( −U2H δt )
Rxx+yy (−2γδt) Rxx+yy (−2γδt)
Rz ( −U2H δt )
Rzz ( UH2 δt ) Fswap Fswap
Rz ( −U2H δt )

Figure M5.3.1.: Example 6 qubit quantum circuit diagram for each Trotter step for 1D Fermi-Hubbard model time-
evolution simulation with Ns = 3 physical sites. The gates used in this circuit are described in detail in the description
of this section.

Details of the derivation from the qubit Hamiltonian to the circuit implementation can be found in the source
code provided, with the link in the section below. This circuit is repeated for each Trotter step. The quantum
state at the end of the circuit is then the state of the time-evolved quantum system of interest.

84
M5. Well-studied tasks execution quality metrics

Measurement procedure
1. Specify the parameters for the 1D Fermi-Hubbard model Trotterized time-evolution simulation, including
the number of physical sites Ns , the hopping integral γ, the onsite energy UH , the evolution time for each
Trotter step δt, the number of Trotter steps and the initial configuration of the sites, where it is chosen
which qubits should be initialized into the |1⟩ state.
2. Construct the quantum circuit for time evolution according to the methods described above. Fig. M5.3.1
shows an example circuit for one Trotter step for 3 physical sites corresponding to 6 qubits. For each
Trotter step, repeatedly apply the circuit.
3. Run the circuit on hardware and measure all qubits with a certain number of shots to obtain a probability
distribution of output bitstrings, denoted as Qoutput . The number of shots should be chosen depending
on the level of desired benchmarking precision.
4. Obtain an ideal probability distribution, Qideal . For small systems this can be done in a few different
ways, including noiseless quantum circuit emulation or brute-force matrix exponential calculations.

5. Calculate the normalized fidelity between Qideal and the Qoutput in the following way. Define the fidelity
between two probability distributions Q1 and Q2 over the same set of random events:
!2
X√
Fc (Q1 , Q2 ) = px,1 px,2 , (M5.3.3)
x

where x is a random event, and px,i is the probability of x happening in the distribution Qi . For the
output of quantum computers, the random events x are the bitstrings. The normalized fidelity between
Qideal and Qoutput is then given by [118]:
 
Fc (Qideal , Qoutput ) − Fc (Qideal , Quni )
Fnorm (Qideal , Qoutput ) = max ,0 , (M5.3.4)
1 − Fc (Qideal , Quni )

where Quni is the uniform distribution.

An example of the results one might obtain is shown in Fig. M5.3.2.

Figure M5.3.2.: Results of emulator runs with the methodology outlined in this section to obtain the Fermi-Hubbard
model simulation metric, where the noise model used is specified in Sec. 6.1. The bars show the probability px of
measuring the output bitstrings. The black outlined bar shows the exact probability distribution, and the gray solid bar
shows the results obtained from the emulator with the specified noise model. The normalized fidelity between the two
distributions is calculated using Eq. M5.3.4 and found to be Fnorm (Qideal , Qemulator ) = 0.572.

85
M5. Well-studied tasks execution quality metrics

Assumptions and limitations

• This way of constructing the quantum circuits assumes that the quantum device only supports nearest-
neighbour operations on qubits. If the hardware provides better connectivity between qubits, one may
adjust how the circuit is constructed to tailor for the specific hardware.

• For large systems it can be very difficult or impossible to obtain an ideal probability distribution to
benchmark against. Alternative methods of benchmarking the performance of the circuit may be used,
such as quantum accreditation, which is introduced in metric M4.4.

Source Code
A tutorial to run the 1D Fermi-Hubbard model Trotterized time-evolution simulation and calculate the fidelity
is provided in: the QCMet software repository under:
well_studied_task_execution_quality_metrics/hubbard_model_simulation.

Metric references
[118] T. Lubinski, S. Johri, P. Varosy, J. Coleman, L. Zhao, J. Necaise, C. H. Baldwin, K. Mayer, and T. Proctor,
“Application-Oriented Performance Benchmarks for Quantum Computing”, IEEE Trans. Quantum Eng. 4, 1
(2023).
[591] J. Hubbard, “Electron correlations in narrow energy bands”, Proc. R. Soc. Lond. A 276, 238 (1963).
[592] M. Qin, T. Schäfer, S. Andergassen, P. Corboz, and E. Gull, “The hubbard model: a computational perspective”,
Annu. Rev. Condens. Matter Phys. 13, 275 (2022).
[593] A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, “Dynamical mean-field theory of strongly correlated
fermion systems and the limit of infinite dimensions”, Rev. Mod. Phys. 68, 13 (1996).
[594] G. Kotliar, S. Y. Savrasov, K. Haule, V. S. Oudovenko, O. Parcollet, and C. A. Marianetti, “Electronic structure
calculations with dynamical mean-field theory”, Rev. Mod. Phys. 78, 865 (2006).
[595] A. Paul and T. Birol, “Applications of dft+ dmft in materials science”, Annu. Rev. Mater. Res. 49, 31 (2019).
[596] F. Jamet, A. Agarwal, C. Lupo, D. E. Browne, C. Weber, and I. Rungger, “Krylov variational quantum algorithm
for first principles materials simulations”, arXiv:2105.13298 (2021).
[597] F. Jamet, A. Agarwal, and I. Rungger, “Quantum subspace expansion algorithm for green’s functions”, arXiv:2205.00094
(2022).
[598] F. Jamet, C. Lenihan, L. P. Lindoy, A. Agarwal, E. Fontana, B. A. Martin, and I. Rungger, “Anderson impurity
solver integrating tensor network methods with quantum computing”, arXiv:2304.06587 (2023).
[599] A. Montanaro and S. Stanisic, “Compressed variational quantum eigensolver for the Fermi-Hubbard model”,
arXiv:2006.01179 (2020).
[600] S. Stanisic, J. L. Bosse, F. M. Gambetta, R. A. Santos, W. Mruczkiewicz, T. E. O’Brien, E. Ostby, and A.
Montanaro, “Observing ground-state properties of the Fermi-Hubbard model using a scalable algorithm on a
quantum computer”, Nat. Commun. 13, 5743 (2022).
[601] F. Arute et al., “Observation of separated dynamics of charge and spin in the Fermi-Hubbard model”, arXiv:2010.07965
(2020).

M5.4. Quantum Fourier transform (QFT) metric

This metric evaluates how well a quantum computer can perform the quantum Fourier transform (QFT) [163,
328]. The QFT is widely used as a subroutine in quantum algorithms, most importantly within quantum phase
estimation, which is a central component of Shor’s factorization algorithm [328]. Implementation of QFT has
been demonstrated for small systems, such as within quantum accreditation on 4 qubits [117], while for larger
numbers of qubits higher quality QPUs are required, as well as QEC [602, 603].

Description
In what follows, a brief description of QFT is provided. For a thorough review on QFT, please see Ref. [163]. The
QFT transforms the amplitudes of a quantum state according to the classical discrete Fourier transform (DFT).
The DFT transforms a vector with N elements x = (x0 , x1 , · · · , xN −1 )T into a vector y = (y0 , y1 , · · · , yN −1 )T
according to
N −1
1 X 2πijk/N
yk = √ e xj , (M5.4.1)
N j=0

86
M5. Well-studied tasks execution quality metrics

where k is the index of the element in the transformed vector. Applying the QFT to an arbitrary state
P N −1
j=0 xj |j⟩ gives the following mapping,

N
X −1 N
X −1
xj |j⟩ → yk |k⟩ , (M5.4.2)
j=0 k=0

where the amplitudes yk are given by DFT following Eq. M5.4.1. This mapping can be implemented as a unitary
transformation that can be achieved on a quantum computer.
For an Nq -qubit basis state |j⟩, one may write the number j in its binary form as a string of Nq binary digits
j1 j2 · · · jn , so that each digit jn is either 0 or 1, corresponding to a qubit in a state of |0⟩ or |1⟩, respectively.
By adopting the so-called binary fraction notation where 0.ja ja+1 · · · jb := ja /2 + ja+1 /4 + · · · + jb /2b−a+1 , the
QFT mapping for |j⟩ is equivalent to the following mapping [163]:

|0⟩ + e2πi0.jNq |1⟩ ⊗ |0⟩ + e2πi0.jNq −1 jNq |1⟩ ⊗ · · · ⊗ |0⟩ + e2πi0.j1 j2 ···jNq |1⟩




|j⟩ → . (M5.4.3)
2Nq /2
From Eq. M5.4.3, one can then derive a quantum circuit to implement QFT within the gate-based model by
defining a rotation gate Rk as  
1 0
Rk = k . (M5.4.4)
0 e2πi/2
The first part of the QFT circuit is shown in Fig. M5.4.1. Each qubit has a Hadamard gate applied, followed
by a sequence of controlled-R2 up to controlled-RNq +1−i gates, where i is the index of the qubit, controlled
sequentially by all following qubits. At the end of this circuit, the first qubit gives the last term in the tensor
product in Eq. M5.4.3, the second qubit gives the second-last term, etc. Therefore, a second part of circuit
needs to be added, which consists of SWAP gates acting on each pair of qubits with index i and index Nq + 1 − i,
so that the QFT is obtained.

|j1 ⟩ H R2 · · · RNq −1 RNq |0⟩ + e2πi0.j1 ···jNq |1⟩

|j2 ⟩ • ··· H · · · RNq −2 RNq −1 · · · |0⟩ + e2πi0.j2 ···jNq |1⟩

.. ..
. .
jNq −1 • • · · · H R2 |0⟩ + e2πi0.jNq −1 jNq |1⟩

j Nq • • ··· • H |0⟩ + e2πi0.jNq |1⟩

Figure M5.4.1.: Part of the quantum circuit diagram for QFT. Normalization coefficients are omitted. The output of
this circuit is a QFT-transformed state in a reversed order of qubits. Therefore, SWAP gates need to be added at the
end of this circuit for each pair of qubits with index i and index Nq + 1 − i, so that QFT is achieved.

Measurement procedure
Since QFT is not a metric in itself, one can devise a benchmarking method using QFT on a quantum computer
in different ways. Here a measurement procedure to benchmark the normalized fidelity of hardware outputs
and ideal results is presented. This method follows from the QED-C application oriented benchmarks [118],
wherein it is named as method 1 within three methods of QFT benchmarks.
1. Initialize Nq qubits to an arbitrarily chosen initial state |a⟩, where a ≤ 2Nq − 1 ∈ N.
2. Apply the QFT circuit as outlined in the section above.
3. For each qubit with index i ∈ {1, 2, · · · , Nq }, apply an Rz (π/2i−1 ) gate.

4. Apply the inverse of the QFT circuit.

5. Measure all qubits with a certain number of shots to obtain a probability distribution of output bitstrings.
The number of shots should be chosen based on the desired benchmarking precision. The ideal result
should be a distribution where the bitstring representation of the number a + 1 has a probability of 1, and
all other bitstrings have 0 probability.

87
M6. Speed metrics

6. Compute the normalized fidelity between the hardware output distribution, denoted as Qoutput , and the
ideal distribution, denoted as Qideal , in the following way. Define the fidelity between two probability
distributions P1 and P2 over the same set of random events:
!2
X√
Fc (Q1 , Q2 ) = px,1 px,2 , (M5.4.5)
x

where x is a random event, and px,i is the probability of x happening in the distribution Qi . For the
output of quantum computers, the random events x are the bitstrings. The normalized fidelity between
Qideal and Qoutput is then given by [118]:
 
Fc (Qideal , Qoutput ) − Fc (Qideal , Quni )
Fnorm (Qideal , Qoutput ) = max ,0 , (M5.4.6)
1 − Fc (Qideal , Quni )

where Quni is the uniform distribution.

Assumptions and limitations

• The controlled-Rk gates need to be decomposed into the native gates of the device, which introduces
multiple single and two qubit gates per controlled-Rk . In addition, the QFT circuit contains controlled-
Rk gates for every pair of qubits. Therefore, if the hardware does not support all-to-all connectivity, extra
gates are needed for non-connected qubits.

Source code
A tutorial to run the QFT and calculate the fidelity is provided in the QCMet software repository under:
well_studied_task_execution_quality_metrics/qft. The described measurement procedure follows from
method 1 within the QFT benchmarks in the QED-C application oriented benchmarks suite [118].

Metric references
[117] S. Ferracin, S. T. Merkel, D. McKay, and A. Datta, “Experimental Accreditation of Outputs of Noisy Quantum
Computers”, Phys. Rev. A 104, 042603 (2021).
[118] T. Lubinski, S. Johri, P. Varosy, J. Coleman, L. Zhao, J. Necaise, C. H. Baldwin, K. Mayer, and T. Proctor,
“Application-Oriented Performance Benchmarks for Quantum Computing”, IEEE Trans. Quantum Eng. 4, 1
(2023).
[163] M. A. Nielsen and I. Chuang, “Quantum Computation and Quantum Information”, Camb. Univ. Press 2010.
[328] P. W. Shor, “Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum
Computer”, SIAM J. Comput. 26, 1484 (1997).
[602] K. Mayer, C. Ryan-Anderson, N. Brown, E. Durso-Sabina, C. H. Baldwin, D. Hayes, J. M. Dreiling, C. Foltz,
J. P. Gaebler, T. M. Gatterman, et al., “Benchmarking logical three-qubit quantum fourier transform encoded in
the steane code on a trapped-ion quantum computer”, arXiv:2404.08616 (2024).
[603] B. Park and D. Ahn, “Reducing cnot count in quantum fourier transform for the linear nearest-neighbor archi-
tecture”, Sci. Rep. 13, 8638 (2023).

M6.Speed metrics

M6.1. Time taken to execute a general single- or multi-qubit gate

This metric gives the time taken to apply a gate that executes any general single- or multi-qubit unitary
operation on a quantum computer.

Description
The time taken to execute gates on a quantum computer is the determining factor for the total computation
time of an algorithm. This metric quantifies the time taken to execute one general Nq -qubit gate. These
general Nq -qubit gates can be decomposed into the native gates on the hardware. However, the size of the

88
M6. Speed metrics

circuits in terms of the number of native gates grows exponentially with Nq , and thus in practice this metric
is only to be evaluated for small values of Nq . Below, we provide the description and methodology to measure
the time taken to execute a general single qubit gate, tU 1 , corresponding to Nq = 1. The methodology can
then be generalized to the Nq > 1 case, tU Nq , by using appropriate decompositions of Nq -qubit unitary
operations. The decomposition of Nq -qubit unitary operations are typically decomposed to a combination of
single- and two-qubit

gates, and the theoretical worst case number the required number of CNOT gates scales
as 4Nq − 3Nq − 1 /4 [604]. There are many methods to beat the worst case scaling for Nq -qubits, such as the
cosine-sine matrix decomposition [605]. For 2 qubits, Cartan KAK decomposition is often used [606, 607].
All single qubit gates perform rotations on the Bloch sphere. In order to run generic single qubit circuits,
a quantum processing unit (QPU) must be able to perform arbitrary single qubit rotations. The time taken
to perform such a general single qubit gate provides an estimate relevant for the use of the QPU for general
purpose algorithms. Any such general single-qubit unitary operation can be written in matrix form as

−ieiλ sin(θ/2)
 
cos(θ/2)
U (θ, ϕ, λ) = , (M6.1.1)
−ieiϕ sin(θ/2) ei(λ+ϕ) cos(θ/2)

where θ, ϕ, and λ are parameters corresponding to the rotation amount, and to the angles corresponding to the
axis of rotation, respectively.
The physical implementation of a general single qubit rotation is device dependent, because the type of
native single qubit gates on a QPU depends on the hardware platform. For example, on many devices an Rz
gate is performed virtually by adding a phase to the following gate [2, 233], and as such is effectively applied
instantaneously. Given a calibrated pulse implementing a Rx (π/2) rotation, combined with virtual Rz gates for
arbitrary angles, the general single qubit unitary can be decomposed into the following sequence [233]:
π π π π
U1q (θ, ϕ, λ) = Rz (ϕ − )Rx ( )Rz (π − θ)Rx ( )Rz (λ − ). (M6.1.2)
2 2 2 2
For such a decomposition, the total time taken to execute U (θ, ϕ, λ) on a single qubit, tU 1 , is equal to twice the
times taken to execute a single Rx gate, tRx :
tU 1 = 2tRx . (M6.1.3)
Here tRx includes possible idle gate times between two executions of an Rx gate on the device.
The time taken to implement such general unitary operations depends on the hardware For example, a typical
native single qubit gate in a superconducting qubit takes 25 ns [56]. Using Eq. M6.1.2 yields a general single-
qubit unitary time of 50 ns, since the Rz gates are virtual and instantaneous. Trapped ion single qubit gate
times have been reported to be 10 µs [237]. If virtual Rz gates are used, then Eq. M6.1.2 can again be used,
and this yields a general single-qubit unitary time of 20 µs.

Measurement procedure
Method 1
The native gate time is known by the hardware manufacturer, since the manufacturer sets duration of the pulses
and any required idle time after the pulse needed for each gate. This is done as part of the calibration process.
If the hardware manufacturer provides this information, then the measurement procedure for calculating the
general single qubit gate time is as follows:
1. Hardware manufacturer is to provide the pulse level decomposition of the general single qubit unitary
given in Eq. M6.1.1.
2. For each pulse in this decomposition, the hardware manufacturer is to provide the pulse duration, and
potential idle times between pulses. If the these are Rx pulses, it corresponds to providing tRx
3. Summing up the times of all pulses used to decompose Eq. M6.1.1 one obtains the total gate time. If the
decomposition corresponds to the one presented in Eq. M6.1.2, then the total gate time is tU 1 = 2tRx .
For the time taken to execute a general multi-qubit gate, the same process is followed, however this time the
decomposition of the multi-qubit unitary into native pulses must be provided and reported along with the time.

Method 2
If the vendor does not provide the measurement durations, but the total circuit execution time is provided by
the vendor, one can run circuits with different numbers of general single qubit gates and use the execution times
to estimate the time for each general single qubit gate. The measurement procedure is as follows:

89
M6. Speed metrics

1. Set up a general circuit consisting of nU repetitions of a general single qubit unitary U , as given in Eq.
M6.1.1. The rotation angles corresponding to the unitary U are chosen randomly.
2. For different values of nU , the circuit is executed for a chosen number of shots and the circuit execution
time corresponding to each nU is obtained from the vendor. The total execution time can be written as
tnU = nU tU 1 + tO , where tO includes all contributions to the total execution time that are independent
of the number of gates, for example state preparation and measurement time.
3. Perform a linear fit of the results to extract tU 1 and tO .

To ensure that the time spent executing gates is not negligible compared to other operations such as mea-
surement time, the values of nU must be chosen such that for the largest nU , nU tU 1 ≫ tO .
To use this method to measure the time taken to execute a general multi-qubit gate, one must provide the
decomposition of unitary to native gates. This decomposition must be reported along with the execution time
of the gate.

Method 3
If the vendor does not provide both the measurement durations and the total circuit execution time, one can
obtain an estimate of the gate duration in units of the T1 time of another qubit. If it is known or assumed
that the single qubit gate durations on all qubits are identical, then one can use the method to estimate the
duration of a single qubit gate in units of the T1 time of that qubit. The ratio between the gate time and T1
is important, since it determines how many single qubit gate operations one can implement before the T1 noise
dominates the circuit outcomes. If the vendor has provided the physical T1 times, the gate duration in physical
units can also be estimated.
This method is similar to Method 2, but uses another qubit as a clock. The methodology is as follows:
1. Select two qubits, such that crosstalk noise is small, so that operations on one qubit do not unintentionally
induce significant noise on the other qubit.
2. Set up a general circuit consisting of nU repetitions of a general single qubit unitary U acting on qubit 2,
as given in Eq. M6.1.1. The rotation angles corresponding to the unitary U are chosen randomly.
3. Then for each value of nU :
a) Prepare qubit 1 in the |1⟩ state.
b) Subsequently, on qubit 2, apply the the general single qubit gate U nU times.
c) Measure qubit 1, and ensure that the measurement takes place after the gates have been run on qubit
2. One must include a form of barrier to only allow measurements on qubit 1 after all gates on qubit
2 have been applied.
d) Repeat step (c) for a specified number of shots, then extract p1 (nU ), the probability of measuring
qubit 1 in the |1⟩ state given the nU applications of U on qubit 2.

4. After running these circuits for differing values of nU , fit a decay curve using Eq. M2.1.1 for the data of
p1 (nU ) versus nU , where t, the time before measurement, is replaced by nU .
(n1q)
5. After fitting the decay curve to Eq. M2.1.1, it gives T1 as the T1 time of qubit 1 in units of number
(n1q)
of single qubit gates applied on qubit 2. This then means that T1 = T1 t1q , where T1 is the T1 time of
qubit 1 in units of seconds, and t1q is the time taken to execute a general single qubit unitary on qubit 2.
If the T1 time of qubit 2 is known, then an approximate estimate for the single qubit gate time, tU1q , can be
(n1q)
obtained using tU1q = T1 /T1 .

Assumptions and limitations

• Some special single qubit gates can be natively calibrated operations that have been optimized to run on
the QPU, such as Rx (π/2). These can have shorter gate times than those of the general single qubit gate.
• The time taken to execute a general multi-qubit gate depends on the method of transpilation to the native
gate set [605].
• Methods 2 and 3 rely on repeatedly applying a unitary. It is assumed that the user can disable circuit
optimization in order to prevent the compiler combining these repeated unitary gates into a single unitary.

90
M6. Speed metrics

• Method 3 assumes that either the single qubit gate durations are identical for all qubits, or that the T1
times for the qubits are provided by the vendor.
• Method 3 relies on the assumption that cross-talk between the two qubits is small. A large cross-talk noise
can alter the T1 time on one qubit when gates are executed on another qubit, which results in inaccurate
estimates of the gate speed.

Source code
A tutorial showing how to estimate the ratio of the single qubit gate time to T1 is provided in the QCMet
software repository under:
speed_metrics/time_taken_to_execute_a_general_single_or_multi_qubit_gate.

Metric references
[2] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A Quantum Engineer’s
Guide to Superconducting Qubits”, Appl. Phys. Rev. 6, 021318 (2019).
[56] F. Arute et al., “Quantum Supremacy Using a Programmable Superconducting Processor”, Nature 574, 505
(2019).
[233] D. C. McKay, C. J. Wood, S. Sheldon, J. M. Chow, and J. M. Gambetta, “Efficient Z gates for quantum
computing”, Phys. Rev. A 96, 022330 (2017).
[237] T. P. Harty, D. T. C. Allcock, C. J. Ballance, L. Guidoni, H. A. Janacek, N. M. Linke, D. N. Stacey, and D. M.
Lucas, “High-Fidelity Preparation, Gates, Memory, and Readout of a Trapped-Ion Quantum Bit”, Phys. Rev.
Lett. 113, 220501 (2014).
[604] V. V. Shende, I. L. Markov, and S. S. Bullock, “Minimal universal two-qubit controlled-NOT-based circuits”,
Phys. Rev. A 69, 062321 (2004).
[605] M. Möttönen, J. J. Vartiainen, V. Bergholm, and M. M. Salomaa, “Quantum Circuits for General Multiqubit
Gates”, Phys. Rev. Lett. 93, 130502 (2004).
[606] R. R. Tucci, “An introduction to Cartan’s KAK decomposition for QC programmers”, arXiv:quant-ph/0507171
(2005).
[607] S. S. Bullock and G. K. Brennen, “Canonical decompositions of n-qubit quantum computations and concurrence”,
J. Math. Phys 45, 2447 (2004).

M6.2. Time to measure qubits

The time to measure a qubit is the time taken to perform a projective measurement in the computational basis
on that qubit.

Description
In order to obtain the result of a quantum computation, some of the qubits in the device need to be measured
depending on the quantum circuit being run. This metric quantifies the time taken to perform such a projective
measurement on a particular qubit. This does not include any quantum gates applied before the measurement
to change the basis of measurement. Measurement times for quantum computers vary from device to device even
within the same hardware platform. For example, for superconducting qubits, measurement times of 100ns [608]
and 500ns [148] have been reported. For trapped ion qubits, measurement times of 11µs [187] and 46µs [609]
are values that have been reported.

Measurement procedure
The measurement procedure for this metric is analogous to the procedure for the gate times. Methods 2 and 3
rely on the capability to perform mid-circuit measurements (see metric M1.4). Method 2 further relies on the
availability of the time taken for a quantum computation. Method 3 only allows obtaining the measurement
time in units of single qubit gate durations.

Method 1
The information about measurement times on each qubit is obtained from the vendor device documentation.

91
M6. Speed metrics

Method 2
If the vendor does not provide the measurement durations, but the total circuit execution time is provided by
the vendor, one can use method 2 described in metric M6.1, with the general single qubit gates replaced by
mid-circuit measurements.

Method 3
If the vendor does not provide both the measurement durations and the total circuit execution time, one can use
method 3 described in metric M6.1, with the general single qubit gates replaced by mid-circuit measurements.

Assumptions and limitations

• Method 2 assumes that the time taken to run the circuits on the device is available.
• Methods 2 and 3 assume that mid-circuit measurements can be performed, since the measurements needs
to be repeated inside each executed quantum circuit.

Source code
A tutorial for the time taken to measure qubits obtained using method 3 is provided in the QCMet software
repository under:
speed_metrics/time_to_measure_qubits.

Metric references
[148] Google Quantum AI, “Suppressing Quantum Errors by Scaling a Surface Code Logical Qubit”, Nature 614, 676
(2023).
[187] S. Crain, C. Cahall, G. Vrijsen, E. E. Wollman, M. D. Shaw, V. B. Verma, S. W. Nam, and J. Kim, “High-Speed
Low-Crosstalk Detection of a 171Yb+ Qubit Using Superconducting Nanowire Single Photon Detectors”, Nat.
Commun. 2, 1 (2019).
[608] Y. Sunada, S. Kono, J. Ilves, S. Tamate, T. Sugiyama, Y. Tabuchi, and Y. Nakamura, “Fast Readout and Reset of
a Superconducting Qubit Coupled to a Resonator with an Intrinsic Purcell Filter”, Phys. Rev. Appl. 17, 044016
(2022).
[609] S. L. Todaro, V. B. Verma, K. C. McCormick, D. T. C. Allcock, R. P. Mirin, D. J. Wineland, S. W. Nam,
A. C. Wilson, D. Leibfried, and D. H. Slichter, “State Readout of a Trapped Ion Qubit Using a Trap-Integrated
Superconducting Photon Detector”, Phys. Rev. Lett. 126, 010501 (2021).

M6.3. Time to reset qubits

This metric quantifies the time taken to reset all qubits in a device after a measurement to make them reusable
for the next circuit execution. Typically, quantum circuits need to be run a large number of times, also called
the number of shots, in order to extract useful results [163]. The time spent by the device in-between performing
these repetitions contributes to the total computation time [610]. After qubits are measured at the end of a
quantum circuit, they typically need to be reset in preparation for the next circuit. This metric quantifies the
time taken to reset all the qubits in the device.

Description
When a quantum computer finishes running a circuit, the qubits in the device typically undergo either an active
or a passive reset in preparation for running the next circuit or next repetition of the current circuit [610, 611].
A passive reset occurs when the device waits for the qubit to decay to the |0⟩ state [2, 611]. This typically is
on the order of the longest T1 time of all qubits in the device. Alternatively, active reset is done when the reset
time required is much lower than the T1 time or the qubit environment is hot on the scale of the qubit transition
frequency [611]. One simple form of active reset is where after measuring the current state of the device, pulses
are sent to the qubits measured to be the |1⟩ state to bring them to the |0⟩ state [612]. There are also some
strategies where no reset time is required, such as the so-called restless measurement. In that case, instead of
resetting qubits, the measured states of the qubits are defined as the |0⟩ states for the next circuit execution
and the redefinitions are accounted for in post-processing [613].

92
M6. Speed metrics

Measurement procedure
The methods for measuring the reset time are similar to the ones in metrics M6.1 and M6.2. Unlike those,
however, a different qubit cannot be used a clock to estimate the reset time, because the qubit corresponding
to the clock is reset between shots.

Method 1
As in metrics M6.1 and M6.2, the first method is to obtain the details on the reset time from the vendor device
documentation.

Method 2
As in metrics M6.1 and M6.2, if the total circuit execution time is available, it can be used to estimate the reset
times.
The estimation method requires running circuits composed of repetitions of a circuit. Denote the circuit that
is repeated - excluding any measurements - as C. The circuit to be run on the device is constructed by repeating
C for a number of times. If a form of barrier operation is available, it can be added between the repetitions of
C to ensure there is no overlap between each repetition. After the repetitions, all qubits are measured.
The measurement procedure involves independently changing the number of shots and the number of repeti-
tions of C inside each quantum circuit. The time taken for each shot includes the time spent on running C, as
well as the time to measure and reset the qubits. The time spent on running each quantum circuit depends on
the number of repetitions of C. Thus, the total computation time ttot for running ns shots of a circuit composed
of repeating C for m times is given by

ttot (ns , m) = to + ns (m tC + tm + treset ), (M6.3.1)

where to is the overhead time which occurs independently of the circuit, tC is the actual time to run C, tm is
the maximum of the measurements times of all the qubits, and treset is the time to reset the device. The method
to estimate treset is given below. Note that to is different to the overhead time in metric M6.1.
To get an estimate of treset , one can evaluate ttot (ns , m) for a varying number of ns and m and use it to
estimate tm + treset . Then, one can use prior knowledge of tm - obtained via methods described in metric M6.2
- to get an estimate of treset .

Assumptions and limitations

• Method 2 above assumes that the time taken to run the circuits on the device is available.
• Method 2 assumes that the physical time to run a circuit composed of r repetitions of c takes time rtc .
If a barrier operation is not available, some gates in the different repetitions of c might run in parallel,
affecting the estimated treset time.

• Method 2 assumes that the maximum measurement time of the qubits can be obtained via methods
described in metric M6.2.

Source code
A tutorial for the time taken to reset qubits obtained using method 2 is provided in the QCMet software
repository under:
speed_metrics/time_to_reset_qubits.

Metric references
[2] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A Quantum Engineer’s
Guide to Superconducting Qubits”, Appl. Phys. Rev. 6, 021318 (2019).
[163] M. A. Nielsen and I. Chuang, “Quantum Computation and Quantum Information”, Camb. Univ. Press 2010.
[610] M. D. Reed, B. R. Johnson, A. A. Houck, L. DiCarlo, J. M. Chow, D. I. Schuster, L. Frunzio, and R. J. Schoelkopf,
“Fast reset and suppressing spontaneous emission of a superconducting qubit”, Appl. Phys. Lett. 96, 203110
(2010).
[611] K. Geerlings, Z. Leghtas, I. M. Pop, S. Shankar, L. Frunzio, R. J. Schoelkopf, M. Mirrahimi, and M. H. Devoret,
“Demonstrating a Driven Reset Protocol for a Superconducting Qubit”, Phys. Rev. Lett. 110, 120501 (2013).
[612] P. Magnard et al., “Fast and Unconditional All-Microwave Reset of a Superconducting Qubit”, Phys. Rev. Lett.
121, 060502 (2018).

93
M6. Speed metrics

[613] M. Werninghaus, D. J. Egger, and S. Filipp, “High-Speed Calibration and Characterization of Superconducting
Quantum Processors without Qubit Reset”, PRX Quantum 2, 020324 (2021).

M6.4. Overall device speed on reference tasks

This metric gives the time taken for a quantum computer to run reference tasks, such as the well studied tasks
described in the preceding sub-sections.

Description
For practical evaluation of the well studied tasks on a quantum computer, the overall device speed for running
a task is an important aspect when choosing which hardware platform use. Therefore, when choosing a set
of reference tasks for benchmarking purposes, one needs to include widely-used quantum algorithms that are
representative for real-world applications [118].
When submitting a task to a remote quantum hardware platform, it is typical that the circuits in the task
are first put into a waiting queue before the hardware becomes available to execute them [614]. This queue
time should be excluded from the benchmarking process, as well as any time overheads not related to the task,
such as network communication times. Hence, the metric of overall device speed on reference tasks is measured
solely by the time taken to execute the circuits on the device.

Measurement procedure
The measurement procedure presented here uses a set of reference tasks which include the well-studies tasks
described in the preceding sub-sections, namely variational quantum eigensolver (VQE, metric M5.1), quantum
approximate optimization algorithm (QAOA, metric M5.2), Fermi-Hubbard model simulation (FHMS, met-
ric M5.3) and quantum Fourier transform (QFT, metric M5.4). Other representative sets of well studied tasks
can be those used for the determination of the algorithmic qubits metric (metric M4.3), as well as the time taken
to evaluate the quantum volume (metric M4.1), which is related to the definition of the circuit layer operations
per second (CLOPS) proposed by IBM [137]. Note that the set of reference tasks can be extended to include
other widely-used quantum algorithms as needed. The procedure then is:
1. Run the chosen set of reference tasks on the hardware.
2. For each reference task in the set, record and report the time taken to execute the circuits in seconds.

Assumptions and limitations

• Care should be taken depending on the hardware platform used, because this metric is dependent on
how each manufacturer reports the timings. For example, some vendors may report the total time from
submission of circuits to completion, which includes the queue time. In this case the queue time must be
known and must be subtracted from the total time. Due to manufacturers reporting time differently, it
can be difficult to compare speeds of different hardware platforms.
• The well studied tasks can include the execution of quantum circuits and also of classical computations
within a hybrid quantum-classical approach. The metric includes both the classical CPU time in addition
to the time spent executing the quantum circuits.

Source Code
The source code for the tasks of VQE (metric M5.1), QAOA (metric M5.2), FHMS (metric M5.3), QFT
(metric M5.4), and algorithmic qubits (metric M4.3) is given in the corresponding sub-sections.

Metric references
[118] T. Lubinski, S. Johri, P. Varosy, J. Coleman, L. Zhao, J. Necaise, C. H. Baldwin, K. Mayer, and T. Proctor,
“Application-Oriented Performance Benchmarks for Quantum Computing”, IEEE Trans. Quantum Eng. 4, 1
(2023).
[137] A. Wack, H. Paik, A. Javadi-Abhari, P. Jurcevic, I. Faro, J. M. Gambetta, and B. R. Johnson, “Quality, Speed, and
Scale: Three Key Attributes to Measure the Performance of near-Term Quantum Computers”, arXiv:2110.14108
(2021).
[614] G. S. Ravi, K. N. Smith, P. Gokhale, and F. T. Chong, “Quantum Computing in the Cloud: Analyzing job and
machine characteristics”, 2021 IEEE Int Symp Workload Charact. IISWC (2021), 39.

94
M8. Metrics for quantum annealers

M7.Stability metrics

M7.1. Standard deviation of a specified metric evaluated over a time interval

The standard deviation of a metric measured multiple times over a time scale of interest quantifies the variation
of that metric over time. A larger standard deviation corresponds to larger variation of the metric, and hence
corresponds to reduced stability of the device.

Description
A general discussion on the time dependent changes of the metrics is given in Sec. 2. This metric based on the
standard deviation quantifies the overall stability of the measured metric over a specified time scale.

Measurement procedure
1. Determine the the time scale of interest to measure fluctuations of the specified metric, which defines a
time interval, t0 , that spans the considered time scale. This can be on the scale of less than a second,
seconds, minutes, days, weeks or months.

2. Determine the value of the metric a specified number of times, Nt , over the time scale t0 . The times at
which the metric is evaluated are to be distributed approximately uniformly over t0 . A larger Nt provides
a better sampling of the time-dependent fluctuations of the metric.
3. Calculate the average and the standard deviation of the metric over the Nt measurements acquired over
the t0 time interval.

Assumptions and limitations

• The smallest time scale over which the stability can be measured is bounded by the time taken to evaluate
that metric.

Source code
A tutorial for calculating the standard deviation of a specified metric over a time interval, with the example
metric being the Clifford randomized benchmarking average gate error, is provided in the QCMet software
repository under:
stability_metrics/amount_of_fluctuations_of_metrics_over_time.

M8.Metrics for quantum annealers

M8.1. Single qubit control errors

Single qubit control errors are a type of error where the device does not implement the Hamiltonian it was
designed to, and a specific case is where the hi terms from Eq. 4.4.1 deviate from the programmed values.
There are several ways to measure this, which yield different results based on the timescales they probe. Here
this error is denoted as a control error (sometimes also referred to as calibration error [615]), although there are
numerous physical origins of such errors, many of which do not stem directly from the controls.

Description
Not implementing the intended Hamiltonian on a quantum annealer is potentially a serious problem [616],
and can lead to drastically different solutions to the target problem [617]. Since the problem statement on an
annealer is fundamentally analog, noise from the environment, or even just lack of precision in the controls,
can lead to the statement of the problem, which the device actually solves, being different from the one the
user intended. There is extensive work on error mitigation strategies which could help combat these errors, and
these methods have been shown to be somewhat successful [616, 618–621].

95
M8. Metrics for quantum annealers

In this section three different methods to measure this noise are presented, which probe different time scales,
and probe the devices under different conditions. Because single body errors can be more efficiently measured,
only the single body error terms hi are discussed, rather than Jij from Eq. 4.4.1. The error model is therefore
X
Herror = ζi Z i . (M8.1.1)
i

The methods discussed here could likely be extended to measure two-body biases as well. The three methods
discussed here have all been outlined, as well as experimentally compared, in Ref. [622].
The most straightforward method to measure these errors is sampling. This method consists of programming
a Hamiltonian hi = Jij = 0 ∀i, j and counting the number of times each qubit is seen in the 1 or 0 configuration.
The annealing time may affect the error rate measured this way as well, since it will determine the parameters at
which dynamics stop. For this reason the error should be treated as being annealing time dependent, although
in practice this dependence may not be strong enough to significantly affect the results. Up to statistical
fluctuations, the counts of 1 and 0 measurements should be equal if no noise is present. A bias between the two
states can then be inferred from any difference which is seen. For example, if the device is assumed to be in
thermal equilibrium at a known temperature, then the energy difference can be back-calculated from the single
qubit Boltzmann distribution. A disadvantage of this method is that the quantity calculated is the average for
the time period, therefore high frequency fluctuations, with a frequency much faster than the sampling time,
will be missed.
A more complicated method, which is sensitive to higher frequency noise elements, is to again program
a trivial Hamiltonian hi = Jij = 0 ∀i, j, but instead examine the autocorrelation effects in the individual
Z measurements. Again assuming the output obeys a thermal distribution at a known temperature, Taylor
expanding and taking only the first non-trivial term, one has

ζ (i) ζ (j)
Cij = , (M8.1.2)
T2
where T is the known temperature, and the superscripts in this equation indicate sample number rather than
qubit number. Full details can be found in the supplemental material of Ref. [622]. Taking the Fourier transform
of this quantity, one can extract the spectral density of the noise sk , then the total RMS noise can then be
shown to be approximately v
uN  
T uX k−1
q
ζ2 = t |sk |2 exp −2πi , (M8.1.3)
N N
k=1

where N is the number of samples taken. This method is sensitive to a much broader range of frequencies, but
is still limited by the rate at which the samples are taken.
Finally, the third method to measure this noise is to program the Hamiltonian for an Ising spin chain with
frustrated boundaries, given by
n−1
X
Hdomain = J( −Zi Zi+1 + 2(Z1 − Zn )) (M8.1.4)
i=1

where n is the number of qubits. As long as J is large and positive, then the annealer will almost exclusively
sample from states where the chain contains a single domain-wall somewhere on the chain, which form a
degenerate manifold. This is reminiscent of the domain-wall encoding proposed in Ref. [349] and also the
simulation extension in Ref. [623], but crucially contains an extra factor of 2, which excludes the possibility of
all qubits taking the same state from the degenerate manifold. The measurements are then taken by measuring
the relative position of the domain wall on the chain averaging over different local gauges, which randomly change
the definition of |1⟩ and |0⟩ on the qubits, and potentially different ways of programming the Hamiltonian onto
the physical qubits of the device. It can be show mathematically that averaging Boltzmann distributions over
different noise realization yields a characteristic “U” shape [622], which can be related to the noise strength.
In practice the noise strength can be calculated by fitting a numerically calculated Boltzmann distribution to
the actual distribution on the device. For flux-qubit devices, either a correction needs to be applied to the
probability of finding the domain wall on a terminal site, or these points need to be excluded from the fit.
The domain-wall method has the advantage that the effective lower frequency cutoff is set by the dynamics
of the device, and not by the way in which the samples are obtained, a property which could be particularly
important in comparing devices where timescales related to sampling are very different. It is also the only of
these methods which can measure while some of the couplers are turned on.

96
M8. Metrics for quantum annealers

Measurement procedure
For all three methods the bias strength is extracted as a numerical ratio between the strength of the average
errors when the dynamics cease to be effective (freezed out) and the temperature of the device, which is usually
known. When combined with the temperature estimates using the methods described in metric M8.3, this could
be used to calculate the single-qubit error as a fraction of total allowed h values.
The procedure for Method I, the basic sampling method, is as follows:
1. Turn all single and two body Hamiltonian terms off.
2. Perform annealing multiple times to collect statistics, only to track counts rather than the order in which
they were taken.
3. Back calculate bias strength based on assumption of Boltzmann distribution.
The procedure for Method II, the autocorrelation method, is as follows:
1. Turn all single and two body Hamiltonian terms off.
2. Perform annealing multiple times to collect statistics, and track the data time-series, not just aggregate
statistics.
3. Calculate autocorrelation, Fourier transform and integrate over all frequency components.
The procedure for method III, the domain wall method, is as follows:
1. Program frustrated spin chain Hamiltonians (Eq. M8.1.4); a length around 10 works well in practice on
current devices.
2. Perform anneals and collect statistics on domain wall positions, averaging over gauges and chain positions.
3. If there are significant susceptibility effects, which occur when single body terms affect two-body terms
that they overlap with, one needs to perform background susceptibility corrections, or remove terminal
sites from statistics.
4. Fit against numerically calculated distribution to extract bias strengths.

Assumptions and limitations

• All methods assume that the final distribution is a Boltzmann distribution with a known temperature. In
the case of more coherent operation the methods would need to be adapted as discussed below:
– In Methods I and III, the model of system is only used for the final analysis. In a more coherent
systems they could be adapted by comparing to a different classical model.
– In a highly coherent, low control error situation, method III would instead yield a “particle-in-a-box”
distribution, which again could be used in fitting.
– Method II would be more difficult to extend to a more coherent case, but may be possible.
• Method II involves taking a Taylor series of a Boltzmann distribution, and therefore assumes that the
errors are small compared to the temperature.
• An important quantity for computation is the dimensionless ratio of control errors to the strength of the
Ising terms when the annealing dynamics effectively stop (the so-called freeze time), and this quantity is
difficult to know exactly.
• Methods I and II both are limited in the timescales they can probe by the sampling procedure:
– Method I is limited by the total time over which samples are taken. This is problematic, because it
leads to a tradeoff between statistical accuracy, which increases with number of samples taken, and
the range of frequencies it is sensitive to; the highest frequency which can be detected decreases with
total sampling time, because the samples are averaged over this time period, and higher frequencies
would cancel out.
– Method II is limited by the time between samples; this means that errors faster than the time between
the individual samples will be missed.
– Method III should in principle capture all noise which can be treated as a control error, as there is
no maximum or minimum frequency imposed by the measurement method.
• Only method III performs measurements when some of the coupling elements are active.

97
M8. Metrics for quantum annealers

Source code
A tutorial on calculating the single qubit control errors using the domain-wall method is provided in the QCMet
software repository under:
quantum_annealers/single_qubit_control_errors.

Metric references
[349] N. Chancellor, “Domain Wall Encoding of Discrete Variables for Quantum Annealing and QAOA”, Quantum Sci.
Technol. 4, 045004 (2019).
[615] K. L. Pudenz, T. Albash, and D. A. Lidar, “Quantum annealing correction for random Ising problems”, Phys.
Rev. A 91, 042302 (2015).
[616] K. C. Young, R. Blume-Kohout, and D. A. Lidar, “Adiabatic Quantum Optimization with the Wrong Hamilto-
nian”, Phys. Rev. A 88, 062314 (2013).
[617] T. Albash, V. Martin-Mayor, and I. Hen, “Analog Errors in Ising Machines”, Quantum Sci. Technol. 4, 02LT03
(2019).
[618] K. L. Pudenz, T. Albash, and D. A. Lidar, “Error-Corrected Quantum Annealing with Hundreds of Qubits”, Nat.
Commun. 5, 3243 (2014).
[619] K. Nishimura, H. Nishimori, A. J. Ochoa, and H. G. Katzgraber, “Retrieving the Ground State of Spin Glasses
Using Thermal Noise: Performance of Quantum Annealing at Finite Temperatures”, Phys. Rev. E 94, 032105
(2016).
[620] W. Vinci and D. A. Lidar, “Scalable Effective-Temperature Reduction for Quantum Annealers via Nested Quan-
tum Annealing Correction”, Phys. Rev. A 97, 022308 (2018).
[621] J. Bennett, A. Callison, T. O’Leary, M. West, N. Chancellor, and V. Kendon, “Using copies can improve precision
in continuous-time quantum computing”, Quantum Sci. Technol. 8, 035031 (2023).
[622] N. Chancellor, P. J. D. Crowley, T. Ðurić, W. Vinci, M. H. Amin, A. G. Green, P. A. Warburton, and G. Aeppli,
“Error Measurements for a Quantum Annealer Using the One-Dimensional Ising Model with Twisted Boundaries”,
npj Quantum Inf. 8, 73 (2022).
[623] M. Werner, A. García-Sáez, and M. P. Estarellas, “Quantum simulation of one-dimensional fermionic systems
with Ising Hamiltonians”, arXiv:2406.06378 (2024).

M8.2. Size of the largest mappable fully connected problem

Since superconducting qubit based quantum annealers have limited connectivity in the underlying hardware
graph, problems which are incompatible with this graph must be mapped in a way which maps multiple physical
qubits to the same variable in the problem which is being solved. The physical qubits are strongly coupled in a
way which forces them to take the same value, and thus correspond to a single binary variable. For this reason
the size of problem which can be mapped with arbitrary interactions between all variables is typically smaller
than the total number of qubits.

Description
Minor embedding was first proposed as a method of mapping problems which are not compatible with the
hardware graph of an annealer by Choi in Ref. [624]. Minor embedding is a process by which multiple individual
qubits are strongly coupled within a graph minor, a connected subgraph of the original graph, to form a new
logical variable. In a sequel paper a prescription to minor embed to the D-Wave hardware graph available at the
time was demonstrated [625]. An alternative method of mapping problems based on parity and requiring fourth
order interactions was later proposed by Lechner, Hauke, and Zoller [626], known as the LHZ method. While
fourth order interactions are not directly available in most quantum annealing hardware, various mappings can
be used to achieve this encoding by building higher order interactions from lower order ones [627–629]. There
have also been new methods proposed along these lines which do not require higher-order coupling [630].
The largest fully connected graph which can be minor embedded into a specified graph is given by one plus
the treewidth of the graph [631]. In general, finding the optimal tree decomposition of a graph, necessary for
computing the treewidth, is a hard computational problem, and minor embedding is also computationally hard.
In practice, quantum annealers usually have an interaction graph with regular structure, making the process
of computing a clique embedding or tree decomposition substantially easier. Tree decomposition is a process
in which nodes are merged until the graph has a tree structure, which is one containing no loops. A useful
quantity here is known as the treewidth. The idea is to look at decompositions which combine nodes to turn
a graph with loops into one without (a “tree”). The maximum number of nodes which must be combined

98
M8. Metrics for quantum annealers

in the best tree decomposition (minus one by convention) is known as the treewidth. The treewidth of the
best tree decomposition determines the largest clique which can be embedded. For example, the embedding in
Ref. [625] was computed by hand. If some regular structure, which enables finding an embedding by hand, is
not present, then software to compute graph embeddings can be used, for example the minor-miner software
developed by D-Wave Systems Inc.1 which uses the algorithm described in Ref. [632] for heuristic embedding.
Such an embedding has no guarantee to be optimal, but can act as a lower bound of the largest fully connected
problem size which can be mapped.

Measurement procedure
For a connectivity corresponding to a highly structured graph, as many devices including current D-Wave
devices will have, an optimal embedding may be computable by hand, as was done in Ref. [625]. This method
is preferable, because one may be able to obtain a guarantee of maximum size of the graph, not simply a
lower bound. Alternatively, approximate algorithms for finding minor embeddings exist, like those described in
Ref. [632], and can be used as a building block for embedding software such as D-Wave minor-miner2 .
It is important to note that unlike many other metrics, an actual device does not need to be available to test
the size of problem which could be mapped. For that reason this metric is ideal for devices which have not yet
been fabricated to predict this aspect of their performance.

Assumptions and limitations

If a heuristic is used, then this metric is only a lower bound for the best possible mapping, since there is no
guarantee that a better heuristic could not find a better one. However, in practice this may be close enough,
since the existence of better embeddings, which cannot practically be found, will not have major implications
for performance. A larger limitation is that optimization problems are not always fully connected, and may
be possible to be embedded substantially more efficiently than a fully connected graph. This point becomes
particularly relevant if a special-purpose annealer is developed with a hardware graph designed to match the
problem structure [633]. In these cases, it would be better to consider the largest problem with a relevant
structure which can be mapped. Even for general purpose quantum annealers, if problems with a specific
interaction structure are of specific interest to the user, then a more targeted approach using that structure
may be justified. A concrete example here is the common interaction and constraint structure to all travelling
salesman and quadratic assignment problems as used in Ref. [634].

Source code
An open source code for minor embedding (minor-miner) can be found at https://github.com/dwavesystem
s/minorminer.

Metric references
[624] V. Choi, “Minor-Embedding in Adiabatic Quantum Computation: I. The Parameter Setting Problem”, Quantum
Inf. Process. 7, 193 (2008).
[625] V. Choi, “Minor-Embedding in Adiabatic Quantum Computation: II. Minor-universal Graph Design”, Quantum
Inf. Process. 10, 343 (2011).
[626] W. Lechner, P. Hauke, and P. Zoller, “A Quantum Annealing Architecture with All-to-All Connectivity from
Local Interactions”, Sci. Adv. 1, e1500838 (2015).
[627] M. Leib, P. Zoller, and W. Lechner, “A Transmon Quantum Annealer: Decomposing Many-Body Ising Constraints
into Pair Interactions”, Quantum Sci. Technol. 1, 015008 (2016).
[628] A. Rocchetto, S. C. Benjamin, and Ying Li, “Stabilizers as a Design Tool for New Forms of the Lechner-Hauke-
Zoller Annealer”, Sci. Adv. 2, e1601246 (2016).
[629] N. Chancellor, S. Zohren, and P. A. Warburton, “Circuit Design for Multi-Body Interactions in Superconducting
Quantum Annealing Systems with Applications to a Scalable Architecture”, npj Quantum Inf. 3, 21 (2017).
[630] A. Palacios, A. Garcia-Saez, and M. P. Estarellas, “A scalable 2-local architecture for quantum annealing of
all-to-all Ising models”, arXiv:2404.06861 (2024).
[631] R. Diestel, “Graph Theory: 5th Edition”, Grad. Texts Math. Springer Berl. Heidelb. (2017).
[632] J. Cai, W. G. Macready, and A. Roy, “A Practical Heuristic for Finding Graph Minors”, arXiv:1406.2741 (2014).
[633] J. Roffe, S. Zohren, D. Horsman, and N. Chancellor, “Decoding Quantum Error Correction with Ising Model
Hardware”, arXiv:1903.10254 (2019).

1 https://github.com/dwavesystems/minorminer
2 https://github.com/dwavesystems/minorminer

99
M8. Metrics for quantum annealers

[634] J. Berwald, N. Chancellor, and R. Dridi, “Understanding Domain-Wall Encoding Theoretically and Experimen-
tally”, Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 381, 20210410 (2023).

M8.3. Dimensionless sample temperature

The dimensionless sample temperature is the ratio of the coupling strength to the environment temperature
when the dynamics of the quantum annealer stops, known as the freeze time. The dimensionless sample tem-
perature determines the rate of thermal errors. For optimisation applications, a sampling temperature which is
as low as possible is ideal. Certain applications are based on thermal sampling and in these case a finite sam-
pling temperature which is application dependant is desired. The freeze time, and therefore the temperature, is
dependent on the problem which is programmed into an annealer. Since these devices are programmable, com-
parisons between devices can be made using the same problem, although for a fair comparison a problem should
be chosen which is a subgraph of both hardware graphs to avoid additional mapping overheads complicating
the comparison. A review of techniques to estimate temperature can be found in Ref. [635].

Description
Quantum annealers acting in a highly dissipative regime are known to produce statistical distributions, which are
well approximated by thermal distributions. This fact has been used experimentally for probabilistic inference
in Refs. [636, 637], which also included early estimation methods for the dimensionless temperature of the
distribution. This fact has numerous applications in machine learning, including for example Boltzmann and
Helmholtz machines [638–641].
Since quantum annealers do not perfectly sample thermal distributions, any temperature extracted will neces-
sarily be approximate. Known methods for estimating temperature rely on comparing between the distribution,
or possibly multiple distributions, returned by the annealer and a classical distribution. Different approaches
are reviewed in Ref. [635]. In all approaches, with the exception of one which we discuss later, several ingredients
are needed:
• An objective function(s) against which to sample
• A classical sampling technique which operates at a known temperature and can return a good quality
distribution for the given objective function(s)
• A measure of difference between distributions
The annealing schedule does not need to be known to estimate the sampling temperature. The two approaches
explored in Ref. [635] use dynamic programming and parallel tempering techniques to approximate the true ther-
mal distribution with two types of randomly generated problems. The difference comes in how the distributions
are compared. The first technique is to minimize the Kullback-Leibler divergence, which statistically corre-
sponds to finding the the most likely temperature to produce the given distribution, and is often abbreviated
as KL. This quantity takes the form
 
X Qa (x)
DKL [Qa , Qb ] = Qa (x) log , (M8.3.1)
x
Qb (x)

where Qa and Qb are two arbitrary probability distributions, and DKL is the KL divergence. It is important
to note that this formula is not symmetric, and in particular will diverge if even a single Qb (x) = 0 when
Qa (x) ̸= 0. For this reason, the samples from the physical annealer should always be taken as Qa (x), since
otherwise a sample which is never seen in the experiment would cause the distribution to diverge, and the
measure would be ill-defined unless the annealer exhaustively explored the solution space. Note that based on
energy, the Boltzmann probability of a given state can still be estimated, even if it was not directly seen within
the sample set, see Ref. [635] for details. A symmetrized version of this formula exists, where the divergence
is removed, but it was not used in that study. In practice, for models of any reasonable size and temperature,
the likelihood of seeing exactly the same state in two distributions is vanishingly small, so more sophisticated
techniques are needed, such as binning by temperature, or comparing measured quantities [635].
An alternative approach, which does not resort to directly comparing probabilities, is to instead compare
measured quantities, which should be smaller the closer the distribution is to a true thermal distribution. In
Ref. [635] the quantity chosen was the pairwise correlation between variables, but numerous other choices
could be made.
Measurements performed within other works tend to be generalizations on the same central theme, for example
Ref. [636] compared the decoding of different qubits when the Ising model used within the annealer was treated
as a classical error correction code, but did the fitting over multiple objective functions.

100
M8. Metrics for quantum annealers

The only estimate of the temperature that does not require a classical sampling method is given in Ref. [642].
This method relies on rescaling the problem itself, and using the change in probabilities to observe different
energies to calculate the difference in temperature. The used formulation is in terms of inverse temperature.
From this change, the absolute temperature can be calculated based on the known scaling. This technique
relies on the assumption that rescaling the problem statement corresponds linearly to a change in temperature.
However, such assumption is often not true in real quantum annealers. This is often a bad assumption, because
the freeze time of the annealer will depend on the energy scale of the problem.

Measurement procedure
The most common procedure for measuring temperature can be applied as follows:
1. Decide on a Hamiltonian to program for the estimation, this may or may not require minor embedding
(see M8.2).
2. Choose a quantity to measure and use to fit a temperature; how global or local this is will affect the
temperature values measured. [635]; this could also be a KL divergence between the annealer outputs
and a classically determined distribution.
3. Measure the chosen quantity experimentally on the annealer.
4. Perform fitting against a classical estimate of the same quantity using methods such as Monte Carlo.
5. The temperature value which gives the best agreement is the estimated value.
An alternative method presented in Ref. [642] proceeds as follows:
1. Decide on a Hamiltonian to program for the estimation.
2. Choose two bins of energy ranges with different midpoints, such that the annealer has a non-negligible
probability of sampling within each bin.
3. Rescale the overall Hamiltonian and plot the natural logarithm of the ratio of probabilities in each bin
versus the energy difference after rescaling; this plot should be a straight line.
4. The slope of the line is the inverse temperature scaled by Boltzmann’s constant.

Assumptions and limitations

There are numerous challenges to overcome in this area. Key issues are:
1. The distribution returned by an annealer will never exactly be a thermal distribution.
2. Generating a thermal distribution to compare with can be a computationally challenging problem, de-
pending on the objective function.
3. The dimensionless temperature observed may differ for different objective functions.
4. An interesting operating regime for annealers is the diabatic regime, where the final distribution is no
longer approximately thermal.
As a consequence of point 1, systematic effects have been observed in temperature estimations, such as the
more global features of the distribution, which the estimate captures. This effect was reported in Ref. [635],
which called it global warming. This presents a considerable issue, since both global and more local sampling
are likely to be important, so that multiple estimates, which give different results, can be considered equally
valid. Temperature estimates can still be used as an indicative quantity.
Point 2 is a computational difficulty. Depending on the goal of the comparison, it can be circumvented by
using an objective function for which it is not hard to obtain the thermal distribution classically. This approach
was taken for example in Ref. [634]. Alternatively, a large amount of classical computing power may be used,
although at some point this may be impractical. It is worth noting that because of point 3, it may not always
be suitable to choose a classically tractable objective function, since the cases of computational interest will be
those where the problem is classically intractable.
With regards to point 4 this metric is only valid when the timescale of the anneal is effectively longer than
the timescale associated with the coupling to its thermal bath. This regime is sometimes called the quasistatic
regime, inspired by a Kibble-Zurek type approximation. Until recent technological improvements, D-Wave
superconducting quantum annealers could only operate in this regime. However, recent work has demonstrated
operation in a more coherent regime due to the capability to perform faster anneals [643]. Similarly, as more

101
M9. Metrics for boson sampling devices

annealers operating on intrinsically less noisy platforms come into existence, experiments within this regime are
likely to become more common.
In principle, one can always choose to run an annealer slowly enough that it operates in the quasistatic
regime. However, this approach has the drawback that it won’t give accurate information about the coherent
regime, where closed system approximations are valid as opposed to a Kibble Zurek type approximation of
perfect thermalisation and freezing. A comparison only in the quasistatic regime therefore may miss the most
relevant features for performance. There is a need for metrics which measure a quantum annealers’ performance
for sampling in a coherent regime, however we are not aware of any such metrics at the time of writing.

Metric references
[634] J. Berwald, N. Chancellor, and R. Dridi, “Understanding Domain-Wall Encoding Theoretically and Experimen-
tally”, Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 381, 20210410 (2023).
[635] J. Raymond, S. Yarkoni, and E. Andriyash, “Global Warming: Temperature Estimation in Annealers”, Front.
ICT 3 (2016).
[636] N. Chancellor, S. Szoke, W. Vinci, G. Aeppli, and P. A. Warburton, “Maximum-Entropy Inference with a Pro-
grammable Annealer”, Sci. Rep. 6, 22318 (2016).
[637] M. Benedetti, J. Realpe-Gómez, R. Biswas, and A. Perdomo-Ortiz, “Quantum-Assisted Learning of Hardware-
Embedded Probabilistic Graphical Models”, Phys. Rev. X 7, 041052 (2017).
[638] S. H. Adachi and M. P. Henderson, “Application of Quantum Annealing to Training of Deep Neural Networks”,
arXiv:1510.06356 (2015).
[639] M. H. Amin, E. Andriyash, J. Rolfe, B. Kulchytskyy, and R. Melko, “Quantum Boltzmann Machine”, Phys. Rev.
X 8, 021050 (2018).
[640] M. Benedetti, J. Realpe-Gómez, and A. Perdomo-Ortiz, “Quantum-Assisted Helmholtz Machines: A Quan-
tum–Classical Deep Learning Framework for Industrial Datasets in near-Term Devices”, Quantum Sci. Technol.
3, 034007 (2018).
[641] J. Caldeira, J. Job, S. H. Adachi, B. Nord, and G. N. Perdue, “Restricted Boltzmann Machines for Galaxy
Morphology Classification with a Quantum Annealer”, arXiv:1911.06259 (2019).
[642] M. Benedetti, J. Realpe-Gómez, R. Biswas, and A. Perdomo-Ortiz, “Estimation of Effective Temperatures in
Quantum Annealers for Sampling Applications: A Case Study with Possible Applications in Deep Learning”,
Phys. Rev. A 94, 022308 (2016).
[643] A. D. King et al., “Coherent Quantum Annealing in a Programmable 2,000 qubit Ising Chain”, Nat. Phys. 18,
1324 (2022).

M9.Metrics for boson sampling devices

M9.1. Hardware characterization and model as metrics

Hardware characterization of a boson sampling device consists in the development of a set of protocols that
enable extraction of sufficient information on the device, and that allow one to model and predict its behaviour
using the theory of quantum optics. Such simulations may not be scalable, but they allow predicting properties
of devices at moderate sizes.
Boson sampling machines are composed of sources, linear-interferometers and detectors, so that the char-
acterization of the device can be reduced to the characterization of its parts. As opposed to the challenging
characterization of qubit based devices such as superconducting circuits, which are prone to complex cross-talks
and other collective effects, characterizing boson sampling hardware is a simpler task. The reason is that the
dynamics of boson sampling devices are represented by the subset of Gaussian operations that can be efficiently
represented and characterized, as opposed to an Nq qubit device, where its full tomography scales exponentially
with the number of qubits Nq .
In the last decades, quantum optics experimental labs have been developing simple techniques to characterize
squeezed states sources, interferometers and detectors. For the squeezing sources the main parameter to extract
is the squeezing itself, the thermal noise, and potentially the distinguishability of the source [644, 645]. As many
experiments have a single source multiplexed over the multiple input of the linear interferometer, one often only
needs to characterize a unique source and the multiplexing interferometer.
Linear-interferometers can be characterized using classical coherent light, where one extracts the transmission
matrix [646] and two-point coherent correlations to obtain the phases [647–649]. Typical experiments need to

102
M9. Metrics for boson sampling devices

have a coherent source to stabilize the interferometer, so that the tomography of the linear-optics circuit can
be done using that resource.
The detectors are usually characterized using sources of single-photons or attenuated coherent states that
have very small average photon numbers. This allows one to extract the efficiency (false negative) and dark
counts (false positive) information [650].
While the independent characterization of the different elements of a boson sampling device is the common
approach in the field, there is no guarantee that, once merged into the final boson sampling device, all the
properties will be preserved, or that we may have access to all the components independently.
It is therefore preferable to have techniques that allow one to extract all information from the actual final
device conceived as a unique and non-divisible entity, specially when all the platform is integrated in a single
chip. This brings some additional challenges that can be overcome. For example, a large-size interferometer
tomography can also be done using single-photons sources, therefore one can easily characterize a boson sampling
on an integrated chip.

M9.2. Quantum advantage demonstration as metric

The Jiuzhang 1.0 and 2.0 demonstrations by UCTS [412, 454], and Xanadu’s implementation in 2022 [40], use
a set of similar validation tests for quantum advantage: (i) Bayesian test against adversarial distributions; (ii)
high output generation test; (iii) the comparison of truncated first to k-th order correlation functions. Each
technique has it own advantages and disadvantages. As we will discuss in more detail at the end of this section,
it is unlikely that an experimental test that fully certifies quantum advantage for sampling problems can be
found.
An important aspect for quantum advantage experiments based on random circuit or boson sampling devices
is the concept of ground truth, which corresponds to the distribution the experiment should be sampling from
if everything worked as it should, and also the one any spoofing strategy should try to emulate.
Our presentation below is an adaptation of the presentation in section 3 in Ref. [651], with some additional
information and discussions.

Bayesian Tests
Bayesian tests compare how good the ground truth is at explaining the observed data relative to other hypotheses
such as thermal states, coherent states, distinguishable squeezed states, and uniform probability distributions.
Here we provide a description of a number of such tests, which in future as the field progresses may become
separate metrics. Consider a set S = {s1 , . . . , sL } of L experimental samples, each of them containing Nc clicks.
A click here refers to the a photon detected in a detector. The probability of obtaining one of these samples,
given that it has Nc clicks, under the hypothesis HYP is given by

p(HYP) (sk )
p(HYP) (sk |Nc ) = , (M9.2.1)
p(HYP) (Nc )

where p(HYP) (sk ) is the probability of sample sk under hypothesis HYP, and p(HYP) (Nc ) is the grouped prob-
ability of obtaining Nc clicks in total, again under the hypothesis HYP. The probability of obtaining the set of
samples S under a given hypothesis HYP takes the form
L
Y
p(HYP) (S|Nc ) = p(HYP) (sk |Nc ). (M9.2.2)
k=1

We define the Bayesian ratio, rB (Nc ), which can be interpreted as the probability assigned to the ground
truth hypothesis GTH versus an alternative hypothesis ALT (thermal states, coherent states, ...) for a given
number of clicks, as

p(GTH) (S|Nc )
rB (Nc ) = . (M9.2.3)
p(GTH) (S|Nc ) + p(ALT) (S|Nc )

The Bayesian test consists in checking the convergence of rB (Nc ) when the number of samples is increased: if
rB (Nc ) → 1 for any Nc , we conclude that the ground truth hypothesis is more likely to describe the experimental
samples. Conversely, if rB (Nc ) → 0 for any Nc , the alternative hypothesis becomes more likely.

103
M9. Metrics for boson sampling devices

An alternative way to express this test is obtained by using entropies by writing

L L
1X   1X  
∆H(Nc ) = − ln p(GTH) (sk |Nc ) + ln p(ALT) (sk |Nc ) ,
L L (M9.2.4)
k=1 k=1
=H(GTH) (Nc ) − H(ALT) (Nc ).

The quantities H(GTH) and H(ALT) are estimators of the cross-entropy, for a given number of counts, of the
ground truth and alternative distribution relative to the real probability distribution of the experimental sam-
−1
ples. One can see that rB (Nc ) = [1 + exp(L∆H(Nc ))] and, for a increasing number of samples, the condition
rB (Nc ) → 1 is equivalent to ∆H(Nc ) < 0, while rB (Nc ) → 0 is equivalent to ∆H(Nc ) > 0. An important
step for the computation of the Bayesian test is the determination of the grouped click probability distributions
p(GTH) (Nc ), for which one can use the efficient method in Ref. [652].
It is important to make two remarks. The first is that Bayesian tests give the degree of confidence of one
hypothesis over another. Therefore, if the ground truth explains better the experimental data than a given
alternative, it does not discard the possibility that there could exist another distribution that can be classically
simulated efficiently, and that also describes the experimental data better. It therefore allows discarding known
suspects, but not unknown ones.
In addition, the Bayesian test relies on the computation of probabilities of individual samples, which for
the ground truth is a computationally hard task for patterns with a high number of clicks. Therefore, the test
cannot be used to verify that a large-size Gaussian boson sampling experiment achieves quantum computational
advantage. Rather, the Bayesian test is used to build up confidence in the correct functioning of the setup by
ruling out possible classical hypotheses explaining the samples.

Heavy Output Generation

The second test looks at how well the samples generated by the experiment have so-called heavy outputs,
corresponding to events with high probability, in the ideal distribution relative to samples generated by classically
efficient methods. One defines the heavy output generation (HOG) ratio, rHOG (Nc ), as

p(SQUE) (S|Nc )
rHOG (Nc ) = , (M9.2.5)
p(SQUE) (S|Nc ) + p(SQUE) (S ′ |Nc )

where S ′ = {s′1 , . . . , s′L } is a set of L samples obtained from the squashed states distribution (each one with Nc
clicks). p(GTH) (S|Nc ) is computed according to Eq. (M9.2.2). It is important to notice that now all the proba-
bilities are computed from the ground truth distribution, and the difference relies on the samples resulting from
the actual experimental data S and those generated from our hypothesis distribution S ′ . When rHOG (Nc ) → 1
we conclude that the experimental samples have higher ground truth probability, while rHOG (Nc ) → 0 when the
samples from the hypothesis distribution have higher ground truth probability. As in the case of the Bayesian
test, we have
L L
1X   1X
ln p(GTH) (s′k |Nc ) ,
 
∆E(Nc ) = − ln p(GTH) (sk |Nc ) +
L L (M9.2.6)
k=1 k=1
′
=E(GTH) (Nc ) − E(GTH) (Nc ).
−1
In terms of the cross-entropy difference ∆E(Nc ) [455], rHOG (Nc ) = [1 + exp(L∆E(Nc ))] and, for increasing L,
the conditions rHOG (Nc ) → 1 and rHOG (Nc ) → 0 are equivalent to ∆E(Nc ) < 0 and ∆E(Nc ) > 0, respectively.

Click cumulants
As we mention before, the Bayesian test relies on the computation of probabilities of individual samples, which
for the ground truth is a computationally hard task for patterns with a high number of clicks. Similar to
qubits-based verification of quantum advantage, any cross-entropy approach is not scalable. To circumvent this
problems a series of test that consist on certifying the correlations in the system have been proposed.
These correlation functions are given by cumulants, truncated correlation defined in terms of moments of a
multidimensional random variable V = (V1 , V2 , . . . , Vn ) as
* +
X Y Y
|π|−1
κ(V1 , . . . , Vn ) = (|π| − 1)!(−1) × Vi , (M9.2.7)
π B∈π i∈B

where π runs through the list of all partitions of {1, ..., n}, B runs through the list of all blocks of the partition

104
M9. Metrics for boson sampling devices

π, and |π| is the number of parts in the partition. Note that the first order cumulants are simply the means
κ(Vi ) = ⟨Vi |Vi ⟩, and that the second order cumulants are the covariances κ(Vi , Vj ) = ⟨Vi Vj |Vi Vj ⟩−⟨Vi |Vi ⟩ ⟨Vj |Vj ⟩.
It is straightforward to see that for binary outcomes, which have zero and non-zero photon number, moments
of the distribution correspond to marginal probabilities, given by

⟨Vi1 . . . Vin |Vi1 . . . Vin ⟩ = p (i1 = 1, . . . , in = 1) . (M9.2.8)

Most experiments compare cumulants up to a constant order (fourth to sixth in general), and compare how
well the correlations in the observed data match the correlations predicted by alternative hypothesis. This
approach suffers from the same weakness as the Bayesian method, where there could be yet to be discovered
hypothesis that can be efficiently simulated on classical devices.

Final discussion
As discussed above, verifying that we are in the hardness regime is itself not computationally tractable for
photonic devices, but it is important to note that the same holds for qubit-based experiments. Indeed, it
was shown in Ref. [453] that for random circuits under a scenario of verification based solely on classical
communication, two possibilities exist: (i) sampling is hard to simulate for classical computers, but verification
is not scalable; (ii) an efficient verification protocol exists, but there also exists a sampling strategy that exploits
the verification tests to efficiently simulate the sampling from the circuit. This result may hold true also for
boson sampling devices.
It is reasonable to conclude that focusing on problems where the solutions can be classically verified, or
provide a quantifiable advantage for an end-user application, may be the only way to provide evidence for
quantum advantage for boson sampling devices.

Metric references
[40] L. S. Madsen et al., “Quantum Computational Advantage with a Programmable Photonic Processor”, Nature
606, 75 (2022).
[412] H.-S. Zhong et al., “Phase-Programmable Gaussian Boson Sampling Using Stimulated Squeezed Light”, Phys.
Rev. Lett. 127, 180502 (2021).
[453] D. Stilck França and R. Garcia-Patron, “A Game of Quantum Advantage: Linking Verification and Simulation”,
Quantum 6, 753 (2022).
[454] H.-S. Zhong et al., “Quantum Computational Advantage Using Photons”, Science 370, 1460 (2020).
[455] B. Villalonga, M. Y. Niu, L. Li, H. Neven, J. C. Platt, V. N. Smelyanskiy, and S. Boixo, “Efficient approximation
of experimental Gaussian boson sampling”, arXiv:2109.11525 (2022).
[644] G. Frascella, S. Agne, F. Y. Khalili, and M. V. Chekhova, “Overcoming detection loss and noise in squeezing-based
optical sensing”, npj Quantum Inf. 7, 72 (2021).
[645] T. Park, H. Stokowski, V. Ansari, S. Gyger, K. K. Multani, O. T. Celik, A. Y. Hwang, D. J. Dean, F. Mayor, T. P.
McKenna, et al., “Single-mode squeezed-light generation and tomography with an integrated optical parametric
oscillator”, Sci. Adv. 10, eadl1814 (2024).
[646] S. Popoff, G. Lerosey, R. Carminati, M. Fink, A. Boccara, and S. Gigan, “Measuring and exploiting the trans-
mission matrix in optics”, Quantum Electron. Laser Sci. Conf., QME6 (2010).
[647] M. Lobino, D. Korystov, C. Kupchak, E. Figueroa, B. C. Sanders, and A. Lvovsky, “Complete characterization
of quantum-optical processes”, Science 322, 563 (2008).
[648] S. Rahimi-Keshari, M. A. Broome, R. Fickler, A. Fedrizzi, T. C. Ralph, and A. G. White, “Direct characterization
of linear-optical networks”, Opt. Express 21, 13450 (2013).
[649] I. Dhand, A. Khalid, H. Lu, and B. C. Sanders, “Accurate and precise characterization of linear optical interfer-
ometers”, J. Opt. 18, 035204 (2016).
[650] N. Maring, A. Fyrillas, M. Pont, E. Ivanov, P. Stepanov, N. Margaria, W. Hease, A. Pishchagin, A. Lemaître,
I. Sagnes, et al., “A versatile single-photon-based quantum computing platform”, Nat. Photonics 18, 603 (2024).
[651] J. Martínez-Cifuentes, K. M. Fonseca-Romero, and N. Quesada, “Classical Models May Be a Better Explanation
of the Jiuzhang 1.0 Gaussian Boson Sampler than Its Targeted Squeezed Light Model”, Quantum 7, 1076 (2023).
[652] P. D. Drummond, B. Opanchuk, A. Dellios, and M. D. Reid, “Simulating Complex Networks in Phase Space:
Gaussian Boson Sampling”, Phys. Rev. A 105, 012427 (2022).

105
M10. Metrics for neutral atoms devices

M10.Metrics for neutral atoms devices

M10.1. Analogue process fidelity

The analogue process fidelity compares and verifies measured state and process fidelities against a classical
model to enable learning of Hamiltonian control parameters and error channels whilst using finite sampling
[382].

Description
As discussed in Sec. 5.3, neutral atom platforms offer large numbers of qubits and programmable spin models
suitable for analogue quantum computation targeting classical optimization problems as well as efficient quan-
tum simulation. In the paradigm of analog optimization, it is important to evaluate the performance of the
hardware accounting both for errors and ensuring the Hamiltonian parameters encoded by the user match those
implemented on the hardware to avoid encoding the wrong classical optimization problem. Whilst there exist
a wide range of metrics for benchmarking digital processors with arbitrary unitary operations, analog based
processors typically offer a limited subset of control parameters such as global driving parameters, local site-
dependent detunings and programmable spin-models using the specific geometric configuration of the atomic
tweezers. To overcome this challenge, in Ref. [382] a benchmarking protocol is proposed that enables a system
to be prepared in an initial state ρ(t), evolved for a time t under the target Hamiltonian H to induce quench
dynamics, and then samples M measurements of the output bit-strings measured in the {|z⟩} basis. This results
in the empirical distribution q(z, t) describing the distribution of strings z measured at time t. This is com-
pared against a theoretical target distribution p(z, t) calculated using classical simulation, and the infinite-time
RT
average pavg (z) = limT →∞ T1 0 p(z, t) dt. These are combined P to obtain the rescaled outcome probabilities
p̃(z, t) = p(z, t)/pavg (z) and the normalization factor Z(t) = z pavg (z)p̃(z, t)2 . From these parameters one can
evaluate
M
2 hX i
F̂d (t) = p̃(zi , t) /Z(t) − 1 ≈ Fd (t), (M10.1.1)
M i=1

which approximates the fidelity F (t) = ⟨Ψ(t)| ρ(t) |Ψ(t)⟩, where ρ(t) represents the state of the experimental
system and |Ψ(t)⟩ the calculated ideal output state expected from the evolution of |Ψ(t = 0)⟩ = |Ψ0 ⟩ under
the target Hamiltonian H. Analysis of statistical errors shows that using this approach percent level precision
is possible using finite samples sizes M ∼ 103 . This works under the same approach as digital benchmark
methods based on deep random unitary circuits, as it can be shown that the rescaled probabilities p̃(z) follow
a Porter-Thomas (PT) distribution with unity mean [382].
This procedure can be used to learn Hamiltonian components by scaling parameters in the computational
model and finding the values which maximize the observed output state fidelity as demonstrated experimentally
for extracting randomised light-shifts and global Rabi frequency on a chain of 15 atoms [653]. By evaluating the
evolution of Fd (t) with t using different error models it is also possible to calibrate noise parameters [654]. This
can also be used to facilitate comparison of analog processors to what would be possible using digital hardware.

Measurement procedure
This procedure follows the approach outlined in Ref. [382], and is as follows:

1. Experimentally prepare an initial state ρ0 that approximates a pure state |Ψ0 ⟩ ⟨Ψ0 |.
2. Evolve the system under its natural Hamiltonian H for a time t.
3. Measure the evolved state ρ(t) in a natural basis, obtaining configurations {zj}M
j=1 .

4. Classically compute
2 2
• p(z, t) ≡ |⟨z|Ψ(t)⟩| = |⟨z| exp(−iHt) |Ψ0 ⟩|
RT
• pavg (z) ≡ limT →∞ T1 0 p(z, t) dt
• p̃(z, t) ≡ p(z, t)/pavg (z)
• Z(t) ≡ z pavg (z)p̃(z, t)2
P
hP i
2 M
5. Evaluate F̂d (t) ≡ M i=1 p̃(zi , t) /Z(t)−1 ≃ Fd (t) which approximates the fidelity F (t) = ⟨Ψ(t)| ρ(t) |Ψ(t)⟩.

106
M10. Metrics for neutral atoms devices

Assumptions and limitations

• The assumption in this work is that the eigenvalues {Ei } of H possess no resonant structures. This
condition is expected to hold for generic ergodic Hamiltonians.
• Whilst this method is limited by the requirement to benchmark against classical calculations, this can be
extended to extrapolate results of approximate classical simulations to provide a global fidelity estimator
for an analog simulator operating outside of the classically simulable regime as demonstrated with arrays
of up to N = 60 entangled atoms [654].
• While this protocol is applicable to generic programmable Hamiltonians, it may fail in special cases where
are long times the system is not described by a Porter-Thomas distribution. This occurs in instances with
weakly- or non-ergodic dynamics, or the presence of correlated non-local errors. See [382] for details.

Metric references
[382] D. K. Mark, J. Choi, A. L. Shaw, M. Endres, and S. Choi, “Benchmarking quantum simulators using ergodic
quantum dynamics”, Phys. Rev. Lett. 131, 110601 (2023).
[653] J. Choi et al., “Preparing random states and benchmarking with many-body quantum chaos”, Nature 613, 468
(2023).
[654] A. L. Shaw, Z. Chen, J. Choi, D. K. Mark, P. Scholl, R. Finkelstein, A. Elben, S. Choi, and M. Endres, “Bench-
marking highly entangled states on a 60-atom analogue quantum simulator”, Nature 628, 71 (2024).

M10.2. Trap lifetime

For atoms trapped in optical tweezers the characteristic 1/e trap lifetime, τ , is relevant, since for N atoms the
effective array lifetime is τ /N , limiting the time useful computation can be performed.

Description
Due to the relatively shallow trap depth of neutral atom tweezers it is possible for atoms to be ejected from the
traps during computation due to collisions with residual background gases [21, 55, 383, 560, 655]. For readout
methods dependent on ejecting atoms from the trap prior to imaging, this leads to increased state preparation
and measurement (SPAM) errors [21], whilst also introducing errors during computation due to stochastic loss
of atoms from the array. For room temperature systems typically τ ∼ 10−20 s, whilst at cryogenic temperatures
lifetimes exceeding 10 minutes have been demonstrated [383].

Measurement procedure
Measurement of trap lifetime is performed as follows:

1. Load atom array with N atoms and determine which sites are loaded initially
2. Hold atoms in the trap for variable hold times t
3. Read out the traps to measure the number of remaining atoms
4. Repeat steps 1-3 for a certain number of shots to calculate the survival probability p(t) = Nremaining (t)/N

5. Fit the survival probability vs t using p(t) = p0 e−t/τ to extract the characteristic trap lifetime τ

Assumptions and limitations

• It is assumed that the losses are due to single-body collisions, as expected for single-atom loading.
• The metric does not take into account the case where controlled dynamics are applied on the system.

107
M10. Metrics for neutral atoms devices

Metric references
[21] K. Wintersperger, F. Dommert, T. Ehmer, A. Hoursanov, J. Klepsch, W. Mauerer, G. Reuber, T. Strohm, M.
Yin, and S. Luber, “Neutral atom quantum computing hardware: performance and end-user perspective”, EPJ
Quantum Technol. 10, 32 (2023) (Cited on pages 3, 10, 107, 108).
[55] S. Ebadi, T. T. Wang, H. Levine, A. Keesling, G. Semeghini, A. Omran, D. Bluvstein, R. Samajdar, H. Pichler,
W. W. Ho, et al., “Quantum phases of matter on a 256-atom programmable quantum simulator”, Nature 595,
227 (2021) (Cited on pages 3, 107).
[383] K.-N. Schymik, S. Pancaldi, F. Nogrette, D. Barredo, J. Paris, A. Browaeys, and T. Lahaye, “Single Atoms with
6000-Second Trapping Lifetimes in Optical-Tweezer Arrays at Cryogenic Temperatures”, Phys. Rev. Appl. 16,
034013 (2021) (Cited on pages 17, 31, 107).
[560] K.-N. Schymik, V. Lienhard, D. Barredo, P. Scholl, H. Williams, A. Browaeys, and T. Lahaye, “Enhanced atom-
by-atom assembly of arbitrary tweezer arrays”, Phys. Rev. A 102, 063107 (2020) (Cited on pages 47, 107).
[655] T. M. Graham et al., “Multi-qubit entanglement and algorithms on a neutral-atom quantum computer”, Nature
604, 457 (2022) (Cited on page 107).

M10.3. Reconfigurable connectivity

For most quantum computing architectures the connectivity is static, and given by the hardware setup. This is
captured by the pairwise connectivity hardware architectural property (See M1.2). For neutral atom systems
with mobile tweezers it is possible to dynamically reconfigure the array geometry during computation to im-
plement arbitrary couplings [19, 21, 23]. The reconfigurable connectivity is a hardware architectural property
that states whether such a capability is available in a system, and if so to what extent.

Description
Using atoms in mobile tweezer traps enables atoms to be transported whilst preserving coherence, allowing
arbitrary qubit connectivity to be implemented, as demonstrated in Ref. [19] for realizing topological coupling
graphs.

Measurement procedure
The degree of reconfigurability is limited by the available number of mobile tweezers and hardware restrictions
on allowed moves, with typical implementations favoring use of parallel column or row operations. Another
approach to measure re-configurable connectivity is by considering reduction in required number of two qubit
gate operations compared to a static qubit connectivity when using an optimized compiler [656].

Assumptions and limitations

• Utility of reconfigurable geometries may be limited without low-level access to specify arbitrary moves.

Metric references
[19] D. Bluvstein et al., “A Quantum Processor Based on Coherent Transport of Entangled Atom Arrays”, Nature
604, 451 (2022).
[21] K. Wintersperger, F. Dommert, T. Ehmer, A. Hoursanov, J. Klepsch, W. Mauerer, G. Reuber, T. Strohm, M.
Yin, and S. Luber, “Neutral atom quantum computing hardware: performance and end-user perspective”, EPJ
Quantum Technol. 10, 32 (2023).
[23] D. Bluvstein, S. J. Evered, A. A. Geim, S. H. Li, H. Zhou, T. Manovitz, S. Ebadi, M. Cain, M. Kalinowski, D.
Hangleiter, et al., “Logical quantum processor based on reconfigurable atom arrays”, Nature 626, 58 (2024).
[656] D. B. Tan, D. Bluvstein, M. D. Lukin, and J. Cong, “Compiling Quantum Circuits for Dynamically Field-
Programmable Neutral Atoms Array Processors”, Quantum 8, 1281 (2024).

108
References

V.Conclusions
In this article a collection of metrics is provided, with a consistent format for each metric that includes its
definition, a description of the methodology, the main assumptions and limitations, and a linked open source
software implementing the methodology. The open source software package transparently demonstrates the
methodology and can also be used in practical evaluations of the value of metrics. As shown in this article,
the level of maturity of the different metrics varies significantly. For example, metrics such as randomized
benchmarking based fidelities are well established, while approaches to benchmark full applications still allow
for large variations based on the many choices of algorithms, parameters, and success criteria used to measure
device performance. Research on metrics and benchmarks is actively progressing, and is particularly important
for the improvement of the efficiency and scalability of metrics across all categories to benchmark quantum
computers with well over a thousand qubits, for metrics on device speed and stability, as well as in benchmarks
for non-circuit based approaches.
One existing problem is that, even to evaluate the well established metrics, many different approaches with
slightly diverging technical details are used in practice. While all approaches provide similar overall qualitative
performance values, these diverging details in the technical implementations make quantitative comparisons
difficult. One of the resulting risks is that it makes it difficult to objectively determine the progress of the
quantum computing field towards quantum advantage. Combined with the fact that there are many largely
different hardware platforms and quantum computing approaches, this makes it important to reduce the metrics
used to evaluate the performance for a given hardware platform to a minimal but representative set agreed
across manufacturers, algorithm developers and end users. At a minimum, to fairly and objectively benchmark
quantum computers across manufacturers, these benchmarks need to follow some agreed common approaches.
As outlined in the Discussion and Outlook section (Sec. 3), the need for such agreement has led to international
standardization activities on metrics and benchmarking, and in that section a number of work items to be
considered for standardization are proposed. A collection of metrics, such as the one presented in this article,
together with the associated open-source software needed to evaluate them, can contribute to the development
of standardized benchmarks for quantum computers and speed up the progress of the field towards practical
quantum advantage.

Acknowledgements
This project was funded and supported by the UK National Quantum Computer Centre [NQCC200921], which
is a UKRI Centre and part of the UK National Quantum Technologies Programme (NQTP). JP acknowl-
edges support from EPSRC (Grant No. EP/T005386/1). AR acknowledges support from UKRI (Grant No.
MR/T041110/1). We thank Yannic Rath, Daniel Rodrigo Albert, Wang Wong, Nikolaos Schoinas, Masum
Uddin, Manognya Acharya, David Headley, Shushmi Chowdhury, and Manav Babel for useful discussions and
for reviewing manuscript and software. We thank the Collaborative Computational Project Quantum Com-
puting (ccp-qc, EPSRC Grant No. EP/T026715/2), the EPSRC Quantum Computing and Simulation Hub
(EP/T001062/1), the UKRI ExCALIBUR project QEVEC (EP/W00772X/2), and the Superconducting Quan-
tum Materials and Systems Center (SQMS, DE-AC02-07CH11359), for facilitating collaboration and discussion.

References
[1] M. Kjaergaard, M. E. Schwartz, J. Braumüller, P. Krantz, J. I.-J. Wang, S. Gustavsson, and W. D. Oliver, “Superconducting qubits:
Current state of play”, Annu. Rev. Condens. Matter Phys. 11, 369 (2020) (Cited on pages 3, 10, 29).
[2] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A Quantum Engineer’s Guide to Supercon-
ducting Qubits”, Appl. Phys. Rev. 6, 021318 (2019) (Cited on pages 3, 9, 21, 29, 37, 49, 52–54, 89, 92).
[3] I. Siddiqi, “Engineering high-coherence superconducting qubits”, Nat. Rev. Mater. 6, 875 (2021) (Cited on pages 3, 9).
[4] M. H. Devoret, A. Wallraff, and J. M. Martinis, “Superconducting qubits: a short review”, arXiv:cond-mat/0411174 (2004) (Cited
on page 3).
[5] G. Wendin, “Quantum Information Processing with Superconducting Circuits: A Review”, Rep. Prog. Phys. 80, 106001 (2017)
(Cited on pages 3, 29).
[6] J. Clarke and F. K. Wilhelm, “Superconducting quantum bits”, Nature 453, 1031 (2008) (Cited on page 3).
[7] M. H. Devoret and R. J. Schoelkopf, “Superconducting circuits for quantum information: an outlook”, Science 339, 1169 (2013)
(Cited on page 3).
[8] J. You and F. Nori, “Superconducting circuits and quantum information”, Phys. Today 58, 42 (2005) (Cited on page 3).
[9] C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, “Trapped-ion quantum computing: Progress and challenges”, Appl.
Phys. Rev. 6, 021314 (2019) (Cited on pages 3, 8–10, 30, 49).
[10] H. Häffner, C. F. Roos, and R. Blatt, “Quantum computing with trapped ions”, Phys. Rep. 469, 155 (2008) (Cited on page 3).
[11] Z. D. Romaszko, S. Hong, M. Siegele, R. K. Puddy, F. R. Lebrun-Gallagher, S. Weidt, and W. K. Hensinger, “Engineering of
microfabricated ion traps and integration of advanced on-chip features”, Nat. Rev. Phys. 2, 285 (2020) (Cited on page 3).
[12] J. I. Cirac and P. Zoller, “Quantum Computations with Cold Trapped Ions”, Phys. Rev. Lett. 74, 4091 (1995) (Cited on pages 3,
30).

109
References

[13] J. Benhelm, G. Kirchmair, C. F. Roos, and R. Blatt, “Towards fault-tolerant quantum computing with trapped ions”, Nat. Phys.
4, 463 (2008) (Cited on page 3).
[14] C. Monroe and J. Kim, “Scaling the ion trap quantum processor”, Science 339, 1164 (2013) (Cited on page 3).
[15] S. Debnath, N. M. Linke, C. Figgatt, K. A. Landsman, K. Wright, and C. Monroe, “Demonstration of a small programmable
quantum computer with atomic qubits”, Nature 536, 63 (2016) (Cited on pages 3, 7).
[16] K. Wright et al., “Benchmarking an 11-Qubit Quantum Computer”, Nat. Commun. 10, 5464 (2019) (Cited on pages 3, 4, 7, 8, 13).
[17] M. Saffman, “Quantum computing with neutral atoms”, Natl. Sci. Rev. 6, 24 (2019) (Cited on pages 3, 10).
[18] L. Henriet, L. Beguin, A. Signoles, T. Lahaye, A. Browaeys, G.-O. Reymond, and C. Jurczak, “Quantum computing with neutral
atoms”, Quantum 4, 327 (2020) (Cited on page 3).
[19] D. Bluvstein et al., “A Quantum Processor Based on Coherent Transport of Entangled Atom Arrays”, Nature 604, 451 (2022)
(Cited on pages 3, 8, 17, 31, 108).
[20] J. Wurtz, A. Bylinskii, B. Braverman, J. Amato-Grill, S. H. Cantu, F. Huber, A. Lukin, F. Liu, P. Weinberg, J. Long, et al.,
“Aquila: QuEra’s 256-qubit neutral-atom quantum computer”, arXiv:2306.11727 (2023) (Cited on pages 3, 7, 47).
[21] K. Wintersperger, F. Dommert, T. Ehmer, A. Hoursanov, J. Klepsch, W. Mauerer, G. Reuber, T. Strohm, M. Yin, and S. Luber,
“Neutral atom quantum computing hardware: performance and end-user perspective”, EPJ Quantum Technol. 10, 32 (2023) (Cited
on pages 3, 10, 107, 108).
[22] T. M. Graham et al., “Multi-Qubit Entanglement and Algorithms on a Neutral-Atom Quantum Computer”, Nature 604, 457 (2022)
(Cited on pages 3, 9, 31).
[23] D. Bluvstein, S. J. Evered, A. A. Geim, S. H. Li, H. Zhou, T. Manovitz, S. Ebadi, M. Cain, M. Kalinowski, D. Hangleiter, et al.,
“Logical quantum processor based on reconfigurable atom arrays”, Nature 626, 58 (2024) (Cited on pages 3, 5, 8, 9, 19, 108).
[24] J. L. O’brien, A. Furusawa, and J. Vučković, “Photonic quantum technologies”, Nat. Photonics 3, 687 (2009) (Cited on page 3).
[25] P. Kok, W. J. Munro, K. Nemoto, T. C. Ralph, J. P. Dowling, and G. J. Milburn, “Linear optical quantum computing with photonic
qubits”, Rev. Mod. Phys. 79, 135 (2007) (Cited on page 3).
[26] J. Wang, F. Sciarrino, A. Laing, and M. G. Thompson, “Integrated photonic quantum technologies”, Nat. Photonics 14, 273 (2020)
(Cited on page 3).
[27] S. Barz, “Quantum computing with photons: introduction to the circuit model, the one-way quantum computer, and the fundamental
principles of photonic experiments”, J. Phys. B: At. Mol. Opt. Phys. 48, 083001 (2015) (Cited on page 3).
[28] F. Flamini, N. Spagnolo, and F. Sciarrino, “Photonic quantum information processing: a review”, Rep. Prog. Phys. 82, 016001
(2018) (Cited on page 3).
[29] S. Slussarenko and G. J. Pryde, “Photonic quantum information processing: a concise review”, Appl. Phys. Rev. 6 (2019) (Cited
on page 3).
[30] E. Knill, R. Laflamme, and G. J. Milburn, “A scheme for efficient quantum computation with linear optics”, Nature 409, 46 (2001)
(Cited on page 3).
[31] G. Burkard, T. D. Ladd, A. Pan, J. M. Nichol, and J. R. Petta, “Semiconductor spin qubits”, Rev. Mod. Phys. 95, 025003 (2023)
(Cited on pages 3, 33).
[32] A. Chatterjee, P. Stevenson, S. De Franceschi, A. Morello, N. P. de Leon, and F. Kuemmeth, “Semiconductor qubits in practice”,
Nat. Rev. Phys. 3, 157 (2021) (Cited on pages 3, 33).
[33] M. D. Michielis, E. Ferraro, E. Prati, L. Hutin, B. Bertrand, E. Charbon, D. J. Ibberson, and M. F. Gonzalez-Zalba, “Silicon spin
qubits from laboratory to industry”, J. Phys. D: Appl. Phys. 56, 363001 (2023) (Cited on pages 3, 33).
[34] R. Maurand, X. Jehl, D. Kotekar-Patil, A. Corna, H. Bohuslavskyi, R. Laviéville, L. Hutin, S. Barraud, M. Vinet, M. Sanquer,
et al., “A CMOS silicon spin qubit”, Nat. Commun. 7, 13575 (2016) (Cited on pages 3, 33).
[35] D. D. Awschalom, L. C. Bassett, A. S. Dzurak, E. L. Hu, and J. R. Petta, “Quantum spintronics: engineering and manipulating
atom-like spins in semiconductors”, Science 339, 1174 (2013) (Cited on page 3).
[36] L. M. Vandersypen and M. A. Eriksson, “Quantum computing with semiconductor spins”, Phys. Today 72, 38 (2019) (Cited on
page 3).
[37] C. Kloeffel and D. Loss, “Prospects for spin-based quantum computing in quantum dots”, Annu. Rev. Condens. Matter Phys. 4,
51 (2013) (Cited on page 3).
[38] Y. Kim, A. Eddins, S. Anand, K. X. Wei, E. Van Den Berg, S. Rosenblatt, H. Nayfeh, Y. Wu, M. Zaletel, K. Temme, et al.,
“Evidence for the utility of quantum computing before fault tolerance”, Nature 618, 500 (2023) (Cited on pages 3, 8, 9).
[39] S. Patra, S. S. Jahromi, S. Singh, and R. Orús, “Efficient tensor network simulation of IBM’s largest quantum processors”, Phys.
Rev. Res. 6, 013326 (2024) (Cited on page 3).
[40] L. S. Madsen et al., “Quantum Computational Advantage with a Programmable Photonic Processor”, Nature 606, 75 (2022) (Cited
on pages 3, 4, 27, 32, 103).
[41] M. Motta and J. E. Rice, “Emerging quantum computing algorithms for quantum chemistry”, Wiley Interdiscip. Rev. Comput.
Mol. Sci. 12, e1580 (2022) (Cited on page 3).
[42] Google Quantum AI and Collaborators, “Hartree-Fock on a superconducting qubit quantum computer”, Science 369, 1084 (2020)
(Cited on pages 3, 7, 13).
[43] A. M. Dalzell, S. McArdle, M. Berta, P. Bienias, C.-F. Chen, A. Gilyén, C. T. Hann, M. J. Kastoryano, E. T. Khabiboulline, A.
Kubica, et al., “Quantum algorithms: a survey of applications and end-to-end complexities”, arXiv:2310.03011 (2023) (Cited on
page 3).
[44] A. M. Childs, D. Maslov, Y. Nam, N. J. Ross, and Y. Su, “Toward the first quantum simulation with quantum speedup”, Proc.
Natl. Acad. Sci. 115, 9456 (2018) (Cited on page 3).
[45] M. W. Johnson et al., “Quantum Annealing with Manufactured Spins”, Nature 473, 194 (2011) (Cited on pages 3, 16).
[46] P. Hauke, H. G. Katzgraber, W. Lechner, H. Nishimori, and W. D. Oliver, “Perspectives of quantum annealing: methods and
implementations”, Rep. Prog. Phys. 83, 054401 (2020) (Cited on page 3).
[47] A. B. Finilla, M. A. Gomez, C. Sebenik, and J. D. Doll, “Quantum Annealing: A New Method for Minimizing Multidimensional f
Unctions”, Chem. Phys. Lett. 219, 343 (1994) (Cited on pages 3, 24).
[48] C. S. Hamilton, R. Kruse, L. Sansoni, S. Barkhofen, C. Silberhorn, and I. Jex, “Gaussian Boson Sampling”, Phys. Rev. Lett. 119,
170501 (2017) (Cited on pages 3, 26).
[49] S. Aaronson and A. Arkhipov, “The Computational Complexity of Linear Optics”, Theory Comput. 9, 143 (2013) (Cited on pages 3,
25).
[50] A. P. Lund, A. Laing, S. Rahimi-Keshari, T. Rudolph, J. L. O’Brien, and T. C. Ralph, “Boson Sampling from a Gaussian State”,
Phys. Rev. Lett. 113, 100502 (2014) (Cited on pages 3, 26).
[51] B. T. Gard, K. R. Motes, J. P. Olson, P. P. Rohde, and J. P. Dowling, “An Introduction to Boson-Sampling”, At. Mesoscale, 167
(2015) (Cited on pages 3, 26, 27).
[52] A. J. Daley, I. Bloch, C. Kokail, S. Flannigan, N. Pearson, M. Troyer, and P. Zoller, “Practical quantum advantage in quantum
simulation”, Nature 607, 667 (2022) (Cited on pages 3, 4).
[53] I. M. Georgescu, S. Ashhab, and F. Nori, “Quantum simulation”, Rev. Mod. Phys. 86, 153 (2014) (Cited on page 3).
[54] I. Buluta and F. Nori, “Quantum simulators”, Science 326, 108 (2009) (Cited on page 3).
[55] S. Ebadi, T. T. Wang, H. Levine, A. Keesling, G. Semeghini, A. Omran, D. Bluvstein, R. Samajdar, H. Pichler, W. W. Ho, et al.,
“Quantum phases of matter on a 256-atom programmable quantum simulator”, Nature 595, 227 (2021) (Cited on pages 3, 107).
[56] F. Arute et al., “Quantum Supremacy Using a Programmable Superconducting Processor”, Nature 574, 505 (2019) (Cited on
pages 4, 7, 8, 12, 22, 27, 89).
[57] Y. Wu et al., “Strong quantum computational advantage using a superconducting quantum processor”, Phys. Rev. Lett. 127,
180501 (2021) (Cited on pages 4, 22).
[58] Q. Zhu, S. Cao, F. Chen, M.-C. Chen, X. Chen, T.-H. Chung, H. Deng, Y. Du, D. Fan, M. Gong, et al., “Quantum computational
advantage via 60-qubit 24-cycle random circuit sampling”, Sci. Bull. 67, 240 (2022) (Cited on pages 4, 22).

110
References

[59] Y.-H. Deng et al., “Gaussian Boson Sampling with Pseudo-Photon-Number Resolving Detectors and Quantum Computational
Advantage”, Phys. Rev. Lett. 131, 150601 (2023) (Cited on pages 4, 27).
[60] T. Proctor, K. Young, A. D. Baczewski, and R. Blume-Kohout, “Benchmarking quantum computers”, arXiv:2407.08828 (2024)
(Cited on pages 4, 5).
[61] N. Herrmann, D. Arya, M. W. Doherty, A. Mingare, J. C. Pillay, F. Preis, and S. Prestel, “Quantum utility–definition and
assessment of a practical quantum advantage”, 2023 IEEE Int. Conf. Quantum Softw. QSW, 162 (2023) (Cited on page 4).
[62] D. P. DiVincenzo, “Two-bit gates are universal for quantum computation”, Phys. Rev. A 51, 1015 (1995) (Cited on page 4).
[63] D. P. DiVincenzo, “Quantum computation”, Science 270, 255 (1995) (Cited on page 4).
[64] C. Monroe, D. M. Meekhof, B. E. King, W. M. Itano, and D. J. Wineland, “Demonstration of a fundamental quantum logic gate”,
Phys. Rev. Lett. 75, 4714 (1995) (Cited on page 4).
[65] M. G. Raymer, M. Beck, and D. McAlister, “Complex wave-field reconstruction using phase-space tomography”, Phys. Rev. Lett.
72, 1137 (1994) (Cited on page 4).
[66] J. F. Poyatos, J. I. Cirac, and P. Zoller, “Complete Characterization of a Quantum Process: The Two-Bit Quantum Gate”, Phys.
Rev. Lett. 78, 390 (1997) (Cited on pages 4, 10).
[67] I. L. Chuang and M. A. Nielsen, “Prescription for experimental determination of the dynamics of a quantum black box”, J. Mod.
Opt. 44, 2455 (1997) (Cited on pages 4, 10, 40).
[68] M. A. Nielsen, E. Knill, and R. Laflamme, “Complete quantum teleportation using nuclear magnetic resonance”, Nature 396, 52
(1998) (Cited on page 4).
[69] R. Laflamme, E. Knill, W. H. Zurek, P. Catasti, and S. Mariappan, “Nmr ghz”, NASA Int. Conf. Quantum Comput. Quantum
Commun., 357 (1998) (Cited on page 4).
[70] Z. Hradil, “Quantum-state estimation”, Phys. Rev. A 55, R1561 (1997) (Cited on pages 4, 10).
[71] B. Schumacher, “Sending Entanglement through Noisy Quantum Channels”, Phys. Rev. A 54, 2614 (1996) (Cited on pages 4, 10,
57).
[72] R. Jozsa, “Fidelity for mixed quantum states”, J. Mod. Opt. 41, 2315 (1994) (Cited on pages 4, 57).
[73] M. A. Nielsen, “The entanglement fidelity and quantum error correction”, arXiv:quant-ph/9606012 (1996) (Cited on page 4).
[74] D. Aharonov, A. Kitaev, and N. Nisan, “Quantum Circuits with Mixed States”, Proc. Thirtieth Annu. ACM Symp. Theory Comput.
Proc. Thirtieth Annu. ACM Symp. Theory Comput. (1998), 20 (Cited on pages 4, 10, 58).
[75] J. A. Jones and M. Mosca, “Implementation of a quantum algorithm on a nuclear magnetic resonance quantum computer”, J.
Chem. Phys 109, 1648 (1998) (Cited on page 4).
[76] I. L. Chuang, L. M. Vandersypen, X. Zhou, D. W. Leung, and S. Lloyd, “Experimental realization of a quantum algorithm”, Nature
393, 143 (1998) (Cited on page 4).
[77] M. A. Nielsen, “A Simple Formula for the Average Gate Fidelity of a Quantum Dynamical Operation”, Phys. Lett. A 303, 249
(2002) (Cited on pages 4, 10, 58).
[78] M. Horodecki, P. Horodecki, and R. Horodecki, “General teleportation channel, singlet fraction, and quasidistillation”, Phys. Rev.
A 60, 1888 (1999) (Cited on pages 4, 10, 57).
[79] E. M. Fortunato, L. Viola, J. Hodges, G. Teklemariam, and D. G. Cory, “Implementation of universal control on a decoherence-free
qubit”, New J. Phys. 4, 5 (2002) (Cited on pages 4, 10).
[80] A. Gilchrist, N. K. Langford, and M. A. Nielsen, “Distance measures to compare real and ideal quantum processes”, Phys. Rev. A
71, 062310 (2005) (Cited on pages 4, 10, 57).
[81] J. Emerson, R. Alicki, and K. Życzkowski, “Scalable Noise Estimation with Random Unitary Operators”, J. Opt. B Quantum
Semiclassical Opt. 7, S347 (2005) (Cited on pages 4, 11, 60).
[82] E. Knill, D. Leibfried, R. Reichle, J. Britton, R. B. Blakestad, J. D. Jost, C. Langer, R. Ozeri, S. Seidelin, and D. J. Wineland,
“Randomized Benchmarking of Quantum Gates”, Phys. Rev. A 77, 012307 (2008) (Cited on pages 4, 11, 60, 61).
[83] J. Emerson, M. Silva, O. Moussa, C. Ryan, M. Laforest, J. Baugh, D. G. Cory, and R. Laflamme, “Symmetrized characterization
of noisy quantum processes”, Science 317, 1893 (2007) (Cited on pages 4, 11).
[84] B. Lévi, C. C. López, J. Emerson, and D. G. Cory, “Efficient Error Characterization in Quantum Information Processing”, Phys.
Rev. A 75, 022314 (2007) (Cited on page 4).
[85] C. Dankert, R. Cleve, J. Emerson, and E. Livine, “Exact and Approximate Unitary 2-Designs and Their Application to Fidelity
Estimation”, Phys. Rev. A 80, 012304 (2009) (Cited on page 4).
[86] E. Magesan, J. M. Gambetta, and J. Emerson, “Characterizing Quantum Gates via Randomized Benchmarking”, Phys. Rev. A 85,
042311 (2012) (Cited on pages 4, 11).
[87] T. J. Proctor, A. Carignan-Dugas, K. Rudinger, E. Nielsen, R. Blume-Kohout, and K. Young, “Direct Randomized Benchmarking
for Multiqubit Devices”, Phys. Rev. Lett. 123, 030503 (2019) (Cited on pages 4, 11).
[88] J. M. Gambetta et al., “Characterization of Addressability by Simultaneous Randomized Benchmarking”, Phys. Rev. Lett. 109,
240504 (2012) (Cited on pages 4, 11, 61).
[89] J. M. Epstein, A. W. Cross, E. Magesan, and J. M. Gambetta, “Investigating the limits of randomized benchmarking protocols”,
Phys. Rev. A 89, 062321 (2014) (Cited on pages 4, 62, 63).
[90] J. P. Gaebler et al., “Randomized Benchmarking of Multiqubit Gates”, Phys. Rev. Lett. 108, 260503 (2012) (Cited on pages 4, 11,
63).
[91] E. Magesan et al., “Efficient Measurement of Quantum Gate Error by Interleaved Randomized Benchmarking”, Phys. Rev. Lett.
109, 080505 (2012) (Cited on pages 4, 11, 63).
[92] S. Sheldon, L. S. Bishop, E. Magesan, S. Filipp, J. M. Chow, and J. M. Gambetta, “Characterizing Errors on Qubit Operations
via Iterative Randomized Benchmarking”, Phys. Rev. A 93, 012301 (2016) (Cited on pages 4, 10, 11, 67).
[93] D. C. McKay, S. Sheldon, J. A. Smolin, J. M. Chow, and J. M. Gambetta, “Three-Qubit Randomized Benchmarking”, Phys. Rev.
Lett. 122, 200502 (2019) (Cited on pages 4, 61).
[94] T. Proctor, S. Seritan, K. Rudinger, E. Nielsen, R. Blume-Kohout, and K. Young, “Scalable Randomized Benchmarking of Quantum
Computers Using Mirror Circuits”, Phys. Rev. Lett. 129, 150502 (2022) (Cited on pages 4, 12).
[95] E. Onorati, A. H. Werner, and J. Eisert, “Randomized Benchmarking for Individual Quantum Gates”, Phys. Rev. Lett. 123, 060501
(2019) (Cited on pages 4, 11).
[96] A. Morvan, V. V. Ramasesh, M. S. Blok, J. M. Kreikebaum, K. O’Brien, L. Chen, B. K. Mitchell, R. K. Naik, D. I. Santiago, and
I. Siddiqi, “Qutrit Randomized Benchmarking”, Phys. Rev. Lett. 126, 210504 (2021) (Cited on pages 4, 11).
[97] L. Govia, P. Jurcevic, C. Wood, N. Kanazawa, S. Merkel, and D. McKay, “A randomized benchmarking suite for mid-circuit
measurements”, New J. Phys. 25, 123016 (2023) (Cited on pages 4, 11, 51).
[98] J. Helsen, I. Roth, E. Onorati, A. H. Werner, and J. Eisert, “General Framework for Randomized Benchmarking”, PRX Quantum
3, 020357 (2022) (Cited on pages 4, 11, 12, 61).
[99] T. Tomesh, P. Gokhale, V. Omole, G. S. Ravi, K. N. Smith, J. Viszlai, X.-C. Wu, N. Hardavellas, M. R. Martonosi, and F. T.
Chong, “SupermarQ: A Scalable Quantum Benchmark Suite”, 2022 IEEE Int Symp High-Perform Comput Arch. HPCA, 587 (2022)
(Cited on pages 4, 13).
[100] R. Kueng, D. M. Long, A. C. Doherty, and S. T. Flammia, “Comparing Experiments to the Fault-Tolerance Threshold”, Phys.
Rev. Lett. 117, 170502 (2016) (Cited on pages 4, 10, 58).
[101] S. T. Merkel, J. M. Gambetta, J. A. Smolin, S. Poletto, A. D. Córcoles, B. R. Johnson, C. A. Ryan, and M. Steffen, “Self-Consistent
Quantum Process Tomography”, Phys. Rev. A 87, 062119 (2013) (Cited on pages 4, 10, 40, 57).
[102] R. Blume-Kohout, J. K. Gamble, E. Nielsen, J. Mizrahi, J. D. Sterk, and P. Maunz, “Robust, self-consistent, closed-form tomography
of quantum logic gates on a trapped ion qubit”, arXiv:1310.4492 (2013) (Cited on pages 4, 10).
[103] E. Nielsen, J. K. Gamble, K. Rudinger, T. Scholten, K. Young, and R. Blume-Kohout, “Gate Set Tomography”, Quantum 5, 557
(2021) (Cited on pages 4, 10, 11, 36, 39–42, 69).
[104] D. Greenbaum, “Introduction to Quantum Gate Set Tomography”, arXiv:1509.02921 (2015) (Cited on pages 4, 10, 36, 39).
[105] R. Brieger, I. Roth, and M. Kliesch, “Compressive gate set tomography”, PRX Quantum 4, 010325 (2023) (Cited on page 4).

111
References

[106] C. Ostrove, K. Rudinger, S. Seritan, K. Young, and R. Blume-Kohout, “Near-minimal gate set tomography experiment designs”,
2023 IEEE Int Conf Quantum Comput Eng QCE 1, 1422 (2023) (Cited on page 4).
[107] S. Cao, D. Lall, M. Bakr, G. Campanaro, S. D. Fasciati, J. Wills, V. Chidambaram, B. Shteynas, I. Rungger, and P. J. Leek,
“Efficient characterization of qudit logical gates with gate set tomography using an error-free virtual Z gate model”, Phys. Rev.
Lett. 133, 120802 (2024) (Cited on pages 4, 10).
[108] G. J. Mooney, C. D. Hill, and L. C. Hollenberg, “Entanglement in a 20-qubit superconducting quantum computer”, Sci. Rep. 9,
13465 (2019) (Cited on page 4).
[109] A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, “Validating Quantum Computers Using Randomized
Model Circuits”, Phys. Rev. A 100, 032328 (2019) (Cited on pages 4, 8, 12, 36, 38, 70, 71).
[110] P. Jurcevic, A. Javadi-Abhari, L. S. Bishop, I. Lauer, D. F. Bogorin, M. Brink, L. Capelluto, O. Günlük, T. Itoko, N. Kanazawa,
et al., “Demonstration of quantum volume 64 on a superconducting quantum computing system”, Quantum Sci. Technol. 6, 025020
(2021) (Cited on pages 4, 12).
[111] E. Pelofske, A. Bärtschi, and S. Eidenbenz, “Quantum volume in practice: What users can expect from nisq devices”, IEEE Trans.
Quantum Eng. 3, 1 (2022) (Cited on pages 4, 5, 12).
[112] K. Mayer, A. Hall, T. Gatterman, S. K. Halit, K. Lee, J. Bohnet, D. Gresh, A. Hankin, K. Gilmore, and J. Gaebler, “Theory of
Mirror Benchmarking and Demonstration on a Quantum Computer”, arXiv:2108.10431 (2021) (Cited on pages 4, 12).
[113] T. Proctor, K. Rudinger, K. Young, E. Nielsen, and R. Blume-Kohout, “Measuring the Capabilities of Quantum Computers”, Nat.
Phys. 18, 75 (2022) (Cited on pages 4, 10, 12, 73, 74).
[114] M. Amico, H. Zhang, P. Jurcevic, L. S. Bishop, P. Nation, A. Wack, and D. C. McKay, “Defining standard strategies for quantum
benchmarks”, arXiv:2303.02108 (2023) (Cited on pages 4, 5, 12).
[115] S. A. Moses et al., “A Race-Track Trapped-Ion Quantum Processor”, Phys. Rev. X 13, 041052 (2023) (Cited on pages 4, 8, 9, 12).
[116] S. Ferracin, T. Kapourniotis, and A. Datta, “Accrediting Outputs of Noisy Intermediate-Scale Quantum Computing Devices”, New
J. Phys. 21, 113038 (2019) (Cited on pages 4, 12, 76–78).
[117] S. Ferracin, S. T. Merkel, D. McKay, and A. Datta, “Experimental Accreditation of Outputs of Noisy Quantum Computers”, Phys.
Rev. A 104, 042603 (2021) (Cited on pages 4, 12, 76–78, 86).
[118] T. Lubinski, S. Johri, P. Varosy, J. Coleman, L. Zhao, J. Necaise, C. H. Baldwin, K. Mayer, and T. Proctor, “Application-Oriented
Performance Benchmarks for Quantum Computing”, IEEE Trans. Quantum Eng. 4, 1 (2023) (Cited on pages 4, 13, 14, 38, 75, 76,
85, 87, 88, 94).
[119] A. Li, S. Stein, S. Krishnamoorthy, and J. Ang, “QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and
Simulation”, ACM Trans. Quantum Comput. 4, 1 (2023) (Cited on pages 4, 13).
[120] T. Patel, A. Potharaju, B. Li, R. B. Roy, and D. Tiwari, “Experimental Evaluation of NISQ Quantum Computers: Error Mea-
surement, Characterization, and Implications”, SC20 Int. Conf. High Perform. Comput. Netw. Storage Anal., 1 (2020) (Cited on
pages 4, 13).
[121] D. Koch, B. Martin, S. Patel, L. Wessing, and P. M. Alsing, “Demonstrating NISQ Era Challenges in Algorithm Design on IBM’s
20 Qubit Quantum Computer”, AIP Adv. 10, 095101 (2020) (Cited on pages 4, 13).
[122] S. Resch and U. R. Karpuzcu, “Benchmarking Quantum Computers and the Impact of Quantum Noise”, ACM Comput. Surv. 54,
1 (2021) (Cited on pages 4, 10, 13).
[123] K. Georgopoulos, C. Emary, and P. Zuliani, “Quantum Computer Benchmarking via Quantum Algorithms”, arXiv:2112.09457
(2021) (Cited on pages 4, 13, 14).
[124] N. Quetschlich, L. Burgholzer, and R. Wille, “MQT Bench: Benchmarking Software and Design Automation Tools for Quantum
Computing”, Quantum 7, 1062 (2023) (Cited on pages 4, 13).
[125] P. Murali, N. M. Linke, M. Martonosi, A. J. Abhari, N. H. Nguyen, and C. H. Alderete, “Full-stack, real-system quantum computer
studies: architectural comparisons and design insights”, Proc 46th Int Symp Comput Arch. ISCA ’19 (2019), 527 (Cited on pages 4,
8, 9, 13, 49).
[126] P.-L. Dallaire-Demers, M. Stęchły, J. F. Gonthier, N. T. Bashige, J. Romero, and Y. Cao, “An Application Benchmark for Fermionic
Quantum Simulations”, arXiv:2003.01862 (2020) (Cited on pages 4, 13).
[127] A. J. McCaskey, Z. P. Parks, J. Jakowski, S. V. Moore, T. D. Morris, T. S. Humble, and R. C. Pooser, “Quantum Chemistry as a
Benchmark for Near-Term Quantum Computers”, npj Quantum Inf. 5, 99 (2019) (Cited on pages 4, 13).
[128] H. Donkers, K. Mesman, Z. Al-Ars, and M. Möller, “QPack Scores: Quantitative Performance Metrics for Application-Oriented
Quantum Computer Benchmarking”, arXiv:2205.12142 (2022) (Cited on pages 4, 13, 14).
[129] N. M. Linke, D. Maslov, M. Roetteler, S. Debnath, C. Figgatt, K. A. Landsman, K. Wright, and C. Monroe, “Experimental
Comparison of Two Quantum Computing Architectures”, Proc. Natl. Acad. Sci. 114, 3305 (2017) (Cited on pages 4, 8, 9, 13, 14,
29).
[130] S. Martiel, T. Ayral, and C. Allouche, “Benchmarking Quantum Coprocessors in an Application-Centric, Hardware-Agnostic, and
Scalable Way”, IEEE Trans. Quantum Eng. 2, 1 (2021) (Cited on pages 4, 13, 16, 81–83).
[131] C.-T. Liao, S. Bahrani, F. F. da Silva, and E. Kashefi, “Benchmarking of Quantum Protocols”, Sci. Rep. 12, 5298 (2022) (Cited
on pages 4, 13).
[132] T. Lubinski, C. Coffrin, C. McGeoch, P. Sathe, J. Apanavicius, D. Bernal Neira, and Q. E. D. C. (.-C. collaboration, “Optimization
applications as quantum performance benchmarks”, ACM Trans. Quantum Comput. (2023) (Cited on pages 4, 14).
[133] T. Lubinski, J. J. Goings, K. Mayer, S. Johri, N. Reddy, A. Mehta, N. Bhatia, S. Rappaport, D. Mills, C. H. Baldwin, et al., “Quan-
tum Algorithm Exploration using Application-Oriented Performance Benchmarks”, arXiv:2402.08985 (2024) (Cited on pages 4, 14).
[134] S. Barz, J. F. Fitzsimons, E. Kashefi, and P. Walther, “Experimental Verification of Quantum Computation”, Nat. Phys. 9, 727
(2013) (Cited on pages 5, 24).
[135] A. Gheorghiu, T. Kapourniotis, and E. Kashefi, “Verification of Quantum Computation: An Overview of Existing Approaches”,
Theory Comput. Syst. 63, 715 (2019) (Cited on pages 5, 76, 77).
[136] J. Eisert, D. Hangleiter, N. Walk, I. Roth, D. Markham, R. Parekh, U. Chabaud, and E. Kashefi, “Quantum Certification and
Benchmarking”, Nat. Rev. Phys. 2, 382 (2020) (Cited on pages 5, 13).
[137] A. Wack, H. Paik, A. Javadi-Abhari, P. Jurcevic, I. Faro, J. M. Gambetta, and B. R. Johnson, “Quality, Speed, and Scale: Three
Key Attributes to Measure the Performance of near-Term Quantum Computers”, arXiv:2110.14108 (2021) (Cited on pages 5, 7,
12, 14, 94).
[138] A. Steane, “Quantum computing”, Rep. Prog. Phys. 61, 117 (1998) (Cited on pages 5, 7).
[139] M. Horowitz and E. Grumbling, “Quantum computing: progress and prospects”, Natl. Acad. Press (2019) (Cited on pages 5, 7).
[140] A. Y. Kitaev, “Quantum computations: algorithms and error correction”, Russ. Math. Surv. 52, 1191 (1997) (Cited on pages 5, 7,
59).
[141] J. Roffe, “Quantum error correction: an introductory guide”, Contemp. Phys. 60, 226 (2019) (Cited on pages 5, 7, 48).
[142] S. J. Devitt, W. J. Munro, and K. Nemoto, “Quantum error correction for beginners”, Rep. Prog. Phys. 76, 076001 (2013) (Cited
on pages 5, 7, 9).
[143] D. A. Lidar and T. A. Brun, “Quantum error correction”, Camb. Univ. Press (2013) (Cited on pages 5, 7).
[144] J. Preskill, “Reliable quantum computers”, Proc. R. Soc. Lond. A 454, 385 (1998) (Cited on pages 5, 7).
[145] E. Knill and R. Laflamme, “Theory of quantum error-correcting codes”, Phys. Rev. A 55, 900 (1997) (Cited on pages 5, 7).
[146] D. Gottesman, “Stabilizer codes and quantum error correction”, Calif. Inst. Technol. (1997) (Cited on pages 5, 7).
[147] D. Gottesman, “An introduction to quantum error correction and fault-tolerant quantum computation”, Quantum Inf. Sci. Its
Contrib. Math. Proc Symp Appl Math, Vol. 68 (2010), 13 (Cited on pages 5, 7, 22).
[148] Google Quantum AI, “Suppressing Quantum Errors by Scaling a Surface Code Logical Qubit”, Nature 614, 676 (2023) (Cited on
pages 5, 9, 10, 19, 22, 50, 91).
[149] M. P. Da Silva, C. Ryan-Anderson, J. M. Bello-Rivas, A. Chernoguzov, J. M. Dreiling, C. Foltz, J. P. Gaebler, T. M. Gatterman,
D. Hayes, N. Hewitt, et al., “Demonstration of logical qubits and repeated error correction with better-than-physical error rates”,
arXiv:2404.02280 (2024) (Cited on pages 5, 19).

112
References

[150] V. V. Sivak, A. Eickbusch, B. Royer, S. Singh, I. Tsioutsios, S. Ganjam, A. Miano, B. L. Brock, A. Z. Ding, L. Frunzio, et al.,
“Real-time quantum error correction beyond break-even”, Nature 616, 50 (2023) (Cited on pages 5, 19).
[151] L. Postler, S. Heußen, I. Pogorelov, M. Rispler, T. Feldker, M. Meth, C. D. Marciniak, R. Stricker, M. Ringbauer, R. Blatt, et al.,
“Demonstration of fault-tolerant universal quantum gate operations”, Nature 605, 675 (2022) (Cited on pages 5, 19).
[152] A. Erhard, H. Poulsen Nautrup, M. Meth, L. Postler, R. Stricker, M. Stadler, V. Negnevitsky, M. Ringbauer, P. Schindler, H. J.
Briegel, et al., “Entangling logical qubits with lattice surgery”, Nature 589, 220 (2021) (Cited on pages 5, 19).
[153] C. Ryan-Anderson et al., “Realization of Real-Time Fault-Tolerant Quantum Error Correction”, Phys. Rev. X 11, 041058 (2021)
(Cited on pages 5, 19).
[154] L. Egan, D. M. Debroy, C. Noel, A. Risinger, D. Zhu, D. Biswas, M. Newman, M. Li, K. R. Brown, M. Cetina, et al., “Fault-tolerant
control of an error-corrected qubit”, Nature 598, 281 (2021) (Cited on pages 5, 19).
[155] Google Quantum AI and Collaborators, “Quantum error correction below the surface code threshold”, arXiv:2408.13687 (2024)
(Cited on pages 5, 19).
[156] B. L. Brock, S. Singh, A. Eickbusch, V. V. Sivak, A. Z. Ding, L. Frunzio, S. M. Girvin, and M. H. Devoret, “Quantum error
correction of qudits beyond break-even”, arXiv:2409.15065 (2024) (Cited on pages 5, 19).
[157] B. W. Reichardt et al., “Demonstration of quantum computation and error correction with a tesseract code”, arXiv:2409.04628
(2024) (Cited on pages 5, 19).
[158] H. Putterman et al., “Hardware-efficient quantum error correction using concatenated bosonic qubits”, arXiv:2409.13025 (2024)
(Cited on pages 5, 19).
[159] A. Acuaviva, D. Aguirre, R. Peña, and M. Sanz, “Benchmarking Quantum Computers: Towards a Standard Performance Evaluation
Approach”, arXiv:2407.10941 (2024) (Cited on page 5).
[160] A. Hashim, L. B. Nguyen, N. Goss, B. Marinelli, R. K. Naik, T. Chistolini, J. Hines, J. Marceaux, Y. Kim, P. Gokhale, et al., “A
practical introduction to benchmarking and characterization of quantum computers”, arXiv:2408.12064 (2024) (Cited on page 5).
[161] A. Javadi-Abhari et al., “Quantum computing with Qiskit”, arXiv:2405.08810 (2024) (Cited on pages 6, 18, 37).
[162] E. Nielsen, S. Seritan, T. Proctor, K. Rudinger, K. Young, A. Russo, R. Blume-Kohout, R. P. Kelly, J. K. Gamble, and L. Saldyt,
“pyGSTio/pyGSTi: Version 0.9.10.1”, version v0.9.10.1, 10.5281/zenodo.6363115 (2022) (Cited on page 6).
[163] M. A. Nielsen and I. Chuang, “Quantum Computation and Quantum Information”, Camb. Univ. Press 2010 (Cited on pages 7–9,
13, 36, 37, 69, 86, 87, 92).
[164] T. D. Ladd, F. Jelezko, R. Laflamme, Y. Nakamura, C. Monroe, and J. L. O’Brien, “Quantum computers”, Nature 464, 45 (2010)
(Cited on page 7).
[165] J. Preskill, “Quantum Computing in the NISQ era and beyond”, Quantum (2018) (Cited on pages 7, 9).
[166] A. D. Corcoles, A. Kandala, A. Javadi-Abhari, D. T. McClure, A. W. Cross, K. Temme, P. D. Nation, M. Steffen, and J. M.
Gambetta, “Challenges and Opportunities of Near-Term Quantum Computing Systems”, Proc. IEEE 108, 1338 (2020) (Cited on
page 7).
[167] IBM, “IBM Unveils 400 Qubit-Plus Quantum Processor and Next-Generation IBM Quantum System Two”, IBM Newsroom, [Ac-
cessed: 1-November-2023] (2022) (Cited on pages 7, 29).
[168] D. Kielpinski, C. Monroe, and D. J. Wineland, “Architecture for a large-scale ion-trap quantum computer”, Nature 417, 709 (2002)
(Cited on page 7).
[169] J. I. Cirac and P. Zoller, “A scalable quantum computer with ions in an array of microtraps”, Nature 404, 579 (2000) (Cited on
page 7).
[170] J. Gambetta, “IBM’s roadmap for scaling quantum technology”, IBM Res. Blog Sept. 2020 (2020) (Cited on page 7).
[171] J. Preskill, “Quantum computing 40 years later”, Feynman Lectures on Computation (CRC Press, 2023), 193 (Cited on page 7).
[172] D. P. DiVincenzo, “The Physical Implementation of Quantum Computation”, Fortschr. Phys. 48, 771 (2000) (Cited on pages 7, 8,
69).
[173] B. O’Gorman, W. J. Huggins, E. G. Rieffel, and K. B. Whaley, “Generalized swap networks for near-term quantum computing”,
arXiv:1905.05118 (2019) (Cited on page 8).
[174] P. Gokhale, T. Tomesh, M. Suchara, and F. T. Chong, “Faster and more reliable quantum swaps via native gates”, arXiv:2109.13199
(2021) (Cited on page 8).
[175] R. Wille, A. Lye, and R. Drechsler, “Optimal SWAP gate insertion for nearest neighbor quantum circuits”, 2014 19th Asia S. Pac.
Autom. Conf ASP-DAC (2014), 489 (Cited on page 8).
[176] A. Holmes, S. Johri, G. G. Guerreschi, J. S. Clarke, and A. Y. Matsuura, “Impact of qubit connectivity on quantum algorithm
performance”, Quantum Sci. Technol. 5, 025009 (2020) (Cited on pages 8, 48).
[177] S. Herbert, “On the depth overhead incurred when running quantum algorithms on near-term quantum computers with limited
qubit connectivity”, arXiv:1805.12570 (2018) (Cited on page 8).
[178] C. G. Almudever, L. Lao, R. Wille, and G. G. Guerreschi, “Realizing quantum algorithms on real quantum computing devices”,
2020 Autom. Test Eur. Conf Exhib DATE (2020), 864 (Cited on page 8).
[179] A. Cowtan, S. Dilkes, R. Duncan, A. Krajenbrink, W. Simmons, and S. Sivarajah, “On the qubit routing problem”, arXiv:1902.08091
(2019) (Cited on pages 8, 48).
[180] W.-H. Lin, B. Tan, M. Y. Niu, J. Kimko, and J. Cong, “Domain-specific quantum architecture optimization”, IEEE J. Emerg. Sel.
Top. Circuits Syst. 12, 624 (2022) (Cited on page 8).
[181] A. Zulehner, A. Paler, and R. Wille, “An Efficient Methodology for Mapping Quantum Circuits to the IBM QX Architectures”,
IEEE Trans Comput-Aided Integr Circuits Syst 38, 1226 (2019) (Cited on page 8).
[182] G. Li, Y. Ding, and Y. Xie, “Tackling the qubit mapping problem for NISQ-era quantum devices”, Proc 24th Int Conf Archit.
Support Program Lang. Oper. Syst (2019), 1001 (Cited on page 8).
[183] H.-L. Huang, X.-Y. Xu, C. Guo, G. Tian, S.-J. Wei, X. Sun, W.-S. Bao, and G.-L. Long, “Near-term quantum computing techniques:
Variational quantum algorithms, error mitigation, circuit compilation, benchmarking and classical simulation”, Sci. China Phys.
Mech. Astron. 66, 250302 (2023) (Cited on page 8).
[184] M. A. Serrano, J. A. Cruz-Lemus, R. Perez-Castillo, and M. Piattini, “Quantum software components and platforms: Overview
and quality assessment”, ACM Comput. Surv. 55, 1 (2022) (Cited on page 8).
[185] M. Y. Siraichi, V. F. dos Santos, C. Collange, and F. M. Q. Pereira, “Qubit allocation”, Proc. 2018 Int. Symp. Code Gen. Optim.
(2018), 113 (Cited on page 8).
[186] S. Sivarajah, S. Dilkes, A. Cowtan, W. Simmons, A. Edgington, and R. Duncan, “T| ket¿: a retargetable compiler for NISQ devices”,
Quantum Sci. Technol. 6, 014003 (2020) (Cited on pages 8, 9).
[187] S. Crain, C. Cahall, G. Vrijsen, E. E. Wollman, M. D. Shaw, V. B. Verma, S. W. Nam, and J. Kim, “High-Speed Low-Crosstalk
Detection of a 171Yb+ Qubit Using Superconducting Nanowire Single Photon Detectors”, Nat. Commun. 2, 1 (2019) (Cited on
pages 8, 49, 91).
[188] V. V. Shende, S. S. Bullock, and I. L. Markov, “Synthesis of quantum logic circuits”, Proc 2005 Asia S. Pac. Autom. Conf (2005),
272 (Cited on page 9).
[189] M. Amy, D. Maslov, M. Mosca, and M. Roetteler, “A meet-in-the-middle algorithm for fast synthesis of depth-optimal quantum
circuits”, IEEE Trans Comput-Aided Integr Circuits Syst 32, 818 (2013) (Cited on page 9).
[190] S. Khatri, R. LaRose, A. Poremba, L. Cincio, A. T. Sornborger, and P. J. Coles, “Quantum-assisted quantum compiling”, Quantum
3, 140 (2019) (Cited on page 9).
[191] Y. Shi, P. Gokhale, P. Murali, J. M. Baker, C. Duckering, Y. Ding, N. C. Brown, C. Chamberland, A. Javadi-Abhari, A. W. Cross,
et al., “Resource-efficient quantum computing by breaking abstractions”, Proc. IEEE 108, 1353 (2020) (Cited on page 9).
[192] M. Alam, A. Ash-Saki, and S. Ghosh, “Circuit Compilation Methodologies for Quantum Approximate Optimization Algorithm”,
2020 53rd Annu IEEEACM Int Symp Microarchit MICRO (2020), 215 (Cited on page 9).
[193] A. McCaskey, E. Dumitrescu, D. Liakh, and T. Humble, “Hybrid programming for near-term quantum computing systems”, 2018
IEEE Int Conf Rebooting Comput ICRC (2018), 1 (Cited on page 9).
[194] D. M. Miller, R. Wille, and Z. Sasanian, “Elementary quantum gate realizations for multiple-control Toffoli gates”, 2011 41st IEEE
Int Symp Mult.-Valued Log (2011), 288 (Cited on page 9).

113
References

[195] T. Häner, D. S. Steiger, K. Svore, and M. Troyer, “A software methodology for compiling quantum programs”, Quantum Sci.
Technol. 3, 020501 (2018) (Cited on page 9).
[196] J. Yirka and Y. Subaşı, “Qubit-efficient entanglement spectroscopy using qubit resets”, Quantum 5, 535 (2021) (Cited on page 9).
[197] R. Yalovetzky, P. Minssen, D. Herman, and M. Pistoia, “Hybrid HHL with dynamic quantum circuits on real hardware”, arXiv:2110.15958
(2021) (Cited on page 9).
[198] E. Chertkov, J. Bohnet, D. Francois, J. Gaebler, D. Gresh, A. Hankin, K. Lee, D. Hayes, B. Neyenhuis, R. Stutz, et al., “Holographic
dynamics simulations with a trapped-ion quantum computer”, Nat. Phys. 18, 1074 (2022) (Cited on page 9).
[199] E. Chertkov, Z. Cheng, A. C. Potter, S. Gopalakrishnan, T. M. Gatterman, J. A. Gerber, K. Gilmore, D. Gresh, A. Hall, A. Hankin,
et al., “Characterizing a non-equilibrium phase transition on a quantum computer”, Nat. Phys. 19, 1799 (2023) (Cited on page 9).
[200] M. Foss-Feig, D. Hayes, J. M. Dreiling, C. Figgatt, J. P. Gaebler, S. A. Moses, J. M. Pino, and A. C. Potter, “Holographic quantum
algorithms for simulating correlated spin systems”, Phys. Rev. Res. 3, 033002 (2021) (Cited on page 9).
[201] M. DeCross, E. Chertkov, M. Kohagen, and M. Foss-Feig, “Qubit-Reuse Compilation with Mid-Circuit Measurement and Reset”,
Phys. Rev. X 13, 041057 (2023) (Cited on pages 9, 50).
[202] A. Y. Kitaev, “Quantum measurements and the Abelian stabilizer problem”, arXiv:quant-ph/9511026 (1995) (Cited on pages 9,
13).
[203] T. Monz, D. Nigg, E. A. Martinez, M. F. Brandl, P. Schindler, R. Rines, S. X. Wang, I. L. Chuang, and R. Blatt, “Realization of
a scalable Shor algorithm”, Science 351, 1068 (2016) (Cited on page 9).
[204] R. B. Griffiths and C.-S. Niu, “Semiclassical Fourier Transform for Quantum Computation”, Phys. Rev. Lett. 76, 3228 (1996)
(Cited on page 9).
[205] S. Parker and M. B. Plenio, “Efficient Factorization with a Single Pure Qubit and logN Mixed Qubits”, Phys. Rev. Lett. 85, 3049
(2000) (Cited on page 9).
[206] M. Mosca and A. Ekert, “The hidden subgroup problem and eigenvalue estimation on a quantum computer”, NASA Int. Conf.
Quantum Comput. Quantum Commun. (1998), 174 (Cited on page 9).
[207] W. H. Zurek, “Decoherence and the transition from quantum to classical”, Phys. Today 44, 36 (1991) (Cited on page 9).
[208] M. Schlosshauer, “Quantum decoherence”, Phys. Rep. 831, 1 (2019) (Cited on pages 9, 10).
[209] C. H. Bennett and D. P. DiVincenzo, “Quantum information and computation”, Nature 404, 247 (2000) (Cited on page 9).
[210] D. Bouwmeester and A. Zeilinger, “The physics of quantum information: basic concepts”, The physics of quantum information:
quantum cryptography, quantum teleportation, quantum computation (Springer, 2000), 1 (Cited on page 9).
[211] H. E. Brandt, “Qubit devices and the issue of quantum decoherence”, Prog. Quantum Electron. 22, 257 (1999) (Cited on page 9).
[212] J. Preskill, “Lecture notes for physics 229: Quantum information and computation”, Calif. Inst. Technol. 16, 1 (1998) (Cited on
page 9).
[213] J. J. Burnett, A. Bengtsson, M. Scigliuzzo, D. Niepce, M. Kudra, P. Delsing, and J. Bylander, “Decoherence benchmarking of
superconducting qubits”, npj Quantum Inf. 5, 54 (2019) (Cited on pages 9, 10, 15).
[214] R. McDermott, “Materials origins of decoherence in superconducting qubits”, IEEE Trans. Appl. Supercon. 19, 2 (2009) (Cited on
page 9).
[215] L. Chirolli and G. Burkard, “Decoherence in solid-state qubits”, Adv. Phys. 57, 225 (2008) (Cited on page 9).
[216] P. V. Klimov et al., “Fluctuations of Energy-Relaxation Times in Superconducting Qubits”, Phys. Rev. Lett. 121, 10.1103/physre
vlett.121.090502 (2018) (Cited on pages 9, 15).
[217] M. Carroll, S. Rosenblatt, P. Jurcevic, I. Lauer, and A. Kandala, “Dynamics of superconducting qubit relaxation times”, npj
Quantum Inf. 8, 132 (2022) (Cited on pages 9, 15).
[218] C. J. Ballance, T. P. Harty, N. M. Linke, M. A. Sepiol, and D. M. Lucas, “High-Fidelity Quantum Logic Gates Using Trapped-Ion
Hyperfine Qubits”, Phys. Rev. Lett. 117, 060504 (2016) (Cited on pages 9, 30).
[219] M. B. Plenio and P. L. Knight, “Decoherence limits to quantum computation using trapped ions”, Proc. R. Soc. Lond. A 453, 2017
(1997) (Cited on pages 9, 10).
[220] D. Kielpinski, “Entanglement and Decoherence in a Trapped-ion Quantum Register”, Univ. Colo. (2001) (Cited on page 9).
[221] Á. Rivas, S. F. Huelga, and M. B. Plenio, “Quantum non-Markovianity: characterization, quantification and detection”, Rep. Prog.
Phys. 77, 094001 (2014) (Cited on pages 9, 56).
[222] H. Breuer and F. Petruccione, “The Theory of Open Quantum Systems”, Oxf. Univ. Press 2002 (Cited on page 9).
[223] A. Agarwal, L. P. Lindoy, D. Lall, F. Jamet, and I. Rungger, “Modelling non-markovian noise in driven superconducting qubits”,
Quantum Sci. Technol. 9, 035017 (2024) (Cited on pages 9, 15, 37, 42, 55, 56, 68).
[224] B. Gulácsi and G. Burkard, “Signatures of non-Markovianity of a superconducting qubit”, Phys. Rev. B 107, 174511 (2023) (Cited
on pages 9, 55).
[225] S. Maniscalco and F. Petruccione, “Non-Markovian dynamics of a qubit”, Phys. Rev. A 73, 012111 (2006) (Cited on page 9).
[226] G. A. White, C. D. Hill, F. A. Pollock, L. C. Hollenberg, and K. Modi, “Demonstration of non-Markovian process characterisation
and control on a quantum processor”, Nat. Commun. 11, 6301 (2020) (Cited on pages 9, 10).
[227] G. White, F. Pollock, L. Hollenberg, K. Modi, and C. Hill, “Non-Markovian Quantum Process Tomography”, PRX Quantum 3,
020344 (2022) (Cited on pages 9, 10).
[228] R. Y. Su, J. Y. Huang, N. D. Stuyck, M. K. Feng, W. Gilbert, T. J. Evans, W. H. Lim, F. E. Hudson, K. W. Chan, W. Huang, et al.,
“Characterizing non-markovian quantum processes by fast bayesian tomography”, arXiv:2307.12452 (2023) (Cited on pages 9, 10).
[229] I. A. Luchnikov, S. V. Vintskevich, D. A. Grigoriev, and S. N. Filippov, “Machine Learning Non-Markovian Quantum Dynamics”,
Phys. Rev. Lett. 124, 140502 (2020) (Cited on pages 9, 10).
[230] Z.-T. Li, C.-C. Zheng, F.-X. Meng, H. Zeng, T. Luan, Z.-C. Zhang, and X.-T. Yu, “Non-Markovian quantum gate set tomography”,
Quantum Sci. Technol. 9, 035027 (2024) (Cited on pages 9, 10).
[231] P. Figueroa-Romero, K. Modi, R. J. Harris, T. M. Stace, and M.-H. Hsieh, “Randomized Benchmarking for Non-Markovian Noise”,
PRX Quantum 2, 040351 (2021) (Cited on pages 9, 61).
[232] E. Lucero et al., “Reduced phase error through optimized control of a superconducting qubit”, Phys. Rev. A 82, 042339 (2010)
(Cited on pages 10, 11, 49).
[233] D. C. McKay, C. J. Wood, S. Sheldon, J. M. Chow, and J. M. Gambetta, “Efficient Z gates for quantum computing”, Phys. Rev.
A 96, 022330 (2017) (Cited on pages 10, 14, 38, 49, 89).
[234] N. Wittler, F. Roy, K. Pack, M. Werninghaus, A. S. Roy, D. J. Egger, S. Filipp, F. K. Wilhelm, and S. Machnes, “Integrated Tool
Set for Control, Calibration, and Characterization of Quantum Devices Applied to Superconducting Qubits”, Phys. Rev. Appl. 15,
034080 (2021) (Cited on pages 10, 11).
[235] J. Wallman, C. Granade, R. Harper, and S. T. Flammia, “Estimating the coherence of noise”, New J. Phys. 17, 113020 (2015)
(Cited on page 10).
[236] Y. Shapira, R. Shaniv, T. Manovitz, N. Akerman, and R. Ozeri, “Robust Entanglement Gates for Trapped-Ion Qubits”, Phys. Rev.
Lett. 121, 180502 (2018) (Cited on page 10).
[237] T. P. Harty, D. T. C. Allcock, C. J. Ballance, L. Guidoni, H. A. Janacek, N. M. Linke, D. N. Stacey, and D. M. Lucas, “High-
Fidelity Preparation, Gates, Memory, and Readout of a Trapped-Ion Quantum Bit”, Phys. Rev. Lett. 113, 220501 (2014) (Cited
on pages 10, 11, 30, 49, 89).
[238] M. Werninghaus, D. J. Egger, F. Roy, S. Machnes, F. K. Wilhelm, and S. Filipp, “Leakage reduction in fast superconducting qubit
gates via optimal control”, npj Quantum Inf. 7, 14 (2021) (Cited on page 10).
[239] F. Motzoi, J. M. Gambetta, P. Rebentrost, and F. K. Wilhelm, “Simple Pulses for Elimination of Leakage in Weakly Nonlinear
Qubits”, Phys. Rev. Lett. 103, 110501 (2009) (Cited on page 10).
[240] Z. Chen et al., “Measuring and Suppressing Quantum State Leakage in a Superconducting Qubit”, Phys. Rev. Lett. 116, 020501
(2016) (Cited on page 10).
[241] C. J. Wood and J. M. Gambetta, “Quantification and characterization of leakage errors”, Phys. Rev. A 97, 032306 (2018) (Cited
on page 10).

114
References

[242] D. Hayes, D. Stack, B. Bjork, A. C. Potter, C. H. Baldwin, and R. P. Stutz, “Eliminating Leakage Errors in Hyperfine Qubits”,
Phys. Rev. Lett. 124, 170501 (2020) (Cited on page 10).
[243] V. Negnevitsky, M. Marinelli, K. K. Mehta, H.-Y. Lo, C. Flühmann, and J. P. Home, “Repeated multi-qubit readout and feedback
with a mixed-species trapped-ion register”, Nature 563, 527 (2018) (Cited on page 10).
[244] H. Levine et al., “Parallel Implementation of High-Fidelity Multiqubit Gates with Neutral Atoms”, Phys. Rev. Lett. 123, 170503
(2019) (Cited on page 10).
[245] T. Xia, M. Lichtman, K. Maller, A. W. Carr, M. J. Piotrowicz, L. Isenhower, and M. Saffman, “Randomized Benchmarking of
Single-Qubit Gates in a 2D Array of Neutral-Atom Qubits”, Phys. Rev. Lett. 114, 100503 (2015) (Cited on page 10).
[246] S. Krinner, S. Lazar, A. Remm, C. Andersen, N. Lacroix, G. Norris, C. Hellings, M. Gabureac, C. Eichler, and A. Wallraff,
“Benchmarking Coherent Errors in Controlled-Phase Gates due to Spectator Qubits”, Phys. Rev. Appl. 14, 024042 (2020) (Cited
on page 10).
[247] M. Sarovar, T. Proctor, K. Rudinger, K. Young, E. Nielsen, and R. Blume-Kohout, “Detecting Crosstalk Errors in Quantum
Information Processors”, Quantum 4, 321 (2020) (Cited on pages 10, 11).
[248] P. Mundada, G. Zhang, T. Hazard, and A. Houck, “Suppression of Qubit Crosstalk in a Tunable Coupling Superconducting Circuit”,
Phys. Rev. Appl. 12, 054023 (2019) (Cited on page 10).
[249] I. Heinz and G. Burkard, “Crosstalk analysis for single-qubit and two-qubit gates in spin qubit arrays”, Phys. Rev. B 104, 045420
(2021) (Cited on page 10).
[250] C. Piltz, T. Sriarunothai, A. F. Varón, and C. Wunderlich, “A trapped-ion-based quantum byte with 10- 5 next-neighbour cross-
talk”, Nat. Commun. 5, 4679 (2014) (Cited on page 10).
[251] K. K. Mehta, C. D. Bruzewicz, R. McConnell, R. J. Ram, J. M. Sage, and J. Chiaverini, “Integrated optical addressing of an ion
qubit”, Nat. Nanotech. 11, 1066 (2016) (Cited on page 10).
[252] I. Pogorelov et al., “Compact Ion-Trap Quantum Computing Demonstrator”, PRX Quantum 2, 020343 (2021) (Cited on page 10).
[253] L. A. Zhukas, P. Svihra, A. Nomerotski, and B. B. Blinov, “High-fidelity simultaneous detection of a trapped-ion qubit register”,
Phys. Rev. A 103, 062614 (2021) (Cited on page 10).
[254] J. P. Gaebler, C. H. Baldwin, S. A. Moses, J. M. Dreiling, C. Figgatt, M. Foss-Feig, D. Hayes, and J. M. Pino, “Suppression of
midcircuit measurement crosstalk errors with micromotion”, Phys. Rev. A 104, 062440 (2021) (Cited on page 10).
[255] V. Tripathi, H. Chen, M. Khezri, K.-W. Yip, E. Levenson-Falk, and D. A. Lidar, “Suppression of Crosstalk in Superconducting
Qubits Using Dynamical Decoupling”, Phys. Rev. Appl. 18, 024068 (2022) (Cited on page 10).
[256] S. Sheldon, E. Magesan, J. M. Chow, and J. M. Gambetta, “Procedure for systematically tuning up cross-talk in the cross-resonance
gate”, Phys. Rev. A 93, 060302 (2016) (Cited on page 10).
[257] P. Zhao, K. Linghu, Z. Li, P. Xu, R. Wang, G. Xue, Y. Jin, and H. Yu, “Quantum Crosstalk Analysis for Simultaneous Gate
Operations on Superconducting Qubits”, PRX Quantum 3, 020301 (2022) (Cited on page 10).
[258] Z. Zhou, R. Sitler, Y. Oda, K. Schultz, and G. Quiroz, “Quantum Crosstalk Robust Quantum Control”, Phys. Rev. Lett. 131,
210802 (2023) (Cited on page 10).
[259] P. Parrado-Rodríguez, C. Ryan-Anderson, A. Bermudez, and M. Müller, “Crosstalk suppression for fault-tolerant quantum error
correction with trapped ions”, Quantum 5, 487 (2021) (Cited on page 10).
[260] C. Ospelkaus, C. E. Langer, J. M. Amini, K. R. Brown, D. Leibfried, and D. J. Wineland, “Trapped-Ion Quantum Logic Gates
Based on Oscillating Magnetic Fields”, Phys. Rev. Lett. 101, 090502 (2008) (Cited on pages 10, 30).
[261] E. Urban, T. A. Johnson, T. Henage, L. Isenhower, D. D. Yavuz, T. G. Walker, and M. Saffman, “Observation of Rydberg blockade
between two atoms”, Nat. Phys. 5, 110 (2009) (Cited on page 10).
[262] H. Levine, A. Keesling, A. Omran, H. Bernien, S. Schwartz, A. S. Zibrov, M. Endres, M. Greiner, V. Vuletić ć, and M. D. Lukin,
“High-Fidelity Control and Entanglement of Rydberg-Atom Qubits”, Phys. Rev. Lett. 121, 123603 (2018) (Cited on page 10).
[263] A. D. Córcoles, J. M. Gambetta, J. M. Chow, J. A. Smolin, M. Ware, J. Strand, B. L. T. Plourde, and M. Steffen, “Process
verification of two-qubit quantum gates by randomized benchmarking”, Phys. Rev. A 87, 030301 (2013) (Cited on page 10).
[264] W. Huang, C. Yang, K. Chan, T. Tanttu, B. Hensen, R. Leon, M. Fogarty, J. Hwang, F. Hudson, K. M. Itoh, et al., “Fidelity
benchmarks for two-qubit gates in silicon”, Nature 569, 532 (2019) (Cited on page 10).
[265] F. Marxer et al., “Long-distance transmon coupler with cz-gate fidelity above 99.8%”, PRX Quantum 4, 010314 (2023) (Cited on
page 10).
[266] G. M. D’Ariano and P. Lo Presti, “Quantum Tomography for Measuring Experimentally the Matrix Elements of an Arbitrary
Quantum Operation”, Phys. Rev. Lett. 86, 4195 (2001) (Cited on page 10).
[267] J. B. Altepeter, D. Branning, E. Jeffrey, T. C. Wei, P. G. Kwiat, R. T. Thew, J. L. O’Brien, M. A. Nielsen, and A. G. White,
“Ancilla-Assisted Quantum Process Tomography”, Phys. Rev. Lett. 90, 193601 (2003) (Cited on page 10).
[268] A. M. Childs, I. L. Chuang, and D. W. Leung, “Realization of quantum process tomography in NMR”, Phys. Rev. A 64, 012314
(2001) (Cited on page 10).
[269] R. C. Bialczak, M. Ansmann, M. Hofheinz, E. Lucero, M. Neeley, A. D. O’Connell, D. Sank, H. Wang, J. Wenner, M. Steffen,
et al., “Quantum process tomography of a universal entangling gate implemented with Josephson phase qubits”, Nat. Phys. 6, 409
(2010) (Cited on page 10).
[270] A. Bendersky, F. Pastawski, and J. P. Paz, “Selective and Efficient Estimation of Parameters for Quantum Process Tomography”,
Phys. Rev. Lett. 100, 190403 (2008) (Cited on page 10).
[271] M. Mohseni, A. T. Rezakhani, and D. A. Lidar, “Quantum-process tomography: Resource analysis of different strategies”, Phys.
Rev. A 77, 032322 (2008) (Cited on page 10).
[272] D. Leibfried, D. M. Meekhof, B. E. King, C. Monroe, W. M. Itano, and D. J. Wineland, “Experimental Determination of the
Motional Quantum State of a Trapped Atom”, Phys. Rev. Lett. 77, 4281 (1996) (Cited on page 10).
[273] K. Banaszek, M. Cramer, and D. Gross, “Focus on quantum tomography”, New J. Phys 15, 125020 (2013) (Cited on page 10).
[274] R. Blume-Kohout, “Optimal, reliable estimation of quantum states”, New J. Phys. 12, 043034 (2010) (Cited on page 10).
[275] M. Christandl and R. Renner, “Reliable Quantum State Tomography”, Phys. Rev. Lett. 109, 120403 (2012) (Cited on page 10).
[276] C. Granade, J. Combes, and D. G. Cory, “Practical bayesian tomography”, New J. Phys. 18, 033024 (2016) (Cited on page 10).
[277] J. Haah, A. W. Harrow, Z. Ji, X. Wu, and N. Yu, “Sample-optimal tomography of quantum states”, Proc. 48th Annu. ACM Symp.
Theory Comput. (2016), 913 (Cited on page 10).
[278] D. F. V. James, P. G. Kwiat, W. J. Munro, and A. G. White, “Measurement of qubits”, Phys. Rev. A 64, 052312 (2001) (Cited on
page 10).
[279] J. A. Smolin, J. M. Gambetta, and G. Smith, “Efficient Method for Computing the Maximum-Likelihood Quantum State from
Measurements with Additive Gaussian Noise”, Phys. Rev. Lett. 108, 070502 (2012) (Cited on page 10).
[280] R. Blume-Kohout, J. K. Gamble, E. Nielsen, K. Rudinger, J. Mizrahi, K. Fortier, and P. Maunz, “Demonstration of Qubit Operations
below a Rigorous Fault Tolerance Threshold with Gate Set Tomography”, Nat. Commun. 8, 14485 (2017) (Cited on pages 10, 11,
40).
[281] J. P. Dehollain, J. T. Muhonen, R. Blume-Kohout, K. M. Rudinger, J. K. Gamble, E. Nielsen, A. Laucht, S. Simmons, R. Kalra,
A. S. Dzurak, et al., “Optimization of a solid-state electron spin qubit using gate set tomography”, New J. Phys. 18, 103018 (2016)
(Cited on page 10).
[282] E. Nielsen, K. Rudinger, T. Proctor, A. Russo, K. Young, and R. Blume-Kohout, “Probing quantum processor performance with
pyGSTi”, Quantum Sci. Technol. 5, 044002 (2020) (Cited on pages 10, 18, 42, 60, 69).
[283] A. G. White, A. Gilchrist, G. J. Pryde, J. L. O’Brien, M. J. Bremner, and N. K. Langford, “Measuring two-qubit gates”, J. Opt.
Soc. Am. B 24, 172 (2007) (Cited on page 10).
[284] C. Guo, K. Modi, and D. Poletti, “Tensor-network-based machine learning of non-Markovian quantum processes”, Phys. Rev. A
102, 062414 (2020) (Cited on page 10).
[285] M. D. Bowdrey, D. K. Oi, A. J. Short, K. Banaszek, and J. A. Jones, “Fidelity of single qubit maps”, Phys. Lett. A 294, 258 (2002)
(Cited on page 10).
[286] D. Aharonov and M. Ben-Or, “Fault-tolerant quantum computation with constant error”, Proc. 29th Annu. ACM Symp. Theory
Comput. STOC ’97 (1997), 176 (Cited on pages 10, 22).

115
References

[287] S. Kimmel, G. H. Low, and T. J. Yoder, “Robust calibration of a universal single-qubit gate set via robust phase estimation”, Phys.
Rev. A 92, 062315 (2015) (Cited on page 11).
[288] E. Magesan, J. M. Gambetta, and J. Emerson, “Scalable and Robust Randomized Benchmarking of Quantum Processes”, Phys.
Rev. Lett. 106, 180504 (2011) (Cited on pages 11, 36, 60).
[289] T. Proctor, K. Rudinger, K. Young, M. Sarovar, and R. Blume-Kohout, “What Randomized Benchmarking Actually Measures”,
Phys. Rev. Lett. 119, 130502 (2017) (Cited on page 11).
[290] W. Huang et al., “Fidelity Benchmarks for Two-Qubit Gates in Silicon”, Nature 569, 532 (2019) (Cited on page 11).
[291] R. Harper and S. T. Flammia, “Estimating the fidelity of T gates using standard interleaved randomized benchmarking”, Quantum
Sci Technol 2, 015008 (2017) (Cited on page 11).
[292] Y. Liu, M. Otten, R. Bassirianjahromi, L. Jiang, and B. Fefferman, “Benchmarking Near-Term Quantum Computers via Random
Circuit Sampling”, arXiv:2105.05232 (2022) (Cited on page 11).
[293] A. Erhard, J. J. Wallman, L. Postler, M. Meth, R. Stricker, E. A. Martinez, P. Schindler, T. Monz, J. Emerson, and R. Blatt,
“Characterizing Large-Scale Quantum Computers via Cycle Benchmarking”, Nat. Commun. 10, 5347 (2019) (Cited on pages 11,
64, 65).
[294] Z. Zhang, S. Chen, Y. Liu, and L. Jiang, “A generalized cycle benchmarking algorithm for characterizing mid-circuit measurements”,
arXiv:2406.02669 (2024) (Cited on page 11).
[295] A. Hashim, S. Seritan, T. Proctor, K. Rudinger, N. Goss, R. K. Naik, J. M. Kreikebaum, D. I. Santiago, and I. Siddiqi, “Bench-
marking quantum logic operations relative to thresholds for fault tolerance”, Npj Quantum Inf. 9, 109 (2023) (Cited on pages 11,
59, 64).
[296] B. K. Mitchell, R. K. Naik, A. Morvan, A. Hashim, J. M. Kreikebaum, B. Marinelli, W. Lavrijsen, K. Nowrouzi, D. I. Santiago, and
I. Siddiqi, “Hardware-Efficient Microwave-Activated Tunable Coupling between Superconducting Qubits”, Phys. Rev. Lett. 127,
200502 (2021) (Cited on page 11).
[297] A. Hashim et al., “Randomized Compiling for Scalable Quantum Computing on a Noisy Superconducting Quantum Processor”,
Phys. Rev. X 11, 041039 (2021) (Cited on page 11).
[298] J. J. Wallman and J. Emerson, “Noise Tailoring for Scalable Quantum Computation via Randomized Compiling”, Phys. Rev. A
94, 052325 (2016) (Cited on pages 11, 64, 66).
[299] D. C. McKay, A. W. Cross, C. J. Wood, and J. M. Gambetta, “Correlated Randomized Benchmarking”, arXiv:2003.02354 (2020)
(Cited on page 11).
[300] S. Lază ăr et al., “Calibration of Drive Nonlinearity for Arbitrary-Angle Single-Qubit Gates Using Error Amplification”, Phys. Rev.
Appl. 20, 024036 (2023) (Cited on page 11).
[301] J. T. Thomas, M. Lababidi, and M. Tian, “Robustness of single-qubit geometric gate against systematic error”, Phys. Rev. A 84,
042335 (2011) (Cited on page 11).
[302] J. Berberich, D. Fink, and C. Holm, “Robustness of quantum algorithms against coherent control errors”, Phys. Rev. A 109, 012417
(2024) (Cited on page 11).
[303] C. J. Trout, M. Li, M. Gutiérrez, Y. Wu, S.-T. Wang, L. Duan, and K. R. Brown, “Simulating the performance of a distance-3
surface code in a linear ion trap”, New J. Phys. 20, 043038 (2018) (Cited on page 11).
[304] A. Vepsäläinen, R. Winik, A. H. Karamlou, J. Braumüller, A. D. Paolo, Y. Sung, B. Kannan, M. Kjaergaard, D. K. Kim, A. J.
Melville, et al., “Improving qubit coherence using closed-loop feedback”, Nat. Commun. 13, 1932 (2022) (Cited on pages 11, 15).
[305] Y. Chen, M. Farahzad, S. Yoo, and T.-C. Wei, “Detector tomography on IBM quantum computers and mitigation of an imperfect
measurement”, Phys. Rev. A 100, 052315 (2019) (Cited on page 11).
[306] M. R. Geller and M. Sun, “Toward efficient correction of multiqubit measurement errors: Pair correlation method”, Quantum Sci
Technol 6, 025009 (2021) (Cited on page 11).
[307] S. Bravyi, S. Sheldon, A. Kandala, D. C. Mckay, and J. M. Gambetta, “Mitigating measurement errors in multiqubit experiments”,
Phys. Rev. A 103, 042605 (2021) (Cited on page 11).
[308] J. Fiuráš šek, “Maximum-likelihood estimation of quantum measurement”, Phys. Rev. A 64, 024102 (2001) (Cited on page 11).
[309] A. C. Keith, C. H. Baldwin, S. Glancy, and E. Knill, “Joint quantum-state and measurement tomography with incomplete mea-
surements”, Phys. Rev. A 98, 042318 (2018) (Cited on page 11).
[310] M. Sun and M. R. Geller, “Efficient characterization of correlated SPAM errors”, arXiv:1810.10523 (2018) (Cited on page 11).
[311] J. M. Chow, L. DiCarlo, J. M. Gambetta, A. Nunnenkamp, L. S. Bishop, L. Frunzio, M. H. Devoret, S. M. Girvin, and R. J.
Schoelkopf, “Detecting highly entangled states with a joint qubit readout”, Phys. Rev. A 81, 062325 (2010) (Cited on page 11).
[312] L. DiCarlo, M. D. Reed, L. Sun, B. R. Johnson, J. M. Chow, J. M. Gambetta, L. Frunzio, S. M. Girvin, M. H. Devoret, and R. J.
Schoelkopf, “Preparation and measurement of three-qubit entanglement in a superconducting circuit”, Nature 467, 574 (2010)
(Cited on page 11).
[313] A. H. Myerson, D. J. Szwer, S. C. Webster, D. T. C. Allcock, M. J. Curtis, G. Imreh, J. A. Sherman, D. N. Stacey, A. M. Steane,
and D. M. Lucas, “High-Fidelity Readout of Trapped-Ion Qubits”, Phys. Rev. Lett. 100, 200502 (2008) (Cited on pages 11, 30).
[314] H. Landa, D. Meirom, N. Kanazawa, M. Fitzpatrick, and C. J. Wood, “Experimental Bayesian estimation of quantum state
preparation, measurement, and gate errors in multiqubit devices”, Phys. Rev. Res. 4, 013199 (2022) (Cited on page 11).
[315] J. S. Lundeen, A. Feito, H. Coldenstrodt-Ronge, K. L. Pregnell, C. Silberhorn, T. C. Ralph, J. Eisert, M. B. Plenio, and I. A.
Walmsley, “Tomography of quantum detectors”, Nat. Phys. 5, 27 (2009) (Cited on page 11).
[316] F. B. Maciejewski, Z. Zimborás, and M. Oszmaniec, “Mitigation of readout noise in near-term quantum devices by classical post-
processing based on detector tomography”, Quantum 4, 257 (2020) (Cited on page 11).
[317] A. Luis and L. L. Sánchez-Soto, “Complete Characterization of Arbitrary Quantum Measurement Processes”, Phys. Rev. Lett. 83,
3573 (1999) (Cited on page 11).
[318] J. Z. Blumoff et al., “Implementing and Characterizing Precise Multiqubit Measurements”, Phys. Rev. X 6, 031041 (2016) (Cited
on page 11).
[319] J. Lin, J. J. Wallman, I. Hincks, and R. Laflamme, “Independent state and measurement characterization for quantum computers”,
Phys. Rev. Res. 3, 033285 (2021) (Cited on page 11).
[320] R. Blume-Kohout and K. C. Young, “A Volumetric Framework for Quantum Computer Benchmarks”, Quantum 4, 362 (2020)
(Cited on pages 12, 38).
[321] D. Mills, S. Sivarajah, T. L. Scholten, and R. Duncan, “Application-Motivated, Holistic Benchmarking of a Full Quantum Computing
Stack”, Quantum 5, 415 (2021) (Cited on page 12).
[322] C. H. Baldwin, K. Mayer, N. C. Brown, C. Ryan-Anderson, and D. Hayes, “Re-Examining the Quantum Volume Test: Ideal
Distributions, Compiler Optimizations, Confidence Intervals, and Scalable Resource Estimations”, Quantum 6, 707 (2022) (Cited
on pages 12, 72).
[323] K. Miller, C. Broomfield, A. Cox, J. Kinast, and B. Rodenburg, “An Improved Volumetric Metric for Quantum Computers via
more Representative Quantum Circuit Shapes”, arXiv:2207.02315 (2022) (Cited on page 12).
[324] IonQ, “Algorithmic Qubits: A Better Single-Number Metric”, IonQ Website, [Accessed: 1-November-2023] (2022) (Cited on pages 12,
13, 75).
[325] J.-S. Chen, E. Nielsen, M. Ebert, V. Inlek, K. Wright, V. Chaplin, A. Maksymov, E. Páez, A. Poudel, P. Maunz, et al., “Bench-
marking a trapped-ion quantum computer with 29 algorithmic qubits”, arXiv:2308.05071 (2023) (Cited on page 12).
[326] S. Boixo, S. V. Isakov, V. N. Smelyanskiy, R. Babbush, N. Ding, Z. Jiang, M. J. Bremner, J. M. Martinis, and H. Neven, “Char-
acterizing quantum supremacy in near-term devices”, Nat. Phys. 14, 595 (2018) (Cited on page 12).
[327] N. P. Sawaya et al., “Hamlib: a library of hamiltonians for benchmarking quantum algorithms and hardware”, 2023 IEEE Int Conf
Quantum Comput Eng QCE, Vol. 02 (2023), 389–390 (Cited on page 13).
[328] P. W. Shor, “Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer”, SIAM J.
Comput. 26, 1484 (1997) (Cited on pages 13, 23, 50, 86).
[329] L. K. Grover, “A Fast Quantum Mechanical Algorithm for Database Search”, Proc 28th Annu ACM Symp Theory Comput (1996),
212 (Cited on page 13).

116
References

[330] J. Tilly et al., “The Variational Quantum Eigensolver: A review of methods and best practices”, Phys. Rep. 986, 1 (2022) (Cited
on pages 13, 15, 23, 78, 79, 81).
[331] E. Bernstein and U. Vazirani, “Quantum Complexity Theory”, SIAM J. Comput. 26, 1411 (1997) (Cited on page 13).
[332] W. van Dam, S. Hallgren, and L. Ip, “Quantum Algorithms for Some Hidden Shift Problems”, SIAM J. Comput. 36, 763 (2006)
(Cited on page 13).
[333] J. R. Finžgar, P. Ross, J. Klepsch, and A. Luckow, “QUARK: A Framework for Quantum Computing Application Benchmarking”,
2022 IEEE Int Conf Quantum Comput Eng QCE (2022), 226 (Cited on page 13).
[334] A. A. Zhukov, E. O. Kiktenko, A. A. Elistratov, W. V. Pogosov, and Y. E. Lozovik, “Quantum Communication Protocols as a
Benchmark for Quantum Computers”, Quantum Inf. Process. 18, 31 (2019) (Cited on page 13).
[335] Y. Dong and L. Lin, “Random Circuit Block-Encoded Matrix and a Proposal of Quantum LINPACK Benchmark”, Phys. Rev. A
103, 062412 (2021) (Cited on page 13).
[336] K. Michielsen, M. Nocon, D. Willsch, F. Jin, T. Lippert, and H. De Raedt, “Benchmarking Gate-Based Quantum Computers”,
Comput. Phys. Commun. 220, 44 (2017) (Cited on page 13).
[337] E. Knill, R. Laflamme, R. Martinez, and C.-H. Tseng, “An Algorithmic Benchmark for Quantum Information Processing”, Nature
404, 368 (2000) (Cited on page 13).
[338] A. Cornelissen, J. Bausch, and A. Gilyén, “Scalable Benchmarks for Gate-Based Quantum Computers”, arXiv:2104.10698 (2021)
(Cited on page 13).
[339] A. Peruzzo, J. McClean, P. Shadbolt, M.-H. Yung, X.-Q. Zhou, P. J. Love, A. Aspuru-Guzik, and J. L. O’brien, “A variational
eigenvalue solver on a photonic quantum processor”, Nat. Commun. 5, 4213 (2014) (Cited on page 13).
[340] A. Kandala, A. Mezzacapo, K. Temme, M. Takita, M. Brink, J. M. Chow, and J. M. Gambetta, “Hardware-efficient variational
quantum eigensolver for small molecules and quantum magnets”, Nature 549, 242 (2017) (Cited on page 13).
[341] P. J. J. O’Malley et al., “Scalable quantum simulation of molecular energies”, Phys. Rev. X 6, 031007 (2016) (Cited on page 13).
[342] Y. Nam, J.-S. Chen, N. C. Pisenti, K. Wright, C. Delaney, D. Maslov, K. R. Brown, S. Allen, J. M. Amini, J. Apisdorf, et al.,
“Ground-state energy estimation of the water molecule on a trapped-ion quantum computer”, npj Quantum Inf. 6, 33 (2020) (Cited
on page 13).
[343] C. Hempel et al., “Quantum chemistry calculations on a trapped-ion quantum simulator”, Phys. Rev. X 8, 031022 (2018) (Cited
on page 13).
[344] Y. Shen, X. Zhang, S. Zhang, J.-N. Zhang, M.-H. Yung, and K. Kim, “Quantum implementation of the unitary coupled cluster for
simulating molecular electronic structure”, Phys. Rev. A 95, 020501 (2017) (Cited on page 13).
[345] E. Farhi, J. Goldstone, and S. Gutmann, “A quantum approximate optimization algorithm”, arXiv:1411.4028 (2014) (Cited on
pages 13, 81, 82).
[346] L. Zhou, S.-T. Wang, S. Choi, H. Pichler, and M. D. Lukin, “Quantum approximate optimization algorithm: performance, mecha-
nism, and implementation on near-term devices”, Phys. Rev. X 10, 021067 (2020) (Cited on page 13).
[347] K. Blekos, D. Brand, A. Ceschini, C.-H. Chou, R.-H. Li, K. Pandya, and A. Summer, “A review on quantum approximate opti-
mization algorithm and its variants”, Phys. Rep. 1068, 1 (2024) (Cited on page 13).
[348] L. Zhou, S.-T. Wang, S. Choi, H. Pichler, and M. D. Lukin, “Quantum Approximate Optimization Algorithm: Performance,
Mechanism, and Implementation on Near-Term Devices”, Phys. Rev. X 10, 021067 (2020) (Cited on pages 13, 81, 82).
[349] N. Chancellor, “Domain Wall Encoding of Discrete Variables for Quantum Annealing and QAOA”, Quantum Sci. Technol. 4, 045004
(2019) (Cited on pages 13, 96).
[350] K. Mesman, Z. Al-Ars, and M. Möller, “Qpack: Quantum approximate optimization algorithms as universal benchmark for quantum
computers”, arXiv:2103.17193 (2021) (Cited on page 14).
[351] K. Sankar, A. Scherer, S. Kako, S. Reifenstein, N. Ghadermarzy, W. B. Krayenhoff, Y. Inui, E. Ng, T. Onodera, P. Ronagh,
et al., “A benchmarking study of quantum algorithms for combinatorial optimization”, Npj Quantum Inf. 10, 64 (2024) (Cited on
page 14).
[352] S. Dasgupta and T. S. Humble, “Stability of noisy quantum computing devices”, arXiv:2105.09472 (2021) (Cited on page 15).
[353] T. Proctor, M. Revelle, E. Nielsen, K. Rudinger, D. Lobser, P. Maunz, R. Blume-Kohout, and K. Young, “Detecting and tracking
drift in quantum information processors”, Nat. Commun. 11, 5396 (2020) (Cited on page 15).
[354] S. Dasgupta and T. S. Humble, “Characterizing the Stability of NISQ Devices”, 2020 IEEE Int Conf Quantum Comput Eng QCE
(2020), 419 (Cited on page 15).
[355] K. Yeter-Aydeniz, Z. Parks, A. Nair, E. Gustafson, A. F. Kemper, R. C. Pooser, Y. Meurice, and P. Dreher, “Measuring NISQ
gate-based qubit stability using a 1+ 1 field theory and cycle benchmarking”, arXiv:2201.02899 (2022) (Cited on page 15).
[356] W. van der Schoot, R. Wezeman, P. T. Eendebak, N. M. Neumann, and F. Phillipson, “Evaluating three levels of quantum metrics
on quantum-inspire hardware”, Quantum Inf. Process. 22, 451 (2023) (Cited on page 15).
[357] S. Dasgupta, “Stability of Quantum Computers”, arXiv:2404.19082 (2024) (Cited on page 15).
[358] M. Alam, A. Ash-Saki, and S. Ghosh, “Addressing Temporal Variations in Qubit Quality Metrics for Parameterized Quantum
Circuits”, 2019 IEEEACM Int Symp Low Power Electron ISLPED (2019), 1 (Cited on page 15).
[359] C. Müller, J. Lisenfeld, A. Shnirman, and S. Poletto, “Interacting two-level defects as sources of fluctuating high-frequency noise
in superconducting circuits”, Phys. Rev. B 92, 035442 (2015) (Cited on page 15).
[360] Y. Wan, D. Kienzler, S. D. Erickson, K. H. Mayer, T. R. Tan, J. J. Wu, H. M. Vasconcelos, S. Glancy, E. Knill, D. J. Wineland,
et al., “Quantum gate teleportation between separated qubits in a trapped-ion processor”, Science 364, 875 (2019) (Cited on
page 15).
[361] B. Merkel, K. Thirumalai, J. E. Tarlton, V. M. Schäfer, C. J. Ballance, T. P. Harty, and D. M. Lucas, “Magnetic field stabilization
system for atomic physics experiments”, Rev. Sci. Instrum. 90 (2019) (Cited on page 15).
[362] S. Schlör, J. Lisenfeld, C. Müller, A. Bilmes, A. Schneider, D. P. Pappas, A. V. Ustinov, and M. Weides, “Correlating Decoherence
in Transmon Qubits: Low Frequency Noise by Single Fluctuators”, Phys. Rev. Lett. 123, 190502 (2019) (Cited on page 15).
[363] S. E. de Graaf, S. Mahashabde, S. E. Kubatkin, A. Ya. Tzalenchuk, and A. V. Danilov, “Quantifying dynamics and interactions of
individual spurious low-energy fluctuators in superconducting circuits”, Phys. Rev. B 103, 174103 (2021) (Cited on page 15).
[364] W. Wei, P. Hao, Z. Ma, H. Zhang, L. Pang, F. Wu, K. Deng, J. Zhang, and Z. Lu, “Measurement and suppression of magnetic field
noise of trapped ion qubit”, J. Phys. B: At. Mol. Opt. Phys. 55, 075001 (2022) (Cited on page 15).
[365] L. Casparis, T. W. Larsen, M. S. Olsen, F. Kuemmeth, P. Krogstrup, J. Nygård, K. D. Petersson, and C. M. Marcus, “Gatemon
Benchmarking and Two-Qubit Operations”, Phys. Rev. Lett. 116, 150505 (2016) (Cited on page 15).
[366] C. H. Yang, K. W. Chan, R. Harper, W. Huang, T. Evans, J. C. C. Hwang, B. Hensen, A. Laucht, T. Tanttu, F. E. Hudson, et al.,
“Silicon qubit fidelities approaching incoherent noise limits via pulse engineering”, Nat. Electron. 2, 151 (2019) (Cited on pages 15,
33).
[367] T. Ruster, C. T. Schmiegelow, H. Kaufmann, C. Warschburger, F. Schmidt-Kaler, and U. G. Poschinger, “A long-lived Zeeman
trapped-ion qubit”, Appl. Phys. B 122, 254 (2016) (Cited on page 15).
[368] S. Schneider and G. J. Milburn, “Decoherence and fidelity in ion traps with fluctuating trap parameters”, Phys. Rev. A 59, 3766
(1999) (Cited on page 15).
[369] D. Aharonov, W. van Dam, J. Kempe, Z. Landau, S. Lloyd, and O. Regev, “Adiabatic Quantum Computation Is Equivalent to
Standard Quantum Computation”, 45th Annu. IEEE Symp. Found. Comput. Sci., 42 (2004) (Cited on page 16).
[370] T. Lanting et al., “Entanglement in a Quantum Annealing Processor”, Phys. Rev. X 4, 021041 (2014) (Cited on page 16).
[371] S. Boixo, V. N. Smelyanskiy, A. Shabani, S. V. Isakov, M. Dykman, V. S. Denchev, M. H. Amin, A. Y. Smirnov, M. Mohseni, and
H. Neven, “Computational Multiqubit Tunnelling in Programmable Quantum Annealers”, Nat. Commun. 7, 10327 (2016) (Cited
on page 16).
[372] N. Chancellor and V. Kendon, “Experimental Test of Search Range in Quantum Annealing”, Phys. Rev. A 104, 012604 (2021)
(Cited on page 16).
[373] W. van der Schoot, D. Leermakers, R. Wezeman, N. Neumann, and F. Phillipson, “Evaluating the Q-score of Quantum Annealers”,
2022 IEEE Int. Conf. Quantum Softw. QSW, 9 (2022) (Cited on pages 16, 83).

117
References

[374] W. van der Schoot, R. Wezeman, N. M. P. Neumann, F. Phillipson, and R. Kooij, “Q-Score Max-Clique: The First Quantum Metric
Evaluation on Multiple Computational Paradigms”, arXiv:2302.00639 (2023) (Cited on page 16).
[375] E. Derbyshire, J. Y. Malo, A. Daley, E. Kashefi, and P. Wallden, “Randomized Benchmarking in the Analogue Setting”, Quantum
Sci. Technol. 5, 034001 (2020) (Cited on page 16).
[376] C. C. McGeoch, “Principles and Guidelines for Quantum Performance Analysis”, Quantum Technol. Optim. Probl., 36 (2019)
(Cited on page 16).
[377] Y. Zhang, D. Niu, A. Shabani, and H. Shapourian, “Quantum volume for photonic quantum processors”, Phys. Rev. Lett. 130,
110602 (2023) (Cited on page 17).
[378] M. Saffman, T. G. Walker, and K. Mølmer, “Quantum information with Rydberg atoms”, Rev. Mod. Phys. 82, 2313 (2010) (Cited
on pages 17, 31).
[379] M. Morgado and S. Whitlock, “Quantum Simulation and Computing with Rydberg-interacting Qubits”, AVS Quantum Sci. 3,
023501 (2021) (Cited on pages 17, 31).
[380] A. Browaeys and T. Lahaye, “Many-Body Physics with Individually Controlled Rydberg Atoms”, Nat. Phys 16, 132 (2020) (Cited
on pages 17, 31).
[381] G. Semeghini et al., “Probing Topological Spin Liquids on a Programmable Quantum Simulator”, Science 374, 1242 (2021) (Cited
on pages 17, 31).
[382] D. K. Mark, J. Choi, A. L. Shaw, M. Endres, and S. Choi, “Benchmarking quantum simulators using ergodic quantum dynamics”,
Phys. Rev. Lett. 131, 110601 (2023) (Cited on pages 17, 106, 107).
[383] K.-N. Schymik, S. Pancaldi, F. Nogrette, D. Barredo, J. Paris, A. Browaeys, and T. Lahaye, “Single Atoms with 6000-Second
Trapping Lifetimes in Optical-Tweezer Arrays at Cryogenic Temperatures”, Phys. Rev. Appl. 16, 034013 (2021) (Cited on pages 17,
31, 107).
[384] H. Buhrman and H. Röhrig, “Distributed quantum computing”, Int. Symp. Math. Found. Comput. Sci., 1 (2003) (Cited on page 18).
[385] D. Cuomo, M. Caleffi, and A. S. Cacciapuoti, “Towards a distributed quantum computing ecosystem”, IET Quantum Commun. 1,
3 (2020) (Cited on page 18).
[386] M. Caleffi, M. Amoretti, D. Ferrari, J. Illiano, A. Manzalini, and A. S. Cacciapuoti, “Distributed quantum computing: a survey”,
Comput. Netw. 254, 110672 (2024) (Cited on pages 18, 19).
[387] A. S. Cacciapuoti, M. Caleffi, F. Tafuri, F. S. Cataliotti, S. Gherardini, and G. Bianchi, “Quantum internet: networking challenges
in distributed quantum computing”, IEEE Netw. 34, 137 (2019) (Cited on page 18).
[388] R. Beals, S. Brierley, O. Gray, A. W. Harrow, S. Kutin, N. Linden, D. Shepherd, and M. Stather, “Efficient distributed quantum
computing”, Proc. R. Soc. Lond. A 469, 20120686 (2013) (Cited on page 18).
[389] R. Van Meter and S. J. Devitt, “The path to scalable distributed quantum computing”, Computer 49, 31 (2016) (Cited on page 18).
[390] J. I. Cirac, A. K. Ekert, S. F. Huelga, and C. Macchiavello, “Distributed quantum computation over noisy channels”, Phys. Rev.
A 59, 4249 (1999) (Cited on page 18).
[391] A. Dahlberg, M. Skrzypczyk, T. Coopmans, L. Wubben, F. Rozpędek, M. Pompili, A. Stolk, P. Pawełczak, R. Knegjens, J. de
Oliveira Filho, et al., “A link layer protocol for quantum networks”, Proc. ACM spec. Interest Group Data Commun., 159 (2019)
(Cited on page 19).
[392] L. J. Stephenson, D. P. Nadlinger, B. C. Nichol, S. An, P. Drmota, T. G. Ballance, K. Thirumalai, J. F. Goodwin, D. M. Lucas,
and C. J. Ballance, “High-rate, high-fidelity entanglement of qubits across an elementary quantum network”, Phys. Rev. Lett. 124,
110501 (2020) (Cited on page 19).
[393] F. Dupuy, C. Goursaud, and F. Guillemin, “A survey of quantum entanglement routing protocols—challenges for wide-area net-
works”, Adv. Quantum Technol. 6, 2200180 (2023) (Cited on page 19).
[394] S. Shi and C. Qian, “Concurrent entanglement routing for quantum networks: model and designs”, Proc. Annu. Conf. ACM Spec.
Interest Group Data Commun. Appl. Technol. Archit. Protoc. Comput. Commun., 62 (2020) (Cited on page 19).
[395] G. Vardoyan and S. Wehner, “Quantum network utility maximization”, 2023 IEEE Int Conf Quantum Comput Eng QCE 1, 1238
(2023) (Cited on page 19).
[396] Y. Lee, W. Dai, D. Towsley, and D. Englund, “Quantum network utility: a framework for benchmarking quantum networks”, Proc.
Natl. Acad. Sci. 121, e2314103121 (2024) (Cited on page 19).
[397] D. Ferrari and M. Amoretti, “A design framework for the simulation of distributed quantum computing”, Proc. 2024 Workshop
High Perform. Quantum Comput. Integr., 4 (2024) (Cited on page 19).
[398] J. Frank, E. Kashefi, D. Leichtle, and M. de Oliveira, “Heuristic-free verification-inspired quantum benchmarking”, arXiv:2404.10739
(2024) (Cited on page 19).
[399] D. Deutsch, “Quantum Theory, the Church–Turing Principle and the Universal Quantum Computer”, Proc. R. Soc. Lond. Math.
Phys. Sci. 400, 97 (1985) (Cited on page 21).
[400] D. Deutsch, “Quantum Computational Networks”, Proc. R. Soc. Lond. Math. Phys. Sci. 425, 73 (1989) (Cited on pages 21, 23).
[401] B. Lekitsch, S. Weidt, A. G. Fowler, K. Mølmer, S. J. Devitt, C. Wunderlich, and W. K. Hensinger, “Blueprint for a microwave
trapped ion quantum computer”, Sci. Adv. 3, e1601540 (2017) (Cited on pages 21, 31).
[402] Q. Liang, M. Kang, M. Li, and Y. Nam, “Pulse optimization for high-precision motional-mode characterization in trapped-ion
quantum computers”, Quantum Sci. Technol. 9, 035007 (2024) (Cited on page 21).
[403] R. Konik, “Quantum coherence confined”, Nat. Phys. 17, 669 (2021) (Cited on page 21).
[404] C. Rigetti et al., “Superconducting qubit in a waveguide cavity with a coherence time approaching 0.1 ms”, Phys. Rev. B 86,
100506 (2012) (Cited on page 21).
[405] P. Shor, “Fault-Tolerant Quantum Computation”, Proc. 37th Conf. Found. Comput. Sci. (1996) (Cited on page 22).
[406] E. Knill, R. Laflamme, and W. H. Zurek, “Resilient quantum computation: error models and thresholds”, Proc. R. Soc. Lond. A
454, 365 (1998) (Cited on page 22).
[407] A. Kitaev, “Fault-tolerant quantum computation by anyons”, Ann. Phys. 303, 2–30 (2003) (Cited on page 22).
[408] A. M. Steane, “Overhead and noise threshold of fault-tolerant quantum error correction”, Phys. Rev. A 68, 042322 (2003) (Cited
on page 22).
[409] A. M. Stephens, “Fault-tolerant thresholds for quantum error correction with the surface code”, Phys. Rev. A 89, 022321 (2014)
(Cited on page 22).
[410] A. W. Cross, D. P. Divincenzo, and B. M. Terhal, “A comparative code study for quantum fault tolerance”, Quantum Info. Comput.
9, 541–572 (2009) (Cited on page 22).
[411] S. Bravyi, A. W. Cross, J. M. Gambetta, D. Maslov, P. Rall, and T. J. Yoder, “High-threshold and low-overhead fault-tolerant
quantum memory”, Nature 627, 778 (2024) (Cited on page 22).
[412] H.-S. Zhong et al., “Phase-Programmable Gaussian Boson Sampling Using Stimulated Squeezed Light”, Phys. Rev. Lett. 127,
180502 (2021) (Cited on pages 22, 27, 103).
[413] E. Pednault, J. A. Gunnels, G. Nannicini, L. Horesh, and R. Wisnieff, “Leveraging secondary storage to simulate deep 54-qubit
sycamore circuits”, arXiv:1910.09534 (2019) (Cited on page 22).
[414] Y. Liu, X. Liu, F. Li, H. Fu, Y. Yang, J. Song, P. Zhao, Z. Wang, D. Peng, H. Chen, et al., “Closing the" quantum supremacy" gap:
achieving real-time simulation of a random quantum circuit using a new sunway supercomputer”, Proc. int. conf. high perform.
comput. netw., storage, anal. (2021), 1 (Cited on page 22).
[415] Y. Zhou, E. M. Stoudenmire, and X. Waintal, “What limits the simulation of quantum computers?”, Phys. Rev. X 10, 041038
(2020) (Cited on page 22).
[416] A. Zlokapa, B. Villalonga, S. Boixo, and D. A. Lidar, “Boundaries of quantum supremacy via random circuit sampling”, npj
Quantum Inf. 9, 36 (2023) (Cited on page 22).
[417] Google Quantum AI and Collaborators, “Phase transitions in random circuit sampling”, Nature 634, 328–333 (2024) (Cited on
page 22).
[418] A. Barenco, C. H. Bennett, R. Cleve, D. P. DiVincenzo, N. Margolus, P. Shor, T. Sleator, J. A. Smolin, and H. Weinfurter,
“Elementary gates for quantum computation”, Phys. Rev. A 52, 3457 (1995) (Cited on page 23).

118
References

[419] P. Boykin, T. Mor, M. Pulver, V. Roychowdhury, and F. Vatan, “A new universal and fault-tolerant quantum basis”, Inf. Process.
Lett. 75, 101 (2000) (Cited on page 23).
[420] R. Raussendorf and H. J. Briegel, “A One-Way Quantum Computer”, Phys. Rev. Lett. 86, 5188 (2001) (Cited on page 23).
[421] T. Rudolph, “Why I Am Optimistic about the Silicon-Photonic Route to Quantum Computing”, APL photonics 2, 030901 (2017)
(Cited on page 24).
[422] A. Broadbent, J. Fitzsimons, and E. Kashefi, “Universal Blind Quantum Computation”, 2009 50th Annu. IEEE Symp. Found.
Comput. Sci., 517 (2009) (Cited on page 24).
[423] S. Barz, E. Kashefi, A. Broadbent, J. F. Fitzsimons, A. Zeilinger, and P. Walther, “Demonstration of Blind Quantum Computing”,
Science 335, 303 (2012) (Cited on page 24).
[424] R. Raussendorf, “Contextuality in Measurement-Based Quantum Computation”, Phys. Rev. A 88, 022322 (2013) (Cited on page 24).
[425] T. Kadowaki and H. Nishimori, “Quantum Annealing in the Transverse Ising Model”, Phys. Rev. E 58, 5355 (1998) (Cited on
page 24).
[426] A. D. King, J. Raymond, T. Lanting, R. Harris, A. Zucca, F. Altomare, A. J. Berkley, K. Boothby, S. Ejtemaee, C. Enderud, et al.,
“Quantum critical dynamics in a 5,000-qubit programmable spin glass”, Nature 617, 61 (2023) (Cited on page 24).
[427] H. M. Bauza and D. A. Lidar, “Scaling advantage in approximate optimization with quantum annealing”, arXiv:2401.07184 (2024)
(Cited on page 24).
[428] A. D. King et al., “Scaling Advantage over Path-Integral Monte Carlo in Quantum Simulation of Geometrically Frustrated Magnets”,
Nat. Commun. 12, 1113 (2021) (Cited on page 24).
[429] T. Albash and D. A. Lidar, “Adiabatic Quantum Computation”, Rev. Mod. Phys. 90, 015002 (2018) (Cited on pages 24, 25).
[430] E. J. Crosson and D. A. Lidar, “Prospects for Quantum Enhancement with Diabatic Quantum Annealing”, Nat. Rev. Phys. 3, 466
(2021) (Cited on pages 24, 25).
[431] A. Callison and N. Chancellor, “Hybrid Quantum-Classical Algorithms in the Noisy Intermediate-Scale Quantum Era and Beyond”,
Phys. Rev. A 106, 010101 (2022) (Cited on page 24).
[432] S. Yarkoni, E. Raponi, T. Bäck, and S. Schmitt, “Quantum annealing for industry applications: introduction and review”, Rep.
Prog. Phys. 85, 104001 (2022) (Cited on page 24).
[433] J. J. Berwald, “The Mathematics of Quantum-Enabled Applications on the D-Wave Quantum Computer”, Not. Am. Math. Soc.
66, 832 (2019) (Cited on page 24).
[434] F. Barahona, “On the computational complexity of ising spin glass models”, J. Phys. A: Math. Gen. 15, 3241 (1982) (Cited on
page 24).
[435] S. A. Cook, “The complexity of theorem-proving procedures”, Proceedings of the third annual acm symposium on theory of
computing, STOC ’71 (1971), 151–158 (Cited on page 24).
[436] R. M. Karp, “Reducibility among combinatorial problems”, Complexity of computer computations, Proceedings of a symposium
on the Complexity of Computer Computations (1972), 85 (Cited on page 24).
[437] E. Farhi, J. Goldstone, S. Gutmann, and M. Sipser, “Quantum Computation by Adiabatic Evolution”, arXiv:0001106 (2000) (Cited
on page 25).
[438] A. Callison, M. Festenstein, J. Chen, L. Nita, V. Kendon, and N. Chancellor, “Energetic Perspective on Rapid Quenches in Quantum
Annealing”, PRX Quantum 2, 010338 (2021) (Cited on page 25).
[439] R. J. Banks, G. S. Raftis, D. E. Browne, and P. Warburton, “Continuous-time quantum optimisation without the adiabatic prin-
ciple”, arXiv:2407.03910 (2024) (Cited on page 25).
[440] S. Schulz, D. Willsch, and K. Michielsen, “Guided quantum walk”, Phys. Rev. Res. 6, 013312 (2024) (Cited on page 25).
[441] A. Callison, N. Chancellor, F. Mintert, and V. Kendon, “Finding spin glass ground states using quantum walks”, New J. Phys. 21,
123022 (2019) (Cited on page 25).
[442] J. Kempe, A. Kitaev, and O. Regev, “The Complexity of the Local Hamiltonian Problem”, SIAM J. Comput. 35, 1070 (2006)
(Cited on page 25).
[443] R. Oliveira and B. M. Terhal, “The Complexity of Quantum Spin Systems on a Two-Dimensional Square Lattice”, Quantum Info.
Comput. 8, 900 (2008) (Cited on page 25).
[444] J. D. Biamonte and P. J. Love, “Realizable Hamiltonians for Universal Adiabatic Quantum Computers”, Phys. Rev. A 78, 012352
(2008) (Cited on page 25).
[445] T. Inagaki et al., “A Coherent Ising Machine for 2000-Node Optimization Problems”, Science 354, 603 (2016) (Cited on page 25).
[446] P. L. McMahon et al., “A Fully Programmable 100-Spin Coherent Ising Machine with All-to-All Connections”, Science 354, 614
(2016) (Cited on page 25).
[447] L. Nguyen, M.-A. Miri, R. J. Rupert, W. Dyk, S. Wu, N. Vrahoretis, I. Huang, M. Begliarbekov, N. Chancellor, U. Chukwu, et al.,
“Entropy computing: a paradigm for optimization in an open quantum system”, arXiv:2407.04512 (2024) (Cited on page 25).
[448] C. Shen, Z. Zhang, and L.-M. Duan, “Scalable Implementation of Boson Sampling with Trapped Ions”, Phys. Rev. Lett. 112,
050504 (2014) (Cited on page 26).
[449] B. Peropadre, G. G. Guerreschi, J. Huh, and A. Aspuru-Guzik, “Proposal for Microwave Boson Sampling”, Phys. Rev. Lett. 117,
140505 (2016) (Cited on page 26).
[450] C. Weedbrook, S. Pirandola, R. García-Patrón, N. J. Cerf, T. C. Ralph, J. H. Shapiro, and S. Lloyd, “Gaussian Quantum Infor-
mation”, Rev. Mod. Phys. 84, 621 (2012) (Cited on pages 26, 32).
[451] P. Clifford and R. Clifford, “The Classical Complexity of Boson Sampling”, Proc. 29th Annu. ACM-SIAM Symp. Discrete Algorithms
(2017), 146 (Cited on page 27).
[452] A. W. Harrow and A. Montanaro, “Quantum Computational Supremacy”, Nature 549, 203 (2017) (Cited on page 27).
[453] D. Stilck França and R. Garcia-Patron, “A Game of Quantum Advantage: Linking Verification and Simulation”, Quantum 6, 753
(2022) (Cited on pages 27, 105).
[454] H.-S. Zhong et al., “Quantum Computational Advantage Using Photons”, Science 370, 1460 (2020) (Cited on pages 27, 103).
[455] B. Villalonga, M. Y. Niu, L. Li, H. Neven, J. C. Platt, V. N. Smelyanskiy, and S. Boixo, “Efficient approximation of experimental
Gaussian boson sampling”, arXiv:2109.11525 (2022) (Cited on pages 27, 104).
[456] C. Oh, M. Liu, Y. Alexeev, B. Fefferman, and L. Jiang, “Classical algorithm for simulating experimental gaussian boson sampling”,
Nat. Phys., 1 (2024) (Cited on page 27).
[457] M. Lubasch, A. A. Valido, J. J. Renema, W. S. Kolthammer, D. Jaksch, M. S. Kim, I. Walmsley, and R. García-Patrón, “Tensor
Network States in Time-Bin Quantum Optics”, Phys. Rev. A 97, 062304 (2018) (Cited on page 27).
[458] A. E. Moylett, R. l García-Patrón, J. J. Renema, and P. S. Turner, “Classically Simulating Near-Term Partially-Distinguishable
and Lossy Boson Sampling”, Quantum Sci. Technol. 5, 015001 (2019) (Cited on page 28).
[459] T. R. Bromley, J. M. Arrazola, S. Jahangiri, J. Izaac, N. s Quesada, A. D. Gran, M. Schuld, J. Swinarton, Z. Zabaneh, and N.
Killoran, “Applications of Near-Term Photonic Quantum Computers: Software and Algorithms”, Quantum Sci. Technol. 5, 034010
(2020) (Cited on page 28).
[460] J. Huh, G. G. Guerreschi, B. Peropadre, J. R. McClean, and A. Aspuru-Guzik, “Boson Sampling for Molecular Vibronic Spectra”,
Nat. Photonics 9, 615 (2015) (Cited on page 28).
[461] F. Santoro, R. Improta, A. Lami, J. Bloino, and V. Barone, “Effective method to compute Franck-Condon integrals for optical
spectra of large molecules in solution”, J. Chem. Phys 126, 084509 (2007) (Cited on page 28).
[462] J. M. Arrazola and T. R. Bromley, “Using Gaussian Boson Sampling to Find Dense Subgraphs”, Phys. Rev. Lett. 121, 030503
(2018) (Cited on page 28).
[463] L. Banchi, M. Fingerhuth, T. Babej, C. Ing, and J. M. Arrazola, “Molecular docking with Gaussian Boson Sampling”, Sci. Adv. 6,
eaax1950 (2020) (Cited on page 28).
[464] S. E. de Graaf, S. Un, A. G. Shard, and T. Lindström, “Chemical and Structural Identification of Material Defects in Supercon-
ducting Quantum Circuits”, Mater. Quantum Technol. 2, 032001 (2022) (Cited on page 29).
[465] A. Somoroff, Q. Ficheux, R. A. Mencia, H. Xiong, R. Kuzmin, and V. E. Manucharyan, “Millisecond Coherence in a Superconducting
Qubit”, Phys. Rev. Lett. 130, 267001 (2023) (Cited on page 29).

119
References

[466] S. Krinner, S. Storz, P. Kurpiers, P. Magnard, J. Heinsoo, R. Keller, J. Luetolf, C. Eichler, and A. Wallraff, “Engineering cryogenic
setups for 100-qubit scale superconducting circuit systems”, EPJ Quantum Technol. 6, 2 (2019) (Cited on page 29).
[467] D.-R. W. Yost, M. E. Schwartz, J. Mallek, D. Rosenberg, C. Stull, J. L. Yoder, G. Calusine, M. Cook, R. Das, A. L. Day, et al.,
“Solid-state qubits integrated with superconducting through-silicon vias”, npj Quantum Inf. 6, 59 (2020) (Cited on page 29).
[468] D. Rosenberg, S. Weber, D. Conway, D. Yost, J. Mallek, G. Calusine, R. Das, D. Kim, M. Schwartz, W. Woods, et al., “3d
integration and packaging for solid-state qubits”, arXiv:1906.11146 (2019) (Cited on page 29).
[469] B. Foxen, J. Mutus, E. Lucero, R. Graff, A. Megrant, Y. Chen, C. Quintana, B. Burkett, J. Kelly, E. Jeffrey, et al., “Qubit
compatible superconducting interconnects”, Quantum Sci. Technol. 3, 014005 (2017) (Cited on page 29).
[470] Z. Chen, A. Megrant, J. Kelly, R. Barends, J. Bochmann, Y. Chen, B. Chiaro, A. Dunsworth, E. Jeffrey, J. Mutus, et al., “Fabrication
and characterization of aluminum airbridges for superconducting microwave circuits”, Appl. Phys. Lett. 104 (2014) (Cited on
page 29).
[471] D. Rosenberg, D. Kim, R. Das, D. Yost, S. Gustavsson, D. Hover, P. Krantz, A. Melville, L. Racz, G. Samach, et al., “3d integrated
superconducting qubits”, npj Quantum Inf. 3, 42 (2017) (Cited on page 29).
[472] Y. Sung et al., “Realization of High-Fidelity Cz and z z-Free Iswap Gates with a Tunable Coupler”, Phys. Rev. X 11, 021058 (2021)
(Cited on page 29).
[473] H. Levine et al., “Demonstrating a long-coherence dual-rail erasure qubit using tunable transmons”, Phys. Rev. X 14, 011051
(2024) (Cited on page 29).
[474] D. J. Wineland, C. Monroe, W. M. Itano, D. Leibfried, B. E. King, and D. M. Meekhof, “Experimental issues in coherent quantum-
state manipulation of trapped atomic ions”, J. Res. Natl. Inst. Stand. Tech. 103, 259 (1998) (Cited on page 29).
[475] J. G. Bohnet, B. C. Sawyer, J. W. Britton, M. L. Wall, A. M. Rey, M. Foss-Feig, and J. J. Bollinger, “Quantum spin dynamics and
entanglement generation with hundreds of trapped ions”, Science 352, 1297 (2016) (Cited on page 29).
[476] S. Jain, J. Alonso, M. Grau, and J. P. Home, “Scalable Arrays of Micro-Penning Traps for Quantum Computing and Simulation”,
Phys. Rev. X 10, 031027 (2020) (Cited on page 29).
[477] M. A. Rowe et al., “Transport of quantum states and separation of ions in a dual RF ion trap”, Quantum Info. Comput. 2, 257
(2002) (Cited on page 30).
[478] G. Wilpers, P. See, P. Gill, and A. G. Sinclair, “A monolithic array of three-dimensional ion traps fabricated with conventional
semiconductor technology”, Nat. Nanotechnol. 7, 572 (2012) (Cited on page 30).
[479] J. Chiaverini, R. B. Blakestad, J. Britton, J. D. Jost, C. Langer, D. Leibfried, R. Ozeri, and D. J. Wineland, “Surface-electrode
architecture for ion-trap quantum information processing”, Quantum Info. Comput. 5, 419 (2005) (Cited on page 30).
[480] F. Mintert and C. Wunderlich, “Ion-Trap Quantum Logic Using Long-Wavelength Radiation”, Phys. Rev. Lett. 87, 257904 (2001)
(Cited on page 30).
[481] D. T. C. Allcock, W. C. Campbell, J. Chiaverini, I. L. Chuang, E. R. Hudson, I. D. Moore, A. Ransford, C. Roman, J. M. Sage,
and D. J. Wineland, “Omg blueprint for trapped ion quantum computing with metastable states”, Appl. Phys. Lett. 119, 214002
(2021) (Cited on page 30).
[482] A. Sørensen and K. Mølmer, “Entanglement and quantum computation with ions in thermal motion”, Phys. Rev. A 62, 022311
(2000) (Cited on pages 30, 49).
[483] D. Leibfried et al., “Experimental demonstration of a robust, high-fidelity geometric two ion-qubit phase gate”, Nature 422, 412
(2003) (Cited on page 30).
+
[484] J. P. Gaebler et al., “High-Fidelity Universal Gate Set for 9 Be Ion Qubits”, Phys. Rev. Lett. 117, 060505 (2016) (Cited on
page 30).
[485] C. Ryan-Anderson et al., “Implementing fault-tolerant entangling gates on the five-qubit code and the color code”, arXiv:2208.01863
(2022) (Cited on pages 30, 50).
[486] C. Monroe, R. Raussendorf, A. Ruthven, K. R. Brown, P. Maunz, L.-M. Duan, and J. Kim, “Large-scale modular quantum-computer
architecture with atomic memory and photonic interconnects”, Phys. Rev. A 89, 022317 (2014) (Cited on page 31).
[487] M. Endres, H. Bernien, A. Keesling, H. Levine, E. R. Anschuetz, A. Krajenbrink, C. Senko, V. Vuletic, M. Greiner, and M. D.
Lukin, “Atom-by-atom assembly of defect-free one-dimensional cold atom arrays”, Science 354, 1024 (2016) (Cited on page 31).
[488] D. Barredo, S. de Léséleuc, V. Lienhard, T. Lahaye, and A. Browaeys, “An atom-by-atom assembler of defect-free arbitrary two-
dimensional atomic arrays”, Science 354, 1021 (2016) (Cited on page 31).
[489] D. Barredo, V. Lienhard, S. de Léséleuc, T. Lahaye, and A. Browaeys, “Synthetic Three-Dimensional Atomic Structures Assembled
Atom by Atom”, Nature 561, 79 (2018) (Cited on page 31).
[490] P. Huft, Y. Song, T. M. Graham, K. Jooya, S. Deshpande, C. Fang, M. Kats, and M. Saffman, “Simple, passive design for large
optical trap arrays for single atoms”, Phys. Rev. A 105, 063111 (2022) (Cited on page 31).
[491] K. Barnes et al., “Assembly and Coherent Control of a Register of Nuclear Spin Qubits”, Nat. Commun. 13, 2779 (2022) (Cited
on page 31).
[492] B. Nikolov, E. Diamond-Hitchcock, J. Bass, N. L. R. Spong, and J. D. Pritchard, “Randomized Benchmarking Using Nondestructive
Readout in a Two-Dimensional Atom Array”, Phys. Rev. Lett. 131, 030602 (2023) (Cited on page 31).
[493] D. Jaksch, J. I. Cirac, P. Zoller, S. L. Rolston, R. Côté, and M. D. Lukin, “Fast Quantum Gates for Neutral Atoms”, Phys. Rev.
Lett. 85, 2208 (2000) (Cited on page 31).
[494] M. D. Lukin, M. Fleischhauer, R. Cote, L. M. Duan, D. Jaksch, J. I. Cirac, and P. Zoller, “Dipole Blockade and Quantum Information
Processing in Mesoscopic Atomic Ensembles”, Phys. Rev. Lett. 87, 037901 (2001) (Cited on page 31).
[495] X.-F. Shi, “Quantum logic and entanglement by neutral Rydberg atoms: methods and fidelity”, Quantum Sci. Technol. 7, 023002
(2022) (Cited on page 31).
[496] S. J. Evered et al., “High-Fidelity Parallel Entangling Gates on a Neutral-Atom Quantum Computer”, Nature 622, 268 (2023)
(Cited on page 31).
[497] Q. Xu, J. P. Bonilla Ataides, C. A. Pattison, N. Raveendran, D. Bluvstein, J. Wurtz, B. Vasić, M. D. Lukin, L. Jiang, and H.
Zhou, “Constant-overhead fault-tolerant quantum computation with reconfigurable atom arrays”, Nat. Phys., 1 (2024) (Cited on
page 31).
[498] T. M. Graham, L. Phuttitarn, R. Chinnarasu, Y. Song, C. Poole, K. Jooya, J. Scott, A. Scott, P. Eichler, and M. Saffman,
“Midcircuit measurements on a single-species neutral alkali atom quantum processor”, Phys. Rev. X 13, 041051 (2023) (Cited on
page 31).
[499] J. W. Lis, A. Senoo, W. F. McGrew, F. Rönchen, A. Jenkins, and A. M. Kaufman, “Midcircuit Operations Using the omg Archi-
tecture in Neutral Atom Arrays”, Phys. Rev. X 13, 041035 (2023) (Cited on page 31).
[500] I. Cong, H. Levine, A. Keesling, D. Bluvstein, S.-T. Wang, and M. D. Lukin, “Hardware-Efficient, Fault-Tolerant Quantum Com-
putation with Rydberg Atoms”, Phys. Rev. X 12, 021049 (2022) (Cited on page 31).
[501] Y. Wu, S. Kolkowitz, S. Puri, and J. D. Thompson, “Erasure Conversion for Fault-Tolerant Quantum Computing in Alkaline Earth
Rydberg Atom Arrays”, Nat. Commun. 13, 4657 (2022) (Cited on page 31).
[502] D. Petrosyan, S. Norrell, C. Poole, and M. Saffman, “Fast measurements and multiqubit gates in dual-species atomic arrays”, Phys.
Rev. A 110, 042404 (2024) (Cited on page 31).
[503] C. Corlett, I. Čepaitė, A. J. Daley, C. Gustiani, G. Pelegrí, J. D. Pritchard, N. Linden, and P. Skrzypczyk, “Speeding up quantum
measurement using space-time trade-off”, arXiv:2407.17342 (2024) (Cited on page 31).
[504] H. Pichler, S.-T. Wang, L. Zhou, S. Choi, and M. D. Lukin, “Quantum Optimization for Maximum Independent Set Using Rydberg
Atom Arrays”, arXiv:1808.10816 (2018) (Cited on page 31).
[505] S. Ebadi et al., “Quantum Optimization of Maximum Independent Set Using Rydberg Atom Arrays”, Science 376, 1209 (2022)
(Cited on page 31).
[506] M.-T. Nguyen, J.-G. Liu, J. Wurtz, M. D. Lukin, S.-T. Wang, and H. Pichler, “Quantum Optimization with Arbitrary Connectivity
Using Rydberg Atom Arrays”, PRX Quantum 4, 010316 (2023) (Cited on page 31).
[507] M. Lanthaler, C. Dlaska, K. Ender, and W. Lechner, “Rydberg-Blockade-Based Parity Quantum Optimization”, Phys. Rev. Lett.
130, 220601 (2023) (Cited on page 31).

120
References

[508] A. G. de Oliveira, E. Diamond-Hitchcock, D. M. Walker, M. T. Wells-Pestell, G. Pelegrí, C. J. Picken, G. P. A. Malcolm, A. J. Daley,

J. Bass, and J. D. Pritchard, “Demonstration of weighted graph optimization on a rydberg atom array using local light-shifts”,
arXiv:2404.02658 (2024) (Cited on page 31).
[509] J. Park, S. Jeong, M. Kim, K. Kim, A. Byun, L. Vignoli, L.-P. Henry, L. Henriet, and J. Ahn, “Rydberg-atom experiment for the
integer factorization problem”, Phys. Rev. Res. 6, 023241 (2024) (Cited on page 31).
[510] E. Knill, R. Laflamme, and G. J. Milburn, “A Scheme for Efficient Quantum Computation with Linear Optics”, Nature 409, 46
(2001) (Cited on page 32).
[511] M. Koashi, T. Yamamoto, and N. Imoto, “Probabilistic manipulation of entangled photons”, Phys. Rev. A 63, 030301 (2001) (Cited
on page 32).
[512] I. Walmsley, “Light in quantum computing and simulation: perspective”, Opt. Quantum 1, 35 (2023) (Cited on page 32).
[513] K. T. Kaczmarek et al., “High-speed noise-free optical quantum memory”, Phys. Rev. A 97, 042316 (2018) (Cited on page 32).
[514] S. Bartolucci et al., “Fusion-based quantum computation”, Nat. Commun. 14, 912 (2023) (Cited on page 32).
[515] S. Shi, B. Xu, K. Zhang, G. S. Ye, D. S. Xiang, Y. Liu, J. Wang, D. Su, and L. Li, “High-fidelity photonic quantum logic gate
based on near-optimal Rydberg single-photon source”, Nat. Commun. 13, 4454 (2022) (Cited on page 32).
[516] M. Hibat-Allah, M. Mauri, J. Carrasquilla, and A. Perdomo-Ortiz, “A framework for demonstrating practical quantum advantage:
comparing quantum against classical generative models”, Commun. Phys. 7, 68 (2024) (Cited on page 32).
[517] J. E. Bourassa et al., “Blueprint for a scalable photonic fault-tolerant quantum computer”, Quantum 5, 392 (2021) (Cited on
page 32).
[518] J. Floyd and P. Kwiat, “Long-distance entanglement distribution through satellite intermediary entanglement swapping”, Proc.
SPIE Quantum Comput. Commun. Simul. III, Vol. 12446 (2023), 124460L (Cited on page 32).
[519] C. Crépeau, D. Gottesman, and A. Smith, “Secure multi-party quantum computation”, Proc. 34th Annu. ACM Symp. Theory
Comput. (2002), 643 (Cited on page 32).
[520] W. Kozlowski, S. Wehner, Q. R. V. Meter, A. S. Cacciapuoti, M. Caleffi, and S. Nagayama, “RFC 9340: Architectural Principles
for a Quantum Internet”, RFC, 9340 (2023) (Cited on page 32).
[521] L. d’Avossa, M. Caleffi, C. Wang, J. Illiano, S. Zorzetti, and A. S. Cacciapuoti, “Towards the Quantum Internet: Entanglement
Rate Analysis of High-Efficiency Electro-Optic Transducer”, 2023 IEEE Int Conf Quantum Comput Eng QCE (2023), 1325 (Cited
on page 32).
[522] D. Gottesman, T. Jennewein, and S. Croke, “Longer-Baseline Telescopes Using Quantum Repeaters”, Phys. Rev. Lett. 109, 070503
(2012) (Cited on page 32).
[523] J. F. Fitzsimons and E. Kashefi, “Unconditionally verifiable blind quantum computation”, Phys. Rev. A 96, 012303 (2017) (Cited
on page 32).
[524] V. Giovannetti, S. Lloyd, and L. Maccone, “Quantum-enhanced measurements: Beating the standard quantum limit”, Science 306,
1330 (2004) (Cited on page 32).
[525] E. N. Fokoua, S. A. Mousavi, G. T. Jasion, D. J. Richardson, and F. Poletti, “Loss in hollow-core optical fibers: mechanisms, scaling
rules, and limits”, Adv. Opt. Photon. 15, 1 (2023) (Cited on page 32).
[526] Y. Chen et al., “Hollow Core DNANF Optical Fiber with ¡0.11 dB/km Loss”, Opt. Fiber Commun Conf OFC, Th4A.8 (2024)
(Cited on page 33).
[527] P. Stano and D. Loss, “Review of performance metrics of spin qubits in gated semiconducting nanostructures”, Nat. Rev. Phys. 4,
672–688 (2024) (Cited on page 33).
[528] A. M. J. Zwerver, T. Krähenmann, T. F. Watson, L. Lampert, H. C. George, R. Pillarisetty, S. A. Bojarski, P. Amin, S. V.
Amitonov, J. M. Boter, et al., “Qubits made by advanced semiconductor manufacturing”, Nat. Electron 5, 184 (2022) (Cited on
page 33).
[529] L. C. Camenzind, S. Geyer, A. Fuhrer, R. J. Warburton, D. M. Zumbühl, and A. V. Kuhlmann, “A hole spin qubit in a fin field-effect
transistor above 4 kelvin”, Nat. Electron. 5, 178 (2022) (Cited on page 33).
[530] M. de Kruijf, S. Geyer, T. Berger, M. Mergenthaler, F. Braakman, R. J. Warburton, and A. V. Kuhlmann, “A compact and versatile
cryogenic probe station for quantum device testing”, Rev. Sci. Instrum. 94, 054707 (2023) (Cited on page 33).
[531] S. Neyens, O. K. Zietz, T. F. Watson, F. Luthi, A. Nethwewala, H. C. George, E. Henry, M. Islam, A. J. Wagner, F. Borjans, et al.,
“Probing single electrons across 300-mm spin qubit wafers”, Nature 629, 80 (2024) (Cited on page 33).
[532] J. Eastoe, G. M. Noah, D. Dutta, A. Rossi, J. D. Fletcher, and A. Gomez-Saiz, “Efficient system for bulk characterization of
cryogenic CMOS components”, arXiv:2404.11451 (2024) (Cited on page 33).
[533] E. J. Thomas, V. N. Ciriano-Tejel, D. F. Wise, D. Prete, M. de Kruijf, D. J. Ibberson, G. M. Noah, A. Gomez-Saiz, M. F. Gonzalez-
Zalba, M. A. Johnson, et al., “Rapid cryogenic characterisation of 1024 integrated silicon quantum dots”, arXiv:2310.20434 (2023)
(Cited on page 33).
[534] C. H. Yang, R. C. C. Leon, J. C. C. Hwang, A. Saraiva, T. Tanttu, W. Huang, J. Camirand Lemyre, K. W. Chan, K. Y. Tan,
F. E. Hudson, et al., “Operation of a silicon quantum processor unit cell above one kelvin”, Nature 580, 350 (2020) (Cited on
page 33).
[535] X. Xue, B. Patra, J. P. van Dijk, N. Samkharadze, S. Subramanian, A. Corna, B. Paquelet Wuetz, C. Jeon, F. Sheikh, E. Juarez-
Hernandez, et al., “CMOS-based cryogenic control of silicon quantum circuits”, Nature 593, 205 (2021) (Cited on page 33).
[536] S. K. Bartee et al., “Spin Qubits with Integrated millikelvin CMOS Control”, arXiv:2407.15151 (2024) (Cited on page 33).
[537] L. M. K. Vandersypen, H. Bluhm, J. S. Clarke, A. S. Dzurak, R. Ishihara, A. Morello, D. J. Reilly, L. R. Schreiber, and M.
Veldhorst, “Interfacing spin qubits in quantum dots and donors—hot, dense, and coherent”, npj Quantum Inf. 3, 34 (2017) (Cited
on page 33).
[538] M. Veldhorst, J. C. C. Hwang, C. H. Yang, A. W. Leenstra, B. de Ronde, J. P. Dehollain, J. T. Muhonen, F. E. Hudson, K. M. Itoh,
A. t Morello, et al., “An addressable quantum dot qubit with fault-tolerant control-fidelity”, Nat. Nanotech 9, 981 (2014) (Cited
on page 33).
[539] A. R. Mills, C. R. Guinn, M. J. Gullans, A. J. Sigillito, M. M. Feldman, E. Nielsen, and J. R. Petta, “Two-qubit silicon quantum
processor with operation fidelity exceeding 99%”, Sci. Adv. 8, eabn5130 (2022) (Cited on page 33).
[540] R. M. Jock, N. T. Jacobson, M. Rudolph, D. R. Ward, M. S. Carroll, and D. R. Luhman, “A silicon singlet–triplet qubit driven by
spin-valley coupling”, Nat Commun. 13, 641 (2022) (Cited on page 33).
[541] T. F. Watson, S. G. J. Philips, E. Kawakami, D. R. Ward, P. Scarlino, M. Veldhorst, D. E. Savage, M. G. Lagally, M. Friesen,
S. N. Coppersmith, et al., “A programmable two-qubit quantum processor in silicon”, Nature 555, 633 (2018) (Cited on page 33).
[542] S. G. Philips, M. T. Mądzik, S. V. Amitonov, S. L. de Snoo, M. Russ, N. Kalhor, C. Volk, W. I. Lawrie, D. Brousse, L. Tryputen,
et al., “Universal control of a six-qubit quantum processor in silicon”, Nature 609, 919 (2022) (Cited on page 33).
[543] A. Zwerver, S. Amitonov, S. de Snoo, M. Mądzik, M. Rimbach-Russ, A. Sammak, G. Scappucci, and L. Vandersypen, “Shuttling
an Electron Spin through a Silicon Quantum Dot Array”, PRX Quantum 4, 030303 (2023) (Cited on page 33).
[544] X. Mi, M. Benito, S. Putz, D. M. Zajac, J. M. Taylor, G. Burkard, and J. R. Petta, “A coherent spin–photon interface in silicon”,
Nature 555, 599 (2018) (Cited on page 33).
[545] M. Veldhorst, H. G. J. Eenink, C.-H. Yang, and A. S. Dzurak, “Silicon CMOS architecture for a spin-based quantum computer”,
Nat Commun. 8, 1766 (2017) (Cited on page 33).
[546] R. Li et al., “A crossbar network for silicon quantum dot qubits”, Sci. Adv. 4, eaar3960 (2018) (Cited on page 33).
[547] A. Adedoyin, J. Ambrosiano, P. Anisimov, W. Casper, G. Chennupati, C. Coffrin, H. Djidjev, D. Gunter, S. Karra, N. Lemons,
et al., “Quantum algorithm implementations for beginners”, arXiv:1804.03719 (2018) (Cited on page 35).
[548] O. Di Matteo, “Understanding the Haar Measure”, Pennylane Website, [Accessed: 25-June-2024] (2021) (Cited on page 36).
[549] J. Watrous, “The Theory of Quantum Information”, Camb. Univ. Press 2018 (Cited on page 36).
[550] E. S. Meckes, “The Random Matrix Theory of the Classical Compact Groups”, Camb. Univ. Press 2019 (Cited on page 36).
[551] R. Bhandari and N. A. Peters, “On the general constraints in single qubit quantum process tomography”, Sci. Rep. 6, 26004 (2016)
(Cited on page 36).
[552] J. K. Iverson and J. Preskill, “Coherence in logical quantum channels”, New J. Phys. 22, 073066 (2020) (Cited on page 36).

121
References

[553] A. Rodriguez-Blanco, K. B. Whaley, and A. Bermudez, “Suppressing amplitude damping in trapped ions: Discrete weak measure-
ments for a nonunitary probabilistic noise filter”, Phys. Rev. A 107, 052409 (2023) (Cited on page 37).
[554] J. Ghosh, A. G. Fowler, and M. R. Geller, “Surface code with decoherence: An analysis of three superconducting architectures”,
Phys. Rev. A 86, 062318 (2012) (Cited on page 37).
[555] M. Urbanek, B. Nachman, V. R. Pascuzzi, A. He, C. W. Bauer, and W. A. de Jong, “Mitigating Depolarizing Noise on Quantum
Computers with Noise-Estimation Circuits”, Phys. Rev. Lett. 127, 270502 (2021) (Cited on page 38).
[556] E. Magesan and J. M. Gambetta, “Effective Hamiltonian models of the cross-resonance gate”, Phys. Rev. A 101, 052308 (2020)
(Cited on page 38).
[557] M. Malekakhlagh, E. Magesan, and D. C. McKay, “First-principles analysis of cross-resonance gate operation”, Phys. Rev. A 102,
042605 (2020) (Cited on page 38).
[558] K. Vogel and H. Risken, “Determination of quasiprobability distributions in terms of probability distributions for the rotated
quadrature phase”, Phys. Rev. A 40, 2847 (1989) (Cited on page 40).
[559] C. J. Wood, J. D. Biamonte, and D. G. Cory, “Tensor Networks and Graphical Calculus for Open Quantum Systems”, Quantum
Info. Comput. 15, 759 (2015) (Cited on page 40).
[560] K.-N. Schymik, V. Lienhard, D. Barredo, P. Scholl, H. Williams, A. Browaeys, and T. Lahaye, “Enhanced atom-by-atom assembly
of arbitrary tweezer arrays”, Phys. Rev. A 102, 063107 (2020) (Cited on pages 47, 107).
[561] L. Z. Cohen, I. H. Kim, S. D. Bartlett, and B. J. Brown, “Low-overhead fault-tolerant quantum computing using long-range
connectivity”, Sci. Adv. 8, eabn1717 (2022) (Cited on page 48).
[562] P. Murali, D. C. Mckay, M. Martonosi, and A. Javadi-Abhari, “Software Mitigation of Crosstalk on Noisy Intermediate-Scale
Quantum Computers”, Proc 25th Int Conf Arch. Support Progra Lang Oper. Syst, ASPLOS ’20 (2020), 1001 (Cited on page 48).
[563] E. Martin-Lopez, A. Laing, T. Lawson, R. Alvarez, X.-Q. Zhou, and J. L. O’brien, “Experimental realization of Shor’s quantum
factoring algorithm using qubit recycling”, Nat. Photonics 6, 773 (2012) (Cited on page 50).
[564] D. E. Browne, E. Kashefi, M. Mhalla, and S. Perdrix, “Generalized flow and determinism in measurement-based quantum compu-
tation”, New J. Phys. 9, 250 (2007) (Cited on page 50).
[565] K. Rudinger, G. J. Ribeill, L. C. Govia, M. Ware, E. Nielsen, K. Young, T. A. Ohki, R. Blume-Kohout, and T. Proctor, “Charac-
terizing Midcircuit Measurements on a Superconducting Qubit Using Gate Set Tomography”, Phys. Rev. Appl. 17, 014014 (2022)
(Cited on page 51).
[566] D. M. Berns, W. D. Oliver, S. O. Valenzuela, A. V. Shytov, K. K. Berggren, L. S. Levitov, and T. P. Orlando, “Coherent Quasi-
classical Dynamics of a Persistent Current Qubit”, Phys. Rev. Lett. 97, 150502 (2006) (Cited on page 52).
[567] G. Ithier et al., “Decoherence in a superconducting quantum bit circuit”, Phys. Rev. B 72, 134519 (2005) (Cited on pages 53, 54).
[568] E. L. Hahn, “Spin Echoes”, Phys. Rev. 80, 580 (1950) (Cited on page 54).
[569] K. Young, S. Bartlett, R. J. Blume-Kohout, J. K. Gamble, D. Lobser, P. Maunz, E. Nielsen, T. J. Proctor, M. Revelle, and K. M.
Rudinger, “Diagnosing and Destroying Non-Markovian Noise”, Tech Rep Sandia Natl. LabSNL-CA Livermore CA U. S. (2020)
(Cited on page 55).
[570] M. J. W. Hall, J. D. Cresser, L. Li, and E. Andersson, “Canonical form of master equations and characterization of non-
markovianity”, Phys. Rev. A 89, 042120 (2014) (Cited on page 56).
[571] Y. R. Sanders, J. J. Wallman, and B. C. Sanders, “Bounding quantum gate error rate based on reported average fidelity”, New J.
Phys. 18, 012002 (2015) (Cited on page 59).
[572] M. M. Wilde, “From classical to quantum Shannon theory”, arXiv:1106.1445 (2011) (Cited on page 59).
[573] J. Watrous, “Simpler semidefinite programs for completely bounded norms”, arXiv:1207.5726 (2012) (Cited on page 59).
[574] A. Yurtsever, J. A. Tropp, O. Fercoq, M. Udell, and V. Cevher, “Scalable Semidefinite Programming”, SIAM J. Math. Data Sci.
3, 171 (2021) (Cited on page 59).
[575] C. H. Baldwin, B. J. Bjork, J. P. Gaebler, D. Hayes, and D. Stack, “Subspace benchmarking high-fidelity entangling operations
with trapped ions”, Phys. Rev. Res. 2, 013317 (2020) (Cited on page 61).
[576] J. J. Wallman, “Randomized benchmarking with gate-dependent noise”, Quantum 2, 47 (2018) (Cited on page 61).
[577] L. Chen, H.-X. Li, Y. Lu, C. W. Warren, C. J. Križan, S. Kosen, M. Rommel, S. Ahmed, A. Osman, J. Biznárová, et al., “Transmon
qubit readout fidelity at the threshold for quantum error correction without a quantum-limited amplifier”, Npj Quantum Inf. 9, 26
(2023) (Cited on page 69).
[578] S. Aaronson and L. Chen, “Complexity-Theoretic Foundations of Quantum Supremacy Experiments”, Proc 32nd Comput Complex.
Conf (2016), 22:1 (Cited on pages 70, 71).
[579] N. Moll et al., “Quantum Optimization Using Variational Algorithms on Near-Term Quantum Devices”, Quantum Sci. Technol. 3,
030503 (2018) (Cited on page 71).
[580] K. Bharti et al., “Noisy intermediate-scale quantum algorithms”, Rev. Mod. Phys. 94, 015004 (2022) (Cited on pages 78, 81, 82).
[581] E. Fontana, N. Fitzpatrick, D. M. Ramo, R. Duncan, and I. Rungger, “Evaluating the noise resilience of variational quantum
algorithms”, Phys. Rev. A 104, 022403 (2021) (Cited on page 79).
[582] J. Hubbard and B. H. Flowers, “Electron correlations in narrow energy bands”, Proc R Soc Lond. A 276, 238 (1963) (Cited on
page 79).
[583] M. A. Nielsen, “The Fermionic canonical commutation relations and the Jordan-Wigner transform”, Tech Rep Sch. Phys. Sci. Univ.
Qld. 59, 75 (2005) (Cited on page 80).
[584] D. Wecker, M. B. Hastings, and M. Troyer, “Progress towards practical quantum variational algorithms”, Phys. Rev. A 92, 042303
(2015) (Cited on page 80).
[585] J.-M. Reiner, F. Wilhelm-Mauch, G. Schön, and M. Marthaler, “Finding the ground state of the Hubbard model by variational
methods on a quantum computer with gate errors”, Quantum Sci. Technol. 4, 035005 (2019) (Cited on page 80).
[586] B. Anselme Martin, P. Simon, and M. J. Rančić, “Simulating strongly interacting Hubbard chains with the variational Hamiltonian
ansatz on a quantum computer”, Phys. Rev. Res. 4, 023190 (2022) (Cited on page 80).
[587] Z. Jiang, K. J. Sung, K. Kechedzhi, V. N. Smelyanskiy, and S. Boixo, “Quantum Algorithms to Simulate Many-Body Physics of
Correlated Fermions”, Phys. Rev. Appl. 9, 044036 (2018) (Cited on page 80).
[588] J. Håstad, “Some optimal inapproximability results”, J. ACM 48, 798 (2001) (Cited on page 81).
[589] K. J. Ahn and S. Guha, “Graph sparsification in the semi-streaming model”, Int Colloq Autom. Lang Program (2009), 328 (Cited
on page 81).
[590] M. J. D. Powell, “A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Inter-
polation”, Advances in Optimization and Numerical Analysis, edited by S. Gomez and J.-P. Hennart (Springer Netherlands,
Dordrecht, 1994), 51 (Cited on page 82).
[591] J. Hubbard, “Electron correlations in narrow energy bands”, Proc. R. Soc. Lond. A 276, 238 (1963) (Cited on page 83).
[592] M. Qin, T. Schäfer, S. Andergassen, P. Corboz, and E. Gull, “The hubbard model: a computational perspective”, Annu. Rev.
Condens. Matter Phys. 13, 275 (2022) (Cited on page 83).
[593] A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, “Dynamical mean-field theory of strongly correlated fermion systems
and the limit of infinite dimensions”, Rev. Mod. Phys. 68, 13 (1996) (Cited on page 83).
[594] G. Kotliar, S. Y. Savrasov, K. Haule, V. S. Oudovenko, O. Parcollet, and C. A. Marianetti, “Electronic structure calculations with
dynamical mean-field theory”, Rev. Mod. Phys. 78, 865 (2006) (Cited on page 83).
[595] A. Paul and T. Birol, “Applications of dft+ dmft in materials science”, Annu. Rev. Mater. Res. 49, 31 (2019) (Cited on page 83).
[596] F. Jamet, A. Agarwal, C. Lupo, D. E. Browne, C. Weber, and I. Rungger, “Krylov variational quantum algorithm for first principles
materials simulations”, arXiv:2105.13298 (2021) (Cited on page 83).
[597] F. Jamet, A. Agarwal, and I. Rungger, “Quantum subspace expansion algorithm for green’s functions”, arXiv:2205.00094 (2022)
(Cited on page 83).
[598] F. Jamet, C. Lenihan, L. P. Lindoy, A. Agarwal, E. Fontana, B. A. Martin, and I. Rungger, “Anderson impurity solver integrating
tensor network methods with quantum computing”, arXiv:2304.06587 (2023) (Cited on page 83).

122
References

[599] A. Montanaro and S. Stanisic, “Compressed variational quantum eigensolver for the Fermi-Hubbard model”, arXiv:2006.01179
(2020) (Cited on page 83).
[600] S. Stanisic, J. L. Bosse, F. M. Gambetta, R. A. Santos, W. Mruczkiewicz, T. E. O’Brien, E. Ostby, and A. Montanaro, “Observing
ground-state properties of the Fermi-Hubbard model using a scalable algorithm on a quantum computer”, Nat. Commun. 13, 5743
(2022) (Cited on page 83).
[601] F. Arute et al., “Observation of separated dynamics of charge and spin in the Fermi-Hubbard model”, arXiv:2010.07965 (2020)
(Cited on page 83).
[602] K. Mayer, C. Ryan-Anderson, N. Brown, E. Durso-Sabina, C. H. Baldwin, D. Hayes, J. M. Dreiling, C. Foltz, J. P. Gaebler, T. M.
Gatterman, et al., “Benchmarking logical three-qubit quantum fourier transform encoded in the steane code on a trapped-ion
quantum computer”, arXiv:2404.08616 (2024) (Cited on page 86).
[603] B. Park and D. Ahn, “Reducing cnot count in quantum fourier transform for the linear nearest-neighbor architecture”, Sci. Rep.
13, 8638 (2023) (Cited on page 86).
[604] V. V. Shende, I. L. Markov, and S. S. Bullock, “Minimal universal two-qubit controlled-NOT-based circuits”, Phys. Rev. A 69,
062321 (2004) (Cited on page 89).
[605] M. Möttönen, J. J. Vartiainen, V. Bergholm, and M. M. Salomaa, “Quantum Circuits for General Multiqubit Gates”, Phys. Rev.
Lett. 93, 130502 (2004) (Cited on pages 89, 90).
[606] R. R. Tucci, “An introduction to Cartan’s KAK decomposition for QC programmers”, arXiv:quant-ph/0507171 (2005) (Cited on
page 89).
[607] S. S. Bullock and G. K. Brennen, “Canonical decompositions of n-qubit quantum computations and concurrence”, J. Math. Phys
45, 2447 (2004) (Cited on page 89).
[608] Y. Sunada, S. Kono, J. Ilves, S. Tamate, T. Sugiyama, Y. Tabuchi, and Y. Nakamura, “Fast Readout and Reset of a Superconducting
Qubit Coupled to a Resonator with an Intrinsic Purcell Filter”, Phys. Rev. Appl. 17, 044016 (2022) (Cited on page 91).
[609] S. L. Todaro, V. B. Verma, K. C. McCormick, D. T. C. Allcock, R. P. Mirin, D. J. Wineland, S. W. Nam, A. C. Wilson, D. Leibfried,
and D. H. Slichter, “State Readout of a Trapped Ion Qubit Using a Trap-Integrated Superconducting Photon Detector”, Phys.
Rev. Lett. 126, 010501 (2021) (Cited on page 91).
[610] M. D. Reed, B. R. Johnson, A. A. Houck, L. DiCarlo, J. M. Chow, D. I. Schuster, L. Frunzio, and R. J. Schoelkopf, “Fast reset
and suppressing spontaneous emission of a superconducting qubit”, Appl. Phys. Lett. 96, 203110 (2010) (Cited on page 92).
[611] K. Geerlings, Z. Leghtas, I. M. Pop, S. Shankar, L. Frunzio, R. J. Schoelkopf, M. Mirrahimi, and M. H. Devoret, “Demonstrating
a Driven Reset Protocol for a Superconducting Qubit”, Phys. Rev. Lett. 110, 120501 (2013) (Cited on page 92).
[612] P. Magnard et al., “Fast and Unconditional All-Microwave Reset of a Superconducting Qubit”, Phys. Rev. Lett. 121, 060502 (2018)
(Cited on page 92).
[613] M. Werninghaus, D. J. Egger, and S. Filipp, “High-Speed Calibration and Characterization of Superconducting Quantum Processors
without Qubit Reset”, PRX Quantum 2, 020324 (2021) (Cited on page 92).
[614] G. S. Ravi, K. N. Smith, P. Gokhale, and F. T. Chong, “Quantum Computing in the Cloud: Analyzing job and machine character-
istics”, 2021 IEEE Int Symp Workload Charact. IISWC (2021), 39 (Cited on page 94).
[615] K. L. Pudenz, T. Albash, and D. A. Lidar, “Quantum annealing correction for random Ising problems”, Phys. Rev. A 91, 042302
(2015) (Cited on page 95).
[616] K. C. Young, R. Blume-Kohout, and D. A. Lidar, “Adiabatic Quantum Optimization with the Wrong Hamiltonian”, Phys. Rev. A
88, 062314 (2013) (Cited on page 95).
[617] T. Albash, V. Martin-Mayor, and I. Hen, “Analog Errors in Ising Machines”, Quantum Sci. Technol. 4, 02LT03 (2019) (Cited on
page 95).
[618] K. L. Pudenz, T. Albash, and D. A. Lidar, “Error-Corrected Quantum Annealing with Hundreds of Qubits”, Nat. Commun. 5,
3243 (2014) (Cited on page 95).
[619] K. Nishimura, H. Nishimori, A. J. Ochoa, and H. G. Katzgraber, “Retrieving the Ground State of Spin Glasses Using Thermal
Noise: Performance of Quantum Annealing at Finite Temperatures”, Phys. Rev. E 94, 032105 (2016) (Cited on page 95).
[620] W. Vinci and D. A. Lidar, “Scalable Effective-Temperature Reduction for Quantum Annealers via Nested Quantum Annealing
Correction”, Phys. Rev. A 97, 022308 (2018) (Cited on page 95).
[621] J. Bennett, A. Callison, T. O’Leary, M. West, N. Chancellor, and V. Kendon, “Using copies can improve precision in continuous-time
quantum computing”, Quantum Sci. Technol. 8, 035031 (2023) (Cited on page 95).
[622] N. Chancellor, P. J. D. Crowley, T. Ðurić, W. Vinci, M. H. Amin, A. G. Green, P. A. Warburton, and G. Aeppli, “Error Measure-
ments for a Quantum Annealer Using the One-Dimensional Ising Model with Twisted Boundaries”, npj Quantum Inf. 8, 73 (2022)
(Cited on page 96).
[623] M. Werner, A. García-Sáez, and M. P. Estarellas, “Quantum simulation of one-dimensional fermionic systems with Ising Hamilto-
nians”, arXiv:2406.06378 (2024) (Cited on page 96).
[624] V. Choi, “Minor-Embedding in Adiabatic Quantum Computation: I. The Parameter Setting Problem”, Quantum Inf. Process. 7,
193 (2008) (Cited on page 98).
[625] V. Choi, “Minor-Embedding in Adiabatic Quantum Computation: II. Minor-universal Graph Design”, Quantum Inf. Process. 10,
343 (2011) (Cited on pages 98, 99).
[626] W. Lechner, P. Hauke, and P. Zoller, “A Quantum Annealing Architecture with All-to-All Connectivity from Local Interactions”,
Sci. Adv. 1, e1500838 (2015) (Cited on page 98).
[627] M. Leib, P. Zoller, and W. Lechner, “A Transmon Quantum Annealer: Decomposing Many-Body Ising Constraints into Pair
Interactions”, Quantum Sci. Technol. 1, 015008 (2016) (Cited on page 98).
[628] A. Rocchetto, S. C. Benjamin, and Ying Li, “Stabilizers as a Design Tool for New Forms of the Lechner-Hauke-Zoller Annealer”,
Sci. Adv. 2, e1601246 (2016) (Cited on page 98).
[629] N. Chancellor, S. Zohren, and P. A. Warburton, “Circuit Design for Multi-Body Interactions in Superconducting Quantum Annealing
Systems with Applications to a Scalable Architecture”, npj Quantum Inf. 3, 21 (2017) (Cited on page 98).
[630] A. Palacios, A. Garcia-Saez, and M. P. Estarellas, “A scalable 2-local architecture for quantum annealing of all-to-all Ising models”,
arXiv:2404.06861 (2024) (Cited on page 98).
[631] R. Diestel, “Graph Theory: 5th Edition”, Grad. Texts Math. Springer Berl. Heidelb. (2017) (Cited on page 98).
[632] J. Cai, W. G. Macready, and A. Roy, “A Practical Heuristic for Finding Graph Minors”, arXiv:1406.2741 (2014) (Cited on page 99).
[633] J. Roffe, S. Zohren, D. Horsman, and N. Chancellor, “Decoding Quantum Error Correction with Ising Model Hardware”, arXiv:1903.10254
(2019) (Cited on page 99).
[634] J. Berwald, N. Chancellor, and R. Dridi, “Understanding Domain-Wall Encoding Theoretically and Experimentally”, Philos. Trans.
R. Soc. A: Math. Phys. Eng. Sci. 381, 20210410 (2023) (Cited on pages 99, 101).
[635] J. Raymond, S. Yarkoni, and E. Andriyash, “Global Warming: Temperature Estimation in Annealers”, Front. ICT 3 (2016) (Cited
on pages 100, 101).
[636] N. Chancellor, S. Szoke, W. Vinci, G. Aeppli, and P. A. Warburton, “Maximum-Entropy Inference with a Programmable Annealer”,
Sci. Rep. 6, 22318 (2016) (Cited on page 100).
[637] M. Benedetti, J. Realpe-Gómez, R. Biswas, and A. Perdomo-Ortiz, “Quantum-Assisted Learning of Hardware-Embedded Proba-
bilistic Graphical Models”, Phys. Rev. X 7, 041052 (2017) (Cited on page 100).
[638] S. H. Adachi and M. P. Henderson, “Application of Quantum Annealing to Training of Deep Neural Networks”, arXiv:1510.06356
(2015) (Cited on page 100).
[639] M. H. Amin, E. Andriyash, J. Rolfe, B. Kulchytskyy, and R. Melko, “Quantum Boltzmann Machine”, Phys. Rev. X 8, 021050
(2018) (Cited on page 100).
[640] M. Benedetti, J. Realpe-Gómez, and A. Perdomo-Ortiz, “Quantum-Assisted Helmholtz Machines: A Quantum–Classical Deep
Learning Framework for Industrial Datasets in near-Term Devices”, Quantum Sci. Technol. 3, 034007 (2018) (Cited on page 100).
[641] J. Caldeira, J. Job, S. H. Adachi, B. Nord, and G. N. Perdue, “Restricted Boltzmann Machines for Galaxy Morphology Classification
with a Quantum Annealer”, arXiv:1911.06259 (2019) (Cited on page 100).

123
References

[642] M. Benedetti, J. Realpe-Gómez, R. Biswas, and A. Perdomo-Ortiz, “Estimation of Effective Temperatures in Quantum Annealers
for Sampling Applications: A Case Study with Possible Applications in Deep Learning”, Phys. Rev. A 94, 022308 (2016) (Cited on
page 101).
[643] A. D. King et al., “Coherent Quantum Annealing in a Programmable 2,000 qubit Ising Chain”, Nat. Phys. 18, 1324 (2022) (Cited
on page 101).
[644] G. Frascella, S. Agne, F. Y. Khalili, and M. V. Chekhova, “Overcoming detection loss and noise in squeezing-based optical sensing”,
npj Quantum Inf. 7, 72 (2021) (Cited on page 102).
[645] T. Park, H. Stokowski, V. Ansari, S. Gyger, K. K. Multani, O. T. Celik, A. Y. Hwang, D. J. Dean, F. Mayor, T. P. McKenna, et al.,
“Single-mode squeezed-light generation and tomography with an integrated optical parametric oscillator”, Sci. Adv. 10, eadl1814
(2024) (Cited on page 102).
[646] S. Popoff, G. Lerosey, R. Carminati, M. Fink, A. Boccara, and S. Gigan, “Measuring and exploiting the transmission matrix in
optics”, Quantum Electron. Laser Sci. Conf., QME6 (2010) (Cited on page 102).
[647] M. Lobino, D. Korystov, C. Kupchak, E. Figueroa, B. C. Sanders, and A. Lvovsky, “Complete characterization of quantum-optical
processes”, Science 322, 563 (2008) (Cited on page 102).
[648] S. Rahimi-Keshari, M. A. Broome, R. Fickler, A. Fedrizzi, T. C. Ralph, and A. G. White, “Direct characterization of linear-optical
networks”, Opt. Express 21, 13450 (2013) (Cited on page 102).
[649] I. Dhand, A. Khalid, H. Lu, and B. C. Sanders, “Accurate and precise characterization of linear optical interferometers”, J. Opt.
18, 035204 (2016) (Cited on page 102).
[650] N. Maring, A. Fyrillas, M. Pont, E. Ivanov, P. Stepanov, N. Margaria, W. Hease, A. Pishchagin, A. Lemaître, I. Sagnes, et al., “A
versatile single-photon-based quantum computing platform”, Nat. Photonics 18, 603 (2024) (Cited on page 103).
[651] J. Martínez-Cifuentes, K. M. Fonseca-Romero, and N. Quesada, “Classical Models May Be a Better Explanation of the Jiuzhang
1.0 Gaussian Boson Sampler than Its Targeted Squeezed Light Model”, Quantum 7, 1076 (2023) (Cited on page 103).
[652] P. D. Drummond, B. Opanchuk, A. Dellios, and M. D. Reid, “Simulating Complex Networks in Phase Space: Gaussian Boson
Sampling”, Phys. Rev. A 105, 012427 (2022) (Cited on page 104).
[653] J. Choi et al., “Preparing random states and benchmarking with many-body quantum chaos”, Nature 613, 468 (2023) (Cited on
page 106).
[654] A. L. Shaw, Z. Chen, J. Choi, D. K. Mark, P. Scholl, R. Finkelstein, A. Elben, S. Choi, and M. Endres, “Benchmarking highly
entangled states on a 60-atom analogue quantum simulator”, Nature 628, 71 (2024) (Cited on pages 106, 107).
[655] T. M. Graham et al., “Multi-qubit entanglement and algorithms on a neutral-atom quantum computer”, Nature 604, 457 (2022)
(Cited on page 107).
[656] D. B. Tan, D. Bluvstein, M. D. Lukin, and J. Cong, “Compiling Quantum Circuits for Dynamically Field-Programmable Neutral
Atoms Array Processors”, Quantum 8, 1281 (2024) (Cited on page 108).

124