supermarq.qcvv

A toolkit of QCVV routines.

Submodules

Classes

BenchmarkingExperiment

Base class for gate benchmarking experiments.

BenchmarkingResults

A dataclass for storing the results of the experiment. Requires subclassing for

IRB

Interleaved random benchmarking (IRB) experiment.

IRBResults

Data structure for the IRB experiment results.

Sample

A sample circuit to use along with any data about the circuit

XEB

Cross-entropy benchmarking (XEB) experiment.

XEBResults

Results from an XEB experiment.

XEBSample

The samples used in XEB experiments.

Package Contents

class supermarq.qcvv.BenchmarkingExperiment(num_qubits: int, *, random_seed: int | numpy.random.Generator | None = None, **kwargs: Any)

Bases: abc.ABC, Generic[ResultsT]

Base class for gate benchmarking experiments.

The interface for implementing these experiments is as follows:

  1. First instantiate the desired experiment object

    experiment = ExampleExperiment(<<args/kwargs>>)

  2. Prepare the circuits and run the experiment on the desired target. This can either be a custom simulator or a real device name. For example:

    noise_model = cirq.depolarize(p=0.01, n_qubits=1)
sim = cirq.DensityMatrixSimulator(noise=noise_model)

experiment.prepare_experiment(<<args/kwargs>>)
experiment.run_with_simulator(simulator=sim, <<args/kwargs>>)

  3. Then we analyse the results. If the target was a local simulator this will be available as soon as the run_with_simulator() method has finished executing. On the other hand if a real device was accessed via Superstaq then it may take time for the data to be available from the server. The collect_data() will return True when all data has been collected and is ready to be analysed.

    if experiment.collect_data():
    results = experiment.analyze_results(<<args>>)

  4. The final results of the experiment will be stored in the results attribute as a BenchmarkingResults of values, while all the data from the experiment will be stored in the raw_data attribute as a DataFrame. Some experiments may include additional data attributes for data generated during the analysis.

    results = experiment.results
data = experiment.raw_data

Additionally it is possible to pre-compile the experimental circuits for a given device using

experiment.prepare_experiment(<<args/kwargs>>)
experiment.compile_circuits(target=<<target_name>>)

And then to run the experiment using a custom callable function for evaluating the circuits. For example this could be a function that uses a connection to a local device.

experiment.run_with_callable(<<function_name>>)

When implementing a new experiment, 4 methods need to be implemented:

  1. _build_circuits(): Given a number of circuits and an iterable of the different numbers

    of layers to use, return a list of Sample objects that need to be sampled during the experiment.

  2. _process_probabilities(): Take the probability distribution over the

    computational basis resulting from running each circuit and combine the relevant details into a pandas.DataFrame.

  3. analyze_results(): Analyse the data in the raw_data dataframe and return a

    BenchmarkingResults object containing the results of the experiment.

  4. plot_results(): Produce any relevant plots that are useful for understanding the

    results of the experiment.

Additionally the BenchmarkingResults dataclass needs subclassing to hold the specific results of the new experiment.

abstract analyze_results(plot_results: bool = True) → ResultsT

Perform the experiment analysis and store the results in the results attribute.

Parameters:

plot_results – Whether to generate plots of the results. Defaults to False.

Returns:

A named tuple of the final results from the experiment.

collect_data(force: bool = False) → bool

Collect the data from the samples and process it into the raw_data attribute.

If the data is successfully stored in the raw_data attribute then the function will return True.

If either not all jobs submitted to the server have completed, or not all samples have probability results then no data will be saved in raw_data and the function will return False. This check can be overridden with force=True in which case only the samples which have probability results will be used to generate the results dataframe.

Parameters:

force – Whether to override the check that all data is present. Defaults to False.

Raises:

  • RuntimeError – If force=True but there are no samples with any data.

  • RuntimeError – If the experiment has not yet been run.

Returns:

Whether the results dataframe has been successfully created.

compile_circuits(target: str, **kwargs: Any) → None

Compiles the experiment circuits for the given device. Useful if the samples are not going to be run via Superstaq.

Parameters:

  • target – The device to compile to.

  • kwargs – Additional desired compile options.

abstract plot_results() → None

Plot the results of the experiment

prepare_experiment(num_circuits: int, cycle_depths: collections.abc.Iterable[int], overwrite: bool = False) → None

Prepares the circuits needed for the experiment

Parameters:

  • num_circuits – Number of circuits to run.

  • cycle_depths – An iterable of the different layer depths to use during the experiment.

  • overwrite – Whether to force an experiment run even if there is existing data that would be over written in the process. Defaults to False.

Raises:

  • RuntimeError – If the experiment has already been run once and the overwrite argument is not True

  • ValueError – If any of the cycle depths provided negative or zero.

run_on_device(target: str, repetitions: int = 10000, method: str | None = None, overwrite: bool = False, **target_options: Any) → cirq_superstaq.Job

Submit the circuit samples to the desired target device and store the resulting probabilities.

The set of circuits is partitioned as necessary to not exceed the maximum circuits that can be submitted to the given target device. The function then waits for the jobs to complete before saving the resulting probability distributions.

Parameters:

  • target – The name of a Superstaq target.

  • repetitions – The number of shots to sample. Defaults to 10,000.

  • method – Optional method to use on the Superstaq device. Defaults to None corresponding to normal running.

  • target_options – Optional configuration dictionary passed when submitting the job.

  • overwrite – Whether to force an experiment run even if there is existing data that would be over written in the process. Defaults to False.

Returns:

The superstaq job containing all the circuits submitted as part of the experiment.

run_with_callable(circuit_eval_func: collections.abc.Callable[[cirq.Circuit], dict[str, float]], overwrite: bool = False, **kwargs: Any) → None

Evaluates the probabilities for each circuit using a user provided callable function. This function should take a circuit as input and return a dictionary of probabilities for each bitstring (including states with zero probability).

Parameters:

  • circuit_eval_func – The custom function to use when evaluating circuit probabilities.

  • overwrite – Whether to force an experiment run even if there is existing data that would be over written in the process. Defaults to False.

  • kwargs – Additional arguments to pass to the custom function.

Raises:

  • RuntimeError – If the returned probabilities dictionary keys is missing include an incorrect number of bits.

  • RuntimeError – If the returned probabilities dictionary values do not sum to 1.0.

run_with_simulator(simulator: cirq.Sampler | None = None, repetitions: int = 10000, overwrite: bool = False) → None

Use the local simulator to sample the circuits and store the resulting probabilities.

Parameters:

  • simulator – A local Sampler to use. If None then the default cirq.Simulator simulator is used. Defaults to None.

  • repetitions – The number of shots to sample. Defaults to 10,000.

  • overwrite – Whether to force an experiment run even if there is existing data that would be over written in the process. Defaults to False.

property num_qubits: int

Returns: The number of qubits used in the experiment

qubits

The qubits used in the experiment.

property raw_data: pandas.DataFrame

The data from the most recently run experiment.

Raises:

RuntimeError – If no results are available.

property results: ResultsT

The results from the most recently run experiment.

Raises:

RuntimeError – If no results are available.

property samples: collections.abc.Sequence[Sample]

The samples generated during the experiment.

Raises:

RuntimeError – If no samples are available.

property targets: frozenset[str]

Returns: A set of the unique target that each sample was submitted to

class supermarq.qcvv.BenchmarkingResults

Bases: abc.ABC

A dataclass for storing the results of the experiment. Requires subclassing for each new experiment type.

experiment_name: str

The name of the experiment.

target: str

The target device that was used.

total_circuits: int

The total number of circuits used in the experiment.

class supermarq.qcvv.IRB(interleaved_gate: cirq.Gate | None = cirq.Z, num_qubits: int | None = 1, clifford_op_gateset: cirq.CompilationTargetGateset = cirq.CZTargetGateset(), *, random_seed: int | numpy.random.Generator | None = None)

Bases: supermarq.qcvv.base_experiment.BenchmarkingExperiment[Union[IRBResults, RBResults]]

Interleaved random benchmarking (IRB) experiment.

IRB estimates the gate error of specified Clifford gate, \(\mathcal{C}^*\). This is achieved by first choosing a random sequence, \(\{\mathcal{C_i}\}_m\) of \(m\) Clifford gates and then using this to generate two circuits. The first is generated by appending to this sequence the single gate that corresponds to the inverse of the original sequence. The second circuit it obtained by inserting the interleaving gate, \(\mathcal{C}^*\) after each gate in the sequence and then again appending the corresponding inverse element of the new circuit. Thus both circuits correspond to the identity operation.

We run both circuits on the specified target and calculate the probability of measuring the resulting state in the ground state, \(p(0...0)\). This gives the circuit fidelity

\[f(m) = 2p(0...0) - 1\]

We can then fit an exponential decay \(\log(f) \sim m\) to this circuit fidelity for each circuit, with decay rates \(\alpha\) and \(\tilde{\alpha}\) for the circuit without and with interleaving respectively. Finally the gate error of the specified gate, \(\mathcal{C}^*\) is estimated as

\[e_{\mathcal{C}^*} = \frac{1}{2} \left(1 - \frac{\tilde{\alpha}}{\alpha}\right)\]

For more details see: https://arxiv.org/abs/1203.4550

analyze_results(plot_results: bool = True) → IRBResults | RBResults

Analyse the experiment results and estimate the interleaved gate error.

Parameters:

plot_results – Whether to generate plots of the results. Defaults to False.

Returns:

A named tuple of the final results from the experiment.

static exp_decay(x: numpy.typing.NDArray[numpy.float_], A: float, alpha: float, B: float) → numpy.typing.NDArray[numpy.float_]

Exponential decay of the form

\[A \alpha^x + B\]
Parameters:

  • x – x

  • A – Decay constant

  • alpha – Decay coefficient

  • B – Additive constant

Returns:

Exponential decay applied to x.

gates_per_clifford(samples: int = 500) → dict[str, float]

Samples a number of random Clifford operations and calculates the average number of single and two qubit gates used to implement them. Note this depends on the gateset chosen for the experiment.

Parameters:

samples – Number of samples to use. Defaults to 500.

Returns:

A dictionary with the average number of one and two qubit gates used.

plot_results() → None

Plot the exponential decay of the circuit fidelity with cycle depth.

random_clifford() → cirq.CliffordGate

Returns: A random clifford gate with the correct number of qubits for the current experiment.

random_single_qubit_clifford() → cirq.SingleQubitCliffordGate

Choose a random single qubit clifford gate.

Returns:

The random clifford gate.

random_two_qubit_clifford() → cirq.CliffordGate

Choose a random two qubit clifford gate.

For algorithm details see https://arxiv.org/pdf/1402.4848 & https://arxiv.org/pdf/1210.7011.

Returns:

The random clifford gate.

clifford_op_gateset

The gateset to use when implementing Clifford operations.

class supermarq.qcvv.IRBResults

Bases: supermarq.qcvv.base_experiment.BenchmarkingResults

Data structure for the IRB experiment results.

average_interleaved_gate_error: float | None

Estimate of the interleaving gate error.

average_interleaved_gate_error_std: float | None

Standard deviation of the estimate for the interleaving gate error.

experiment_name = 'IRB'

The name of the experiment.

irb_decay_coefficient: float | None

Decay coefficient estimate with the interleaving gate.

irb_decay_coefficient_std: float | None

Standard deviation of the decay coefficient estimate with the interleaving gate.

rb_decay_coefficient: float

Decay coefficient estimate without the interleaving gate.

rb_decay_coefficient_std: float

Standard deviation of the decay coefficient estimate without the interleaving gate.

class supermarq.qcvv.Sample

A sample circuit to use along with any data about the circuit that is needed for analysis

property circuit: cirq.Circuit

Returns: The circuit used for the experiment. Defaults to the compiled circuit if available and if not returns the raw circuit.

compiled_circuit: cirq.Circuit | None = None

The compiled circuit. Only used if the circuits are compiled for a specific target.

data: dict[str, Any]

The corresponding data about the circuit

job: cirq_superstaq.Job | None = None

The superstaq job corresponding to the sample. Defaults to None if no job is associated with the sample.

probabilities: dict[str, float] | None = None

The probabilities of the computational basis states

raw_circuit: cirq.Circuit

The raw (i.e. pre-compiled) sample circuit.

property target: str

Returns: The name of the target that the sample was submitted to.

class supermarq.qcvv.XEB(single_qubit_gate_set: list[cirq.Gate] | None = None, two_qubit_gate: cirq.Gate | None = cirq.CZ, *, random_seed: int | numpy.random.Generator | None = None)

Bases: supermarq.qcvv.base_experiment.BenchmarkingExperiment[XEBResults]

Cross-entropy benchmarking (XEB) experiment.

The XEB experiment can be used to estimate the combined fidelity of a repeating cycle of gates. In our case, where we restrict ourselves to two qubits, we use cycles made up of two randomly selected single qubit phased XZ gates and a constant two qubit gate. This is illustrated as follows:

For each randomly generated circuit, with a given number of cycle, we compare the simulated state probabilities, \(p(x)\) with those achieved by running the circuit on a given target, \(\hat{p}(x)\). The fidelity of a circuit containing \(d\) cycles, \(f_d\) can then be estimated as

\[\sum_{x \in \{0, 1\}^n} p(x) \hat{p}(x) - \frac{1}{2^n} = f_d \left(\sum_{x \in \{0, 1\}^n} p(x)^2 - \frac{1}{2^n}\right)\]

We can therefore fit a linear model to estimate the value of \(f_d\). We the estimate the fidelity of the cycle, \(f_{\mathrm{cycle}}\) as

\[f_d = A(f_{cycle})^d\]

Thus fitting another linear model to \(\log(f_d) \sim d\) provides us with an estimate of the cycle fidelity.

For more details see: https://www.nature.com/articles/s41586-019-1666-5

analyze_results(plot_results: bool = True) → XEBResults

Analyse the results and calculate the estimated circuit fidelity.

Parameters:

plot_results (optional) – Whether to generate the data plots. Defaults to True.

Returns:

The final results from the experiment.

plot_results() → None

Plot the experiment data and the corresponding fits.

property circuit_fidelities: pandas.DataFrame

The circuit fidelity calculations from the most recently run experiment.

Raises:

RuntimeError – If no data is available.

property samples: collections.abc.Sequence[XEBSample]

The samples generated during the experiment.

Raises:

RuntimeError – If no samples are available.

single_qubit_gate_set: list[cirq.Gate]

The single qubit gates to randomly sample from

two_qubit_gate: cirq.Gate | None

The two qubit gate to use for interleaving.

class supermarq.qcvv.XEBResults

Bases: supermarq.qcvv.base_experiment.BenchmarkingResults

Results from an XEB experiment.

cycle_fidelity_estimate: float

Estimated cycle fidelity.

cycle_fidelity_estimate_std: float

Standard deviation for the cycle fidelity estimate.

experiment_name = 'XEB'
class supermarq.qcvv.XEBSample

Bases: supermarq.qcvv.base_experiment.Sample

The samples used in XEB experiments.

sum_target_cross_sample_probs() → float

Compute the dot product between the sample and target probabilities

Raises:

RuntimeError – If either the target or sample probabilities have not yet been initialised.

Returns:

The dot product between the sample and target probabilities.

Return type:

float

sum_target_probs_square() → float

Compute the sum of the squared target probabilities.

Raises:

RuntimeError – If no target probabilities have been initialised.

Returns:

The sum of squared target probabilities.

Return type:

float

sample_probabilities: dict[str, float] | None = None

The sample probabilities obtained from the chosen target

target_probabilities: dict[str, float] | None = None

The target probabilities obtained through a noiseless simulator