The results of most scientific papers reporting on advances in quantum computers can’t be replicated, according to a new analysis. This may undermine the credibility of some of the achievements that the quantum computing industry has reported in recent years, kicking off a replication crisis in the fast-growing field. There are a few problems that researchers are certain only a quantum computer could solve, but what these machines could be useful for beyond those select few cases is an open question.
With quantum computing hardware improving rapidly in recent years, researchers across the globe are exploring myriad use cases, from simulating molecules to optimising airline logistics. Yet, for any of these to become routinely used, they will have to work on many different quantum computers, so any demonstration of a truly valuable use of a quantum computer must be reproducible. Quantum computers have finally arrived, but will they ever be useful?
Wolfgang Mauerer at the Technical University of Applied Sciences Regensburg in Germany and his colleagues have now evaluated thousands of scientific papers on quantum computing and found that, currently, most can’t be reproduced. They carried out their analysis in two parts, first manually evaluating a curated sample of 127 papers from the past five years, then using a computer program to automate and generalise this analysis to 4966 papers. To assess the reproducibility of each paper’s results, the team used five criteria.
The first three focused on determining whether the paper included code that could be run on an independent quantum computer and how much instruction and documentation for doing so was provided. The next criterion evaluated available information about hardware. Finally, the team tested whether the provided quantum computing program could run without errors.
Among the 127 papers that the researchers analysed manually, only 24.4 per cent provided code that they could even try running, and 64.5 per cent of that code failed to successfully execute. The larger, automated analysis didn’t include an execution step, but similarly found that only 26.8 per cent of the almost 5000 papers provided enough information to attempt a replication. Compared with standards for traditional computer science studies, this is a rather negative result, says Mauerer.
For conventional computers, researchers can work at a very abstract level, describing and writing code without worrying about whether their computer’s physical characteristics may change from one day to another. This isn’t the case for quantum computers that currently exist and can, for example, be accessed through the cloud, says Mauerer. Quantum machines in themselves are unusually variable,” he says.
Extract — continue reading at the source.
