OpenAI appears to have made a subtle error when publishing its proofs of the Navier-Stokes problem, a team of mathematicians has claimed. The error doesn’t mean that the proofs are incorrect or that OpenAI hasn’t correctly solved the problem, but it does call into question whether mathematical results generated by AI models can always be relied on. On 8 September, OpenAI announced that it had found a solution to the Navier-Stokes problem, one of the most famous open problems in mathematics.
It published the proof in two versions – one written in “natural language”, meaning a combination of English and mathematical symbols, as a human mathematician would write, and another written in the computer code Lean. The Lean proof is meant to be a formalisation of the natural-language version, allowing a computer to mechanically verify that all of its logical statements are true. The problem is, say Hansen and his team, that they don’t match.
But what we’ve shown in this paper is that using this type of AI auto-formalisation can’t serve the same purpose.” The most interesting mathematical discoveries in OpenAI’s 722 new papers To be clear, the researchers aren’t saying that OpenAI has failed to solve the Navier-Stokes problem. It is entirely possible that both the natural-language proof and the Lean one provide a solution, just as there are hundreds of valid proofs of Pythagoras’s theorem. Instead, their point is a more subtle one: that OpenAI’s model has “mistranslated” when converting into Lean.
This mistranslation occurs because the AI has to produce a Lean proof that “compiles”, meaning that the computer code is fully self-consistent and doesn’t produce an error, says Hansen. If, in the process of auto-formalisation, the AI finds a section of the proof that doesn’t compile, it will attempt to find a workaround even if it means diverging from the proof as written in natural language. OpenAI has dumped 722 maths papers – now it must clean up the mess The team’s specific claim hinges on part of the proofs called Lemma 8.6.
In the natural-language proof, an equation in this part requires that a certain value be below m + 4, where m is a whole number. In the Lean proof, the equivalent value is required to be below m + 5, which is mathematically weaker. To understand why, imagine being asked to solve the equation x + 3 = 6, to which the answer is x = 3.
It is possible to write a proof that x must be less than 4, and also that x must be less than 5. Both of these are perfectly true mathematical statements, but they say different things. The latter proof allows more possible answers for x, making it mathematically weaker.
Extract — continue reading at the source.
