The words mathematicians reached for were not the usual ones. “Staggering.” “Overwhelming.” “Unprecedented.” “Surreal.” “Pure insanity.” More than three dozen researchers used them while trying to describe what OpenAI had just done to their field — and, in many cases, to their careers.
In a single release, the company published close to 400 AI-generated mathematical results, spread across more than 700 manuscripts. The material ranges over combinatorics, several branches of geometry, number theory, theoretical computer science, algebra, topology, probability and statistical mechanics, and mathematical physics. The collection is so vast that OpenAI felt the need to publish its own guidance on how to navigate the sprawling repository. Amid the awe, excitement and uncertainty over the sheer scale of the deluge was a deep-seated anxiety about what it all means — and what comes next.
A table of contents that takes an hour
Even a preliminary assessment of what has actually been released is difficult. In the hours and days after the drop, most researchers said they were still struggling to digest the material. Several reported that simply working through the roughly 40-page table of contents and abstracts took them the better part of an hour.
“Just going over the entire list of abstracts is overwhelming,” said Álvaro Lozano-Robledo, a professor of mathematics at the University of Connecticut.
Scattered among the hundreds of papers are formalisations written in Lean, a programming language and proof assistant that lets mathematical arguments be checked by machine. Lean has become an increasingly important tool in the discipline over the past decade; libraries such as mathlib now contain hundreds of thousands of lines of verified mathematics. Formal proofs have proved decisive in assessing some of OpenAI’s earlier claims, giving researchers a degree of confidence that a statement is logically correct even when they cannot follow the reasoning behind it.
Verification varies wildly from paper to paper
But the level of verification differs enormously across the collection. In the repository, OpenAI acknowledged that the results sit “at different stages of verification” and that “many, but not all, of the manuscripts have been formalised.” By its own count, only 300 top-line results out of 719 manuscripts — roughly 42 percent — had been formalised, with the company promising to update the repository as it obtains more.
Several mathematicians complained about the lack of formalisation given the sheer number of results, and stressed that even when Lean code accompanies a paper, evaluation is not instantaneous. Researchers must check that the formalisation actually proves what the manuscript claims, another time-consuming process. Some who dug through the papers said the quality of the computer-verifiable proofs was inconsistent and that the statements being verified did not always map neatly onto the claims made in the accompanying text.
“Hence, I either have to read possibly-not-correct slop, or wait for others to do the same, or wait for someone to formalise them before I can say for sure that the results are even correct.”
That assessment came from Kevin Buzzard, a mathematics professor at Imperial College London and one of the leading figures in the formalisation movement. He said he had identified numerous theorems in algebraic number theory, his own area, of which only around six immediately stood out — and few, if any, appeared to be formally verified. His concerns were echoed by numerous other researchers.
The ‘slopocalypse’ that did not quite arrive
Buzzard was far from alone in worrying about AI “slop” — shorthand for low-quality, frequently erroneous AI-generated material that has been increasingly appearing online, in the real world and in academic literature. Mathematicians say they have seen a sharp rise in such material produced with tools like ChatGPT and Claude in recent years. Much of it is confusing, hard to read and shows little understanding of the subject, and it is especially poor at crediting other researchers.
OpenAI’s earlier mathematical write-ups drew heavy criticism for exactly that: sloppiness and poor or nonexistent attribution. Before the release, several researchers had taken to calling the expected flood of papers the “slopocalypse,” or variations on the theme.
Whether that materialised is hard to judge, largely because of the bewildering volume involved. Early indications suggest OpenAI took more care this time, or at least with some of the papers. Several mathematicians said their first impressions were better than they had expected — though, by their own admission, that was a low bar given the company’s previous publications.
Better, however, is not the same as good, let alone up to the standards normally expected of academic work making claims of this magnitude. With formal verification missing for many of the results, the quality of the accompanying papers becomes crucial: they are the main route by which mathematicians can confirm, understand, scrutinise and contextualise the claims. Producing a rigorous mathematical paper is difficult work even under the best of circumstances. Doing it at this scale, across dozens of specialities and far faster than any human staff could review, is a formidable undertaking — and researchers say it shows.
Many described papers that were difficult, and sometimes practically impossible, to follow.
“The write-up of the problem I know best made little sense after a quick read. If this had been written by a person, I wouldn’t spend any more time trying to understand it. Of course, this leaves 721 other preprints!”
So said Brendan Hassett, a mathematics professor at Brown University, speaking before three papers were retracted. Others pointed to the unusual brevity of the work, with results they would ordinarily expect to be developed over hundreds of pages compressed into a few dozen or fewer. Some noticed papers that appeared to cover ground already trodden by other mathematicians, though they were wary of saying so publicly before a proper review.
Nalini Joshi, a mathematics professor at the University of Sydney, said a quick search revealed little overlap with her own work on integrable systems, but noted that some of the papers she examined “have short bibliographies.” Given past criticism of OpenAI’s crediting practices, she said she was “wary that attribution in the papers may be lacking the complete story.”
Genuinely impressive results
For all the slop and uncertainty, the overarching consensus is that the release contains some genuinely impressive mathematics. While stressing the difficulty of assessing the volume of material — and the need for proper verification — numerous researchers said the work appeared to be of a very high calibre despite shortcomings in its presentation. In a pre-AI world, they said, many of the results would clearly have warranted publication in top-tier journals and could have secured an academic career for their authors. A handful were described as the kind of work that could make a mathematician a serious contender for a Fields Medal, one of the discipline’s highest honours.
Jared Duker Lichtman, a mathematician at Stanford, said there were “tens” of results he would place in that category, including progress toward the Riemann hypothesis, a special case of the Hodge conjecture, and a solution to the four-dimensional Kakeya conjecture. These are major problems. Riemann and Hodge are both among the seven famed Millennium Prize problems, concerned respectively with the distribution of prime numbers and, very roughly, with how complex geometric shapes can be understood in terms of simpler building blocks. Kakeya asks how little space is needed to rotate a needle or pencil in every direction; a proof of the three-dimensional version was among the achievements that earned NYU mathematician Hong Wang a Fields Medal earlier this year.
“It’s not the case that these are just silly problems that no one’s ever heard of. Many of them are like very well-known problems that many people have tried for decades.”
That was the assessment of mathematician Scott Armstrong.
Careers upended overnight
As the dust settled, mathematicians confronted a landscape that had abruptly changed around them. Many had just watched years of work and carefully laid research plans evaporate, or knew colleagues who had. Whether reactions were laced with excitement, dread or despair, there was a profound sense of disorientation.
In Scotland, Colva Roney-Dougal, a professor of mathematics at St Andrews University, described a similarly bleak atmosphere among colleagues and students. The deluge felt somewhat “horrific,” she said, though its arrival brought some relief after weeks of uncertainty.
“I have a bunch of friends and colleagues whose grant proposals have just been wiped out.”
Armstrong said he knew of one group whose entire research programme was practically wiped out by the release, while Tristan Buckmaster, an NYU mathematician who was at the centre of an earlier dispute with OpenAI over the Navier-Stokes problem, said he had already heard of three people whose research programmes were obliterated. Francesco Fournier-Facio, a professor at Heriot-Watt University, said researchers in probability, combinatorics and theoretical computer science appeared “particularly in shock,” adding that some areas of group theory had been “bulldozed.”
Some fields appeared to have been targeted with almost military precision. Armstrong singled out work in the area for which Wang received the Fields Medal, as well as several Millennium Prize problems. A cluster of results in mathematical physics, he said, makes clear that OpenAI is pursuing Yang-Mills theory — the mathematical framework underpinning much of modern particle physics — and its unresolved “mass gap” problem, another of the seven Prize problems.
Elsewhere, researchers expressed surprised relief at how little their own work appeared to have been touched. Roney-Dougal joked that it was fortunate she works in a “very unfashionable area,” though she later messaged to say she was feeling unsure after finding her work cited in one of the papers. Joshi similarly found little overlap, suggesting her field is less driven by long-standing, formally stated conjectures and definitions and more by questions that wax and wane with developments in physics.
A threshold crossed
Whether their own work had been affected or not, most researchers shared a sense that the field had crossed some kind of threshold and was no longer what it had been a day earlier. Constantin Kogler, a researcher at the Institute for Advanced Study, called it the most important single moment in the history of mathematics. Yang-Hui He, a fellow at the London Institute for Mathematical Sciences, reached back millennia, comparing the year’s AI-driven developments to the publication of Euclid’s Elements, one of the discipline’s most influential works.
Few researchers were quite so grand in their assessments, but there was general agreement that mathematics is changing fast — and that mathematicians will have to change with it.
What OpenAI promised, and what it did
OpenAI knew the release would be disruptive and made some effort to soften the blow and engage with the mathematical community. After several bruising encounters with researchers earlier in the year, the company turned to mathematicians themselves for help, working with the newly formed Advisory Group on Mathematics and Artificial Intelligence (AGMAI) on how to release the results responsibly.
AGMAI had already laid out what it believed responsible engagement would look like. AI labs should ideally release papers that a human understands, or otherwise provide support for the additional mathematical activities needed for people to assimilate AI-generated output and identify possible applications of it. Proofs should be formalised where possible, and companies should make public what models and prompts they used. The group also urged labs to stop treating mathematical releases as marketing vehicles to promote their models and to stop testing advanced mathematical problems on proprietary models that are inaccessible to the broader scientific community.
OpenAI followed some of that advice. It said it would fund a series of workshops and conferences around its work in mathematics to help the community process and understand the output, though it provided no details as to what these might look like or when they may occur. It disclosed significantly more information than in previous releases — again, researchers noted, a low bar — including that its model attempted more than 4,000 problems and that a typical result used around three hours of ChatGPT Pro thinking compute. It also implemented protocols for paper revisions and citations and said it would preserve the public release history of the collection. By October 8th, that record already listed numerous corrections, including revisions on more than a dozen manuscripts and the removal of three papers due to a “sign error” that it says invalidates an argument.
The change addresses a long-running source of friction with mathematicians, some of whom speak of a kind of paranoia around the company’s published claims. Researchers point to a habit of quietly altering releases and papers in response to criticism without clearly disclosing those changes.
But OpenAI did not follow all of the group’s recommendations, including some of its most consequential. It did not identify the model behind the release, nor did it disclose the prompts used or the full set of problems the model attempted. It also appeared to acknowledge that its formalisations were lacking, promising to add more as it obtains them, and that the papers were subpar, saying it is committed to further improving the quality of papers through citations, mathematical exposition and presentation of results for better understanding in future releases.
Most importantly, the company indicated it has no plans to stop testing its models on mathematics problems, and indeed suggested it sees doing so as an imperative. It said it wants to directly empower scientists with state-of-the-art capabilities and is working to responsibly release the model that produced the results, arguing that continued evaluation of internal frontier models on mathematics and other sciences accelerates the development of tools to advance those fields. After the release, AGMAI described the publication as “a first step” and reiterated calls for equitable access to compute and research tools, arguing more fundamentally that the future of mathematical research cannot consist only of understanding results produced by AI labs.
Years of work, and a different kind of mathematics
For all the hundreds of long-standing problems OpenAI claims to have solved, mathematicians say an extraordinary amount of work remains. Many estimated that making sense of everything just released could take the community years.
Armstrong likened the sudden arrival of so much new mathematics to the commotion that might follow a brief extraterrestrial visit.
“If the AIs would disappear now, as though there were aliens that came to Earth and then just left, we would be studying this for the next 10 years, trying to understand everything.”
Much of that will be exciting in its own right. Researchers will have to fill in gaps, extract useful ideas and connections, and place solutions in a broader mathematical context — all things that typically go hand in hand with producing solutions in the first place. Lichtman said he was confident that relevant experts care deeply about the results and would be able to digest and present them to the wider world. Work involving explaining, verifying and contextualising results will need to be valued more as AI changes the field, he argued: as a society, we should be rewarding these digestive efforts more.
There may be plenty of mathematics left for humans as well. As far as Lichtman could tell, all of the results, impressive as they are, come out of existing methods and techniques, filling in parts of the known mathematical landscape rather than creating entirely new ones. Lozano-Robledo echoed the point: “They are all using existing techniques in very ingenious ways.”
And the existing map is hardly the limit. “Math research is essentially infinite,” Lozano-Robledo said. “Research will go on.” Yang-Hui He said he was particularly excited to see what emerges next, speculating that researchers may end up creating new ideas and perhaps even new fields of mathematics.
Few mathematicians objected in principle to AI producing new mathematics. Many welcomed it and, to varying degrees, said they were enthusiastic users of AI tools themselves. Most of the unease was directed at how the companies building those tools are going about entering their field.
Looking at the releases from OpenAI and other AI labs, it would be easy to conclude that research mathematics essentially amounts to checking problems off a list. The manner in which the results are unveiled reinforces the idea: a flashy announcement, a preliminary write-up, and the rest tossed to mathematicians to figure out and contextualise. Researchers described that as an impoverished view of how mathematical research actually functions. Solutions certainly matter, but so does everything that happens on the way to finding them — developing ideas, making connections, and finding new questions or
Source:The Verge News