A bust-up between mathematicians is fuelling fears that AI could undermine science
0 / 107
English
中文
A bust-up between mathematicians is fuelling fears that AI could undermine science A bank of white freezers hums away at a former watch factory in central London.
（暂无翻译）
Inside: dozens of small boxes with barcoded glass tubes, collectively containing one hundred billion genes. Just two boxes may hold 10 times more information than all the text on the internet, says Dr Oliver Vince, the facility's co-founder. His firm, Basecamp Research, plans to collect as much DNA as possible to create an artificial intelligence that can speak the language of biology, in the same way that US and Chinese tech firms have hoovered up written material online to create Large Language Models (LLMs), the basis of current AI systems.
（暂无翻译） （暂无翻译） （暂无翻译）
Vince, like many in business and academic research, believes AI will supercharge scientific research.
（暂无翻译）
But increasingly, it also comes with controversy. Last month, OpenAI, the creator of ChatGPT, said it had solved one of maths' great unsolved problems, relating to the famous Navier–Stokes equations.
（暂无翻译） （暂无翻译）
The company said its AI agents had cracked it in 88 hours. The body that runs a prize for whoever can solve the problem said it had "apparently been settled", but that its review would take time. But hours before the announcement, Tristan Buckmaster, a mathematician at New York University, went public with concerns.
（暂无翻译） （暂无翻译） （暂无翻译）
He formed an independent team with Levent Alpöge, a mathematician employed by day by the rival company Anthropic. They had been working towards the same goal, and kept their drafts in OpenAI's coding tool, Codex.
（暂无翻译） （暂无翻译）
Buckmaster said that his co-researcher was tipped off that news of their progress had "reached" OpenAI, though he did not specify how. He asked whether his research, stored in Codex, could have fed into OpenAI's own work, though he stressed he was "not accusing anyone of anything". Basecamp founder, Dr Oliver Vince, says just two boxes may hold 10 times more information than all the text on the internet OpenAI said that nobody at the company, human or AI, saw the other teams' work before it was published.
（暂无翻译） （暂无翻译） （暂无翻译）
Initially it said that while unlikely, it could not "rule out" that anonymised data from use of its products may have helped improve its models.
（暂无翻译）
Days later, it said Buckmaster's Codex prompts from the two months before the announcement "could not have influenced the system in any way". It's a complicated story; we still don't fully know what happened. But it highlights a growing issue about the use of AI in science.
（暂无翻译） （暂无翻译） （暂无翻译）
Usually, those words make us conjure up images of robots building terrible genetically-engineered viruses, or chemical weapons, to wipe out humanity. But there's a more subtle problem that hasn't attracted the same headline-grabbing attention: that AI could undermine faith in the scientific process, raising questions about the novelty and accuracy of research. And there's another concern. It's well known that AI chatbots like Claude or Gemini can sometimes hallucinate things that simply aren't true.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
They will tell you about a film that never existed, or a quote that was never said. Now there is evidence that this is increasingly happening in science. Eventually, it's feared, it will become difficult to separate true scientific discovery from AI-generated fiction. And all of this could threaten to erode - if not explode - the rock on which scientific research rests: trust. One threat is to scientific journals, the guardians of scientific integrity, which are seen to be failing to deal with a tsunami of AI-generated distortions.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
"Generative AI has basically disrupted the whole scientific process," says Dalmeet Singh Chawla, a science journalist who has spent a decade reporting on research misconduct.
（暂无翻译）
He tells the BBC that publishers' integrity staff - the people who screen submissions for accuracy and reliability - will need to "drastically increase [in number and effort] if they're anywhere near the fight".
（暂无翻译）
He says: "Right now, it seems like they're not."
（暂无翻译）
On the face of it, AI seems just the right tool to power scientific research. Scientists deal with huge amounts of information.
（暂无翻译） （暂无翻译）
Telescopes and particle accelerators, for example, collect billions of data points – too large for the brain to comprehend. AI can pick the scientific highlights from the surrounding humdrum. The poster child of scientific AI is the UK-based firm Google DeepMind, which in 2020 solved one of biology's biggest problems: how proteins in living things fold into their precise shapes. It may sound dull, but proteins are the tiny machines that make life work, and the shapes they fold into determine what they can do; from building cells and fighting infection to helping form the tissues and organs of the body.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
Before AI, scientists had to conduct time-consuming experiments in the lab.
（暂无翻译）
Then, Google DeepMind developed an AI model called AlphaFold that accurately predicted 200 million protein folds in just one year.
（暂无翻译）
The equivalent experimental work would have taken hundreds of millions of years in laboratories. The firm gave the results away for free to researchers.
（暂无翻译） （暂无翻译）
The head of DeepMind and the project's lead scientist were both awarded the Nobel Prize for Chemistry. Dr Pushmeet Kohli, who leads DeepMind's science work, told me that AlphaFold could be seen as a "watershed moment". But elsewhere, the use of AI in science has come with accusations of hype.
（暂无翻译） （暂无翻译） （暂无翻译）
Some computer models are accused of exaggerating the significance of their findings, with the human researchers that oversee them then believing some of the claims without enough scepticism.
（暂无翻译）
Basking in its success over AlphaFold, for example, DeepMind turned its attention to predicting the structure of new materials. The work could lead to the development of better batteries, cheaper solar cells and electronic components that are not dependent on rare earth elements. (In theory, no-one would need to argue about mining rights ever again, and Greenland would be safe from invasion.) DeepMind chief executive Sir Demis Hassabis celebrated his company's use of AI to solve one of biology's biggest problems A DeepMind model came up with what the firm claimed were more than two million new crystal structures, announced as "nearly 800 years worth of knowledge" released in one go.
（暂无翻译） （暂无翻译）
Of those, some 380,000 were considered promising enough to make in a laboratory. But then came a warning sign from two chemists at the University of California.
（暂无翻译） （暂无翻译）
Anthony Cheetham and Ram Seshadri went through a random sample of the supposed new crystals.
（暂无翻译）
Writing in the journal Chemistry of Materials, they concluded that they had "yet to find any strikingly novel compounds", arguing that many were "trivial adaptations of known materials". Their verdict was not that the method was wrong, but that the results need more scrutiny and validation from human chemists.
（暂无翻译） （暂无翻译）
DeepMind rejects the charge, saying it tested each compound for novelty. Dismissing a new structure for looking familiar misses the point, the company argues: swapping a single atom is how many of today's most useful materials arrived, from phone components to sunscreen. In those cases, at least, the science was mostly real and the issue was just AI potentially exaggerating its importance. Arguably of even greater concern is the growing risk that scientific journals will become polluted with so-called AI "slop" - where a computer, not a human scientist, is writing the research, leading to fake citations and phantom papers.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
In 2024, computer scientist Prof Dmitry Kobak, of Ghent University, searched more than a million biomedical research papers for trigger words favoured by the LLMs that generate AI writing. His team found that by December last year, nearly 90% had AI fingerprints on them.
（暂无翻译） （暂无翻译）
AI detection technology is not perfect; some forms of the software have labelled the US Constitution, written in the 1780s, as "AI-generated".
（暂无翻译）
But still, Kobak said he was shocked by the scale of apparent AI use.
（暂无翻译）
There is a risk, he told Nature, that "whatever bias the LLM may have will just suddenly permeate the (scientific) literature". Kobak is among many researchers who worries that AI is slowly killing scientific journals. And it matters: research papers are the lifeblood of scientific advance. When a discovery is published, another team in another part of the world might obtain funding to build on the results. Pulsars - an unusual type of neutron star, depicted above by an artist - were discovered in the 1960s partly due to a hunch of scientists The process of peer review - in which research papers are checked by other scientists before they are published - is also being compromised.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
A team led by Weixin Liang and James Zou at Stanford University analysed referee reports submitted to four major computer science conferences, and found up to 17% of the text had been substantially modified by a language model.
（暂无翻译）
A critical part of a scientific paper's integrity involves lead scientists listing earlier research in order to show what they have done is genuinely new. These are called citations.
（暂无翻译） （暂无翻译）
A separate team at Cornell University found that researchers were using AI to compile citations, and that it was inventing bogus research, with around 147,000 pieces of phantom research cited last year.
（暂无翻译）
If AI continues to distort the integrity of the academic process in this way, then researchers will be sent up blind alleys, public money will be wasted, and the fruits of valid research - such as new drugs or green technologies - will be slowed or not realised at all.
（暂无翻译）
Some publishers of scientific journals, such as Elsevier, are taking steps to avert this growing crisis of confidence.
（暂无翻译）
Elsevier's research integrity team has developed some 100 signals that flag a paper for closer human scrutiny. Nature's journals forbid AI-generated images and tell referees never to upload a manuscript into an AI system. The website arXiv, where scientists share their early-stage results, bans researchers for a year if they submit invented references.
（暂无翻译） （暂无翻译） （暂无翻译）
Prof Sarah Main, an Elsevier vice-president, is among those at the firm trying to work out how to responsibly use AI in research.
（暂无翻译）
"What we're calling research-grade AI would focus on ensuring the researcher is in the driving seat," she says.
（暂无翻译）
Scottish scientist Sir Alexander Fleming discovered penicillin in the 1920s As AI agents process the mass of data emerging from particle accelerators, the cosmos and the natural world, how do we know they are getting it right? And there's another worrying question: how do we know that AI agents are not dismissing an important result as an anomaly? The discoveries of penicillin, radioactivity and weird stars called pulsars all came from odd results that could easily have been dismissed.
（暂无翻译） （暂无翻译） （暂无翻译）
Hunches by humans made them into some of the biggest scientific moments of the last century. Can AI really replicate that sense of serendipity and human ingenuity?
（暂无翻译） （暂无翻译）
Kohli of DeepMind does not think the answer is to turn our backs on AI.
（暂无翻译）
That would be "like saying you should be computing without using calculators", he told me.
（暂无翻译）
But he accepts there should be human guardrails: "The agents today are imperfect. Somebody has to take accountability." His test is a simple one: "You have to understand what the agent has done, and you have to take responsibility.
（暂无翻译） （暂无翻译） （暂无翻译）
You can't just specify the problem and let the machine write the paper and submit it as true."
（暂无翻译）
Why some experts increasingly fear AI will take over AI is creating a 'human premium' for art made by people. But will it hold? Tech billionaires seem to be doom prepping. Should we all be worried?
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
Prof Andrea Pauli of the Vienna BioCenter in Austria has followed that test. She spent almost 10 years on a question no-one has yet been able to answer: how a sperm gets into an egg.
（暂无翻译） （暂无翻译）
Responsible use of AI helped her toward an answer.
（暂无翻译）
Her lab found a protein on the surface of zebrafish eggs which pointed to a lock-and-key mechanism for allowing the sperm in. Hers is a case for hope, in a sea of AI scientific slop.
（暂无翻译） （暂无翻译）
It shows that AI can help. This is also a story about humans.
（暂无翻译） （暂无翻译）
Soon after the formation of Basecamp - the lab I visited in a former London watch factory - its co-founder, Oliver Vince, was ill in hospital with leukaemia (from which he has recovered). It motivated him to develop an AI system that stood a better chance of curing cancers. It might not work, he says, but it's worth a shot. "We have a technically feasible path to curing cancer," he says.
（暂无翻译） （暂无翻译） （暂无翻译） （暂无翻译）
Not a promise from Vince, but a direction. The big question is whether AI carries the science there, or quietly erodes the bricks it is built on.
（暂无翻译） （暂无翻译）
BBC InDepth is the home on the website and app for the best analysis, with fresh perspectives that challenge assumptions and deep reporting on the biggest issues of the day. Emma Barnett and John Simpson bring their pick of the most thought-provoking deep reads and analysis, every Saturday. Sign up for the newsletter here Are you personally affected by the issues raised in this story?
（暂无翻译） （暂无翻译） （暂无翻译）
仅供学习使用，如有侵权请联系删除 · 查看原文