Why Artificial Intelligence Could Threaten The Future Of Science
Contributor.
In 1998, Dr. Andrew Wakefield, a British physician and researcher, was the lead author of a now-infamous scientific paper suggesting a possible connection between the measles, mumps, and rubella (MMR) vaccine and autism. Twelve years later, the respected medical journal The Lancet retracted the paper following serious questions about its findings and research integrity. Subsequent investigations uncovered evidence that Wakefield had misrepresented patient data, failed to disclose financial conflicts of interest, and committed serious ethical violations.
Despite the paper being debunked, millions continue to believe the false connection between vaccines and autism, a narrative that has shaped public policy and health outcomes for decades.
Coincidentally, the paper appeared in 1998, the same year Google was founded, and the Internet was beginning to transform how people discovered and shared information. While traditional media initially propelled the controversy, the expanding online world and large language models would eventually give misleading scientific claims unprecedented reach and longevity.
There’s little doubt that the Internet has been a spectacular boon to scientific collaboration, but it has also enabled a darker side that damages research and can cause real harm. And with AI making it easier than ever to produce convincing but potentially fraudulent research, the stakes are getting higher.
The Web Democratizes Science
In 1989, Sir Tim Berners-Lee, a British scientist, proposed the World Wide Web. Working at CERN, the European Organization for Nuclear Research, he recognized a need to connect thousands of scientists, computers, and data sources to support and advance their research efforts. Building on a concept called hypertext, his approach enabled users to click on links in documents to access information stored elsewhere in the world.
His simple idea created the Web we know today, and it has revolutionized global communications. Notably, he chose not to commercialize or patent his invention. In 1993, CERN made the software enabling the Web available royalty free.
Among its many benefits, the Web democratized access to scientific literature and supercharged research and development around the world. There’s little doubt that this capability has accelerated innovation in engineering, medicine, computing, physics, and more.
As the Web matured, researchers, universities, governments, and a growing number of publishers began embracing greater openness and easier access to scientific knowledge. After all, the obvious benefits of sharing and collaboration appeared to be in everyone’s interest.
While the benefits of easier access to science were clear, a largely unanticipated downside began to emerge, with real and sometimes harmful consequences.
The Dark Side Of Online Science
Kent Anderson and Joy Moore have been warning about the risks to scientific research from publishing business models increasingly shaped by the Internet and AI.
Both Anderson and Moore come from decades working in the scholarly publishing industry, a central player in the global research ecosystem. They have helped shape the design, delivery, and monetization of scientific publishing, particularly during its transition to the online world, working at organizations including The New England Journal of Medicine, Nature, and McGraw-Hill.
While they recognize the remarkable benefits of easier access to science online and AI in research, they’re sounding the alarm about some growing risks these technologies may have created.
Online publishing and AI has enabled more researchers to gain visibility and speed-to-publishing for their important work, but it’s also created opportunities for bad or even fake science to sneak in.
Overburdened review processes, new technologies, and evolving business models are among the reasons unreliable and fake research is finding its way into the scientific record.
The problem extends well beyond academia. In today’s connected world, unsubstantiated claims can quickly find their way into nightly news stories, TikTok posts, health advice from purported experts, and even public policy.
Today, AI makes it easier than ever to produce convincing but poorly researched manuscripts, fabricate supporting references, and generate scientific-looking material at scale. Bad actors have many reasons to do this including building fake reputations, championing an agenda, financial incentives, and publish-or-perish pressures.
Some of this content can find its way into legitimate publishing channels, while much more can circulate online without meaningful scientific review. Suddenly, a prominent online influencer is promoting its questionable conclusions to millions, and soon policymakers are paying attention.
It’s easy to see what motivates bad actors, but what about the scholarly publishing platforms? Anderson and Moore say the highly competitive attention economy has played an outsized role in changing behavior.
Historically, many prestigious scientific journals built their commercial success around selectivity, reputation, and institutional subscriptions. But the expansion of open-access publishing and a competitive marketplace has introduced new economic incentives. Under some author-pays models, publishers generate revenue from each accepted paper, creating a potential conflict between maintaining rigorous quality controls and increasing publication volume.
Protecting The Integrity Of Science
Anderson and Moore strongly believe that without action, the problem of publishing bad and fake science will only get worse. The number of academic papers published globally each year is growing rapidly and it’s already in the millions. The percentage of fake papers too is also way up. Compelled by the worsening situation, they recently published their book, How the Internet Disrupted Science, to draw attention to the problem and propose solutions.
As former leaders in the scholarly publishing industry, they understand the issues better than most. Their recommendations for positive change range from practical to aspirational. They believe publishers have a central role to play, given their position as trusted intermediaries between researchers and consumers of scientific literature.
On the practical side, they want to see the online journals return to focusing on the specific science communities they serve. Rather than prioritizing publication volume, they argue that journals should return to making high-integrity scientific content their primary focus.
More controversially, they suggest introducing some friction into how scientific papers reach the broader public, while ensuring researchers retain access. Ultimately supporting greater openness, they recognize that the general public isn’t necessarily equipped to understand and interpret a scientific paper. Anyone who’s ever seen one knows exactly what they mean.
Academic papers containing errors have always existed. What’s changed is the speed, scale, impact, and incentives for spreading poor-quality and fake science. Those motivated to hijack the scientific process now have more tools and greater reach than ever before. Ideological, political, and commercial agendas can influence scientific decision-making, with serious consequences for individuals and society.
The Internet made scientific information easier to distribute, commercial incentives made publication volume more attractive for some publishers, and AI now threatens to make unreliable research dramatically easier to produce.
Whatever the solution, the first step is recognizing the growing problem. After that, we’ll need to find ways to ensure technology continues to push the boundaries of scientific discovery without compromising the integrity of science itself.
Gathering Sources...