Tech09 <a href="https://aitoday7.com/7-best-tech-gadgets-of-august-2026/" title="7 Best Tech Gadgets of August 2026“>August 2026ByEsra Öz
Add ScienceAlert on Google
(Tek Image/Science Photo Library/
An AI system could help scientists identify a promising new drug. But it could also convince them that a biological effect exists when it does not.
Generative AI creates new content by learning common features and relationships from existing examples.
Although the technology is best known for producing text and images, researchers are exploring its use in designing proteins, simulating cells, filling gaps in experimental results, and generating synthetic biological data.
But these systems can hallucinate.
In biological research, this could mean generating a plausible-looking molecular pattern or inference that does not reflect the underlying biology.
Such an error could have tangible consequences.
AI might disregard a drug candidate that would have worked, direct researchers toward an ineffective treatment, conceal a genuine biological effect, or make a nonexistent disease mechanism look like a discovery.
Computational biologist Thomas Burger of Grenoble Alpes University in France explores that problem across 10 potential uses of generative AI in an Opinion article published in Patterns.
Omics experiments can generate vast datasets containing measurements of genes, proteins, and other molecules. AI could help researchers make sense of this enormous volume of information, but subtle changes introduced into such complex data may be difficult to detect.
Burger proposes that these applications do not all carry the same level of risk.
The key difference is whether an AI output is an idea that will later be tested in a real experiment or synthetic data used directly as evidence.
“I have never thought about that so far, but I guess it is possible to have hallucinations that lead to genuine discoveries.” – computational biologist Thomas Burger
Screening potential drugs or proteins is among the comparatively lower-risk applications. A model could rapidly assess a large number of candidates and select a smaller group for laboratory testing.
If the AI makes a mistake, researchers might discard a candidate that would have worked or waste time and money investigating one that ultimately fails. But the selected candidate would still have to demonstrate its effects in a real experiment before being accepted as a discovery.
The danger increases when AI-generated data begins replacing experimental measurements.
Synthetic biological data could help fill in missing measurements, protect patient privacy, create comparison groups, lower research costs, or reduce the number of animals used in experiments.
But if AI inserts a feature that was never present, scientists could believe they had discovered a biological effect that never occurred. The system would no longer be making only an incorrect prediction about an experiment. Its fabrication would have entered the evidence supporting a scientific claim.
“Most of the time, the problem is not about comparing a hallucination and a genuine biological discovery side by side,” Burger told ScienceAlert.
“It is more about real data having been corrupted by hallucination along the course of the complex computational (genAI-aided) workflow that makes it possible to turn raw signals acquired with complex biotechnologies into biologically valid descriptions of molecular mechanisms.”
In other words, while processing genuine data, AI could alter a signal in a way that is difficult to detect. The change could affect the researchers’ conclusion without creating a separate, obviously fabricated finding.
“If, along the process, some signals are distorted, amplified, or changed in any direction that may lead to different final biological conclusions, the investigator will have trouble noticing it unless they have a deep understanding about how the genAI has worked,” Burger said.
A real-world example emerged with AlphaFold 3.
In a 2024 paper in Nature, AlphaFold 3’s developers reported that the model could generate “hallucinated structures” in disordered protein regions, although low confidence scores can alert researchers to the problem.
AI errors may not always make a nonexistent effect appear real. A model might instead add so much distortion to the data that researchers overlook a genuine effect, potentially missing evidence that a treatment actually works.
Burger had not previously considered whether an AI hallucination could lead to a real discovery.
“I have never thought about that so far, but I guess it is possible to have hallucinations that lead to genuine discoveries.”
He compared this possibility with unexpected discoveries arising from laboratory errors.
“Serendipity has long been acknowledged; whether it originates from genAI hallucination or any other wet-lab mistake should not matter in the end, both from a moral viewpoint and from the expected posterior validation level,” Burger said.
What matters is how researchers use the output. If it is treated as an idea to test, a hallucination may remain only a failed hypothesis. If it is treated as a genuine observation, a convincing fabrication could enter the evidence and be mistaken for biological reality.
Even the most exciting result proposed by AI is not a discovery until it is independently verified in a real experiment.
The article was published in Patterns.
This article was fact-checked by Rebecca Dyer and edited by Rebecca Dyer. While we pride ourselves on our process, we are only human. If you spot a mistake, please let us know.
