AI Chatbot Used to Create Fake Clinical Data, Raises Concern

Using the ChatGPT AI chatbot’s underlying technology, researchers conducted a groundbreaking experiment, creating a bogus clinical trial dataset to support a dubious scientific conclusion. The most recent version of the robust language model is GPT-4. Statistics created by artificial intelligence comparing the efficacy of two keratoconus procedures mislead researchers into assuming that one surgery was superior.  

Giuseppe Giannaccare, a research co-author and eye surgeon at the University of Cagliari in Italy, noted that the experiment was designed to demonstrate how simple it is to produce a dataset that is not based on the original data. Giannaccare emphasized that the generated data was intended to contradict existing evidence, emphasizing the possibility for AI to be abused to offer deceptive data. This finding raises new questions regarding bias among researchers and journal editors.  

Elisabeth Bik, a microbiologist and research-integrity consultant based in San Francisco, is concerned that AI has acquired the capacity to build believable datasets, which she describes as the “next level of worry.” She stressed how simple it is for scientists nowadays to fabricate data, whether it is false measurements, faked survey responses, or fabricated vast databases for fictitious studies.  

The collected data, according to the study’s authors, creates a “seemingly authentic database.” However, expert examination indicated that the data was forged, failing authenticity checks. Because of AI’s ability to construct plausible but phony datasets, new concerns have been put on the integrity of scientific conclusions.  

The efficacy of two different therapies for keratoconus, a corneal thinning condition, was investigated in this study. The AI-generated data backed one technique, but clinical research results contradicted this. The simulated dataset of 160 male and 140 female patients demonstrated that those who had the operation recommended by the AI model fared better. 

University of Manchester biostatistician Jack Wilkinson warned of the ease with which false copies of such datasets may be made to appear authentic. He did, however, emphasize the necessity of careful examination by conceding that a screening technique discovered differences in participant sex classification, abnormalities between pre- and post-operative features, and irregular age values.  

EMBO Reports’ main editor, Bernd Pulverer, voiced worry that the current status of peer review may fail to detect well-planned violations of integrity utilizing artificial intelligence-generated false data. He contended that uncovering such examples required upgrading journal quality standards. Wilkinson is in charge of a team charged with creating statistical and non-statistical techniques for evaluating dubious research.

He did, however, warn that breakthroughs in generative AI may eventually provide strategies to avoid these constraints. Finally, the researchers’ use of artificial intelligence to create bogus data for clinical studies raises fundamental questions about the possibility of manipulation in scientific research. Understanding the ramifications of these findings has increased the demand for cutting-edge quality standards and novel solutions to protect the integrity of scientific discoveries.  

Journal Reference  

Naddaf, Miryam. “ChatGPT Generates Fake Data Set to Support Scientific Hypothesis.” Nature News, Nature Publishing Group, 22 Nov. 2023, www.nature.com/articles/d41586-023-03635-w. Accessed 23 Nov. 2023.