How Accurate Is ChatGPT in Predicting Breast Cancer? - medtigo



How Accurate Is ChatGPT in Predicting Breast Cancer?

Share on facebook
Share on twitter
Share on linkedin
Share on whatsapp

As per US News, a new study reveals that ChatGPT, the AI chatbot everyone is talking about, can frequently provide accurate responses to queries about breast cancer. However, it is not yet equipped to replace your doctor.  

The major caveat, according to the researchers, is that the information is not always reliable or only tells a portion of the story. At least for the time being, they advised take your medical queries to a human physician.  

ChatGPT is a chatbot powered by artificial intelligence technology that enables it to have human-like conversations, generating instantaneous responses to virtually any user-generated query. These responses are founded on the chatbot’s “pre-training” with a massive amount of data, including internet-gathered information.  

According to a report from the investment bank UBS, the technology had a record-setting 100 million monthly consumers within two months of its debut in November 2017. ChatGPT has also reportedly aced the college SATs and the U.S. medical licensing exam, garnering widespread media attention. Despite the possibility that the chatbot is a doctor, it remains unclear whether it provides users with reliable medical information.  

The new study, which was published on April 4 in the journal Radiology, evaluated the chatbot’s ability to respond to “fundamental” queries regarding breast cancer screening and prevention.  

Overall, it was discovered that the technology provided accurate responses 88% of the time. It is unclear whether this would be superior to a Google search or your practitioner. Dr. Paul Yi, an assistant professor of diagnostic radiology and nuclear medicine at the University of Maryland School of Medicine, described the accuracy rate as “pretty remarkable.”  

However, Yi also pointed out the limitations of ChatGPT as it currently stands. In the case of health and medicine, he stated, even a 10% error rate could be detrimental. The ability of ChatGPT to rapidly assemble a variety of data into a “chat” is both its advantage and disadvantage. Its responses to complex queries, Yi said, are limited in scope. Therefore, even when technically accurate, they may present a skewed picture.  

When Yi’s team queried ChatGPT about breast cancer screening, they discovered this to be true. In the response, only the recommendations of the American Cancer Society were provided, omitting those of other medical organizations, which differ in some instances.  

Yi added that the average ChatGPT user may not be able to ask follow-up queries or determine whether the response is accurate. The conversational nature of ChatGPT, according to Yi, is an advantage of the technology over traditional internet searches.  

“The disadvantage is that it is difficult to determine whether the information is accurate,” he said. Obviously, Yi acknowledged the veracity of online information has always been a concern. The distinction with ChatGPT lies in its presentation. And the technology’s allure — that conversational tone — can be quite “persuasive,” according to Yi.  


“As with any novel technology,” he said, “I believe it should be viewed with skepticism.” For the study, Yi’s team compiled 25 of the most frequently asked queries by patients regarding breast cancer prevention and screening and then presented them to ChatGPT. Each inquiry was posed three times to determine if and how responses varied.  

The chatbot provided appropriate responses to 22 queries and unreliable answers to three. For one query — “Do I need to schedule my mammogram around my COVID vaccination? For two others, the answers on all three assessments were inconsistent.  

“How do I prevent breast cancer?” is one of these queries. — was extensive and intricate, with a great deal of information (both factual and false) circulating around the internet. Professor of statistics, operations, and data science at Temple University’s Fox School of Business in Philadelphia, Subodha Kumar, stated that this is crucial.  

He stated that the response would be more reliable the more specific the query. When a topic is intricate, and data sources are numerous and, in some cases, dubious, responses will be less reliable and more likely to be biased. And, according to Kumar, the more intricate the topic, the more likely ChatGPT is to “hallucinate.” He noted that this term is used to characterize the chatbot’s documented propensity to “make stuff up.”  

Kumar, who was not involved in the new study, emphasized that ChatGPT’s responses are only as accurate as the data it has been and continues to be fed. “There is no assurance that it will be provided only accurate information,” he said. Kumar noted that as time progresses, the chatbot will collect more data, including from users, so it is conceivable that its accuracy will deteriorate rather than improve.  

“This can be risky when the topic is health care,” he said. According to both researchers, ChatGPT and similar technologies hold tremendous promise. According to Kumar, the chatbot could be a useful “assistance device” for doctors seeking information on a topic quickly but who also possess the ability to place a response in context. Kumar stated, “I would advise against using this for health care information for the average consumer.” 


Free CME credits

Both our subscription plans include Free CME/CPD AMA PRA Category 1 credits.

Digital Certificate PDF

On course completion, you will receive a full-sized presentation quality digital certificate.

medtigo Simulation

A dynamic medical simulation platform designed to train healthcare professionals and students to effectively run code situations through an immersive hands-on experience in a live, interactive 3D environment.

medtigo Points

medtigo points is our unique point redemption system created to award users for interacting on our site. These points can be redeemed for special discounts on the medtigo marketplace as well as towards the membership cost itself.
  • Registration with medtigo = 10 points
  • 1 visit to medtigo’s website = 1 point
  • Interacting with medtigo posts (through comments/clinical cases etc.) = 5 points
  • Attempting a game = 1 point
  • Community Forum post/reply = 5 points

    *Redemption of points can occur only through the medtigo marketplace, courses, or simulation system. Money will not be credited to your bank account. 10 points = $1.

All Your Certificates in One Place

When you have your licenses, certificates and CMEs in one place, it's easier to track your career growth. You can easily share these with hospitals as well, using your medtigo app.

Our Certificate Courses