From Fox News:
Dr. AI will see you now.
It might not be that far from the truth, as more and more physicians are turning to artificial intelligence to ease their busy workloads.
Studies have shown that up to 10% of doctors are now using ChatGPT, a large language model (LLM) made by OpenAI — but just how accurate are its responses?
A team of researchers from the University of Kansas Medical Center decided to find out.
“Every year, about a million new medical articles are published in scientific journals, but busy doctors don’t have that much time to read them,” Dan Parente, the senior study author and an assistant professor at the university, told Fox News Digital.
“We wondered if large language models — in this case, ChatGPT — could help clinicians review the medical literature more quickly and find articles that might be most relevant for them.”
For a new study published in the Annals of Family Medicine, the researchers used ChatGPT 3.5 to summarize 140 peer-reviewed studies from 14 medical journals.
Seven physicians then independently reviewed the chatbot’s responses, rating them on quality, accuracy and bias.
The AI responses were found to be 70% shorter than real physicians’ responses, but the responses rated high in accuracy (92.5%) and quality (90%) and were not found to have bias.
Serious inaccuracies and hallucinations were “uncommon” — found in only four of 140 summaries.
“One problem with large language models is also that they can sometimes ‘hallucinate,’ which means they make up information that just isn’t true,” Parente noted.
“We were worried that this would be a serious problem, but instead we found that serious inaccuracies and hallucination were very rare.”
Out of the 140 summaries, only two were hallucinated, he said.
Minor inaccuracies were a little more common, however — appearing in 20 of 140 summaries.
Based on these findings, Parente noted that ChatGPT could help busy doctors and scientists decide which new articles in medical journals are most worthwhile for them to read.
. . . .
Dr. Harvey Castro, a Dallas-based board-certified emergency medicine physician and national speaker on artificial intelligence in health care, was not involved in the University of Kansas study but offered his insights on ChatGPT use by physicians.
“AI’s integration into health care, particularly for tasks such as interpreting and summarizing complex medical studies, significantly improves clinical decision-making,” he told Fox News Digital.
“This technological support is critical in environments like the ER, where time is of the essence and the workload can be overwhelming.”
Castro noted, however, that ChatGPT and other AI models have some limitations.
“Despite AI’s potential, the presence of inaccuracies in AI-generated summaries — although minimal — raises concerns about the reliability of using AI as the sole source for clinical decision-making,” Castro said.
Link to the rest at Fox News and thanks to F. for the tip.
