From Fox News:
Dr. AI will see you now.
It might not be that far from the truth, as more and more physicians are turning to artificial intelligence to ease their busy workloads.
Studies have shown that up to 10% of doctors are now using ChatGPT, a large language model (LLM) made by OpenAI — but just how accurate are its responses?
A team of researchers from the University of Kansas Medical Center decided to find out.
“Every year, about a million new medical articles are published in scientific journals, but busy doctors don’t have that much time to read them,” Dan Parente, the senior study author and an assistant professor at the university, told Fox News Digital.
“We wondered if large language models — in this case, ChatGPT — could help clinicians review the medical literature more quickly and find articles that might be most relevant for them.”
For a new study published in the Annals of Family Medicine, the researchers used ChatGPT 3.5 to summarize 140 peer-reviewed studies from 14 medical journals.
Seven physicians then independently reviewed the chatbot’s responses, rating them on quality, accuracy and bias.
The AI responses were found to be 70% shorter than real physicians’ responses, but the responses rated high in accuracy (92.5%) and quality (90%) and were not found to have bias.
Serious inaccuracies and hallucinations were “uncommon” — found in only four of 140 summaries.
“One problem with large language models is also that they can sometimes ‘hallucinate,’ which means they make up information that just isn’t true,” Parente noted.
“We were worried that this would be a serious problem, but instead we found that serious inaccuracies and hallucination were very rare.”
Out of the 140 summaries, only two were hallucinated, he said.
Minor inaccuracies were a little more common, however — appearing in 20 of 140 summaries.
Based on these findings, Parente noted that ChatGPT could help busy doctors and scientists decide which new articles in medical journals are most worthwhile for them to read.
. . . .
Dr. Harvey Castro, a Dallas-based board-certified emergency medicine physician and national speaker on artificial intelligence in health care, was not involved in the University of Kansas study but offered his insights on ChatGPT use by physicians.
“AI’s integration into health care, particularly for tasks such as interpreting and summarizing complex medical studies, significantly improves clinical decision-making,” he told Fox News Digital.
“This technological support is critical in environments like the ER, where time is of the essence and the workload can be overwhelming.”
Castro noted, however, that ChatGPT and other AI models have some limitations.
“Despite AI’s potential, the presence of inaccuracies in AI-generated summaries — although minimal — raises concerns about the reliability of using AI as the sole source for clinical decision-making,” Castro said.
Link to the rest at Fox News and thanks to F. for the tip.
This is the latest PBS NOVA about the issue.
A.I. Revolution | Full Documentary | NOVA | PBS
https://www.youtube.com/watch?v=-sB12gk9ESA
A bit of context from 2021, when Microsoft bought NUANCE (two years after first investing in OpenAI) for $20B:
https://www.thestreet.com/investing/microsoft-buys-nuance-in-20-billion-cloud-healthcare-takeover#:~:text=Microsoft%20said%20it%20sees%20the%20takeover%20as%20adding,priority%2C%20and%20healthcare%20is%20its%20most%20urgent%20application.
Microsoft said it sees the takeover as adding to its overall earnings starting from 2023.
“Nuance provides the AI layer at the healthcare point of delivery and is a pioneer in the real-world application of enterprise AI,” said CEO Satya Nadella. “AI is technology’s most important priority, and healthcare is its most urgent application. Together, with our partner ecosystem, we will put advanced AI solutions into the hands of professionals everywhere to drive better decision-making and create more meaningful connections, as we accelerate growth of Microsoft Cloud in Healthcare and Nuance.”
“The pair formed a strategic partnership in October 2019 to accelerate so-called ambient clinical intelligence, or ACI, technologies that uses AI to listen to doctor-patient conversations and automatically record them into electronic health record (EHR) systems.”
—-
Healthcare is one area where every bit of savings in time and money is important. So AI has long been an area of interest. And will be eagerly adopted in every function it can fit. Today it means reading x-rays, sifting medical reports that might interest the doctor, suggesting potential diagnoses, even teaching better bedside manner (seriously).
10% of doctors using ChatGPT is but the tip of the iceberg of *existing* AI use in health care. Way more is in store.