AI bot is better than doctors at diagnosing patients in A&E, study finds
A study shows that an AI bot may be better at diagnosing patients in emergency care than real doctors.
Dutch researchers found that ChatGPT performed as well or better than real medical professionals when presented with the same medical certificates from a patient.
1
The chatbot could one day help doctors reduce wait times for patients, they suggested.
Dr. Hidde ten Berg from Jeroen Bosch Hospital said: “We found that ChatGPT worked well in generating a list of likely diagnoses and suggesting the most likely option.”
“We also found a lot of overlap with doctors’ lists of probable diagnoses.
“Simply put, this means ChatGPT was able to suggest medical diagnoses, similar to how a human doctor would.”
Launched last year by OpenAI, ChatGPT uses one of the most sophisticated language models ever developed.
Previous research has shown that he was able to pass the US medical licensing exam and has better bedside manner than real doctors.
The latest study, presented at the European Emergency Medicine Congress, examined how well it works in the emergency room.
The researchers anonymized information on 30 patients treated in the emergency room of the Jeroen Bosch Hospital in 2022.
They entered medical notes about patients’ signs, symptoms and physical examinations into two versions of ChatGPT – the free version 3.5 and the subscriber version 4.0.
The chatbots were asked to create a list of possible diagnoses for the patients, which were compared to real doctor lists.
Researchers found a large overlap – about 60 percent – between the shortlists created by ChatGPT and the doctors.
Doctors had the correct diagnosis within their five most likely diagnoses 87 percent of the time, compared to 97 percent for ChatGPT version 3.5 and 87 percent for version 4.0.
Dr. ten Berg said: “We included a case of a patient who presented with joint pain that was relieved with painkillers, but redness, joint pain and swelling recurred.”
“In the previous days, the patient had fever and a sore throat.
“A few times the fingertips became discolored. Based on the physical exam and additional testing, doctors believed the most likely diagnosis was probably rheumatic fever, but ChatGPT was correct with the most likely diagnosis being vasculitis.”
He added: “It is important to remember that ChatGPT is not a medical device and there are privacy concerns when ChatGPT is used with medical data.”
“However, there is potential for time savings and shorter waiting times in the emergency room.
“The use of artificial intelligence could be to support doctors with less experience or to help detect rare diseases.”
Independent experts said the study adds research that suggests AI could one day be used to assist doctors in hospitals.
Professor Youri Yordanov from St. Antoine Hospital Emergency Department said: “We are a long way from using ChatGPT in the clinic.
“But it is important that we explore new technologies and consider how they might help doctors and their patients.
“People who need to go to the emergency room want to be treated as quickly as possible and receive a correct diagnosis and treatment for their problem.
“I look forward to further research in this area and hope that it will ultimately support the work of busy healthcare professionals.”