Using artificial intelligence chatbots to seek medical advice can be dangerous because of their tendency to provide inaccurate and inconsistent information, according to a new study by researchers at the University of Oxford.
The research, led by scientists from the Oxford Internet Institute and the Nuffield Department of Primary Care Health Sciences, was published in the journal Nature Medicine.
Researchers asked nearly 1,300 participants to identify possible health conditions and recommend next steps across a range of medical scenarios. Some participants used large language model software to generate potential diagnoses and advice, while others relied on traditional methods, including consulting a general practitioner.
After evaluating the responses, the researchers found that AI tools often delivered a mix of accurate and inaccurate information that users struggled to differentiate. The study concluded that while chatbots can perform well on standardized medical knowledge tests, their use as a clinical support tool poses risks to individuals seeking help for personal symptoms.
Dr. Rebecca Payne, a co-author of the study and a general practitioner, said the findings show that AI is not yet ready to replace physicians.
“Patients need to be aware that asking a large language model about their symptoms can be dangerous,” Payne said. “These systems can give incorrect diagnoses and fail to recognize when urgent medical attention is required.”
The researchers noted that the challenge lies not only in medical knowledge but also in human interaction. Lead author Andrew Bean of the Oxford Internet Institute said that even top-performing large language models struggle in real-world interactions involving nuanced, high-stakes decisions.
“These findings highlight the difficulty of building AI systems that can genuinely support people in sensitive areas like health,” Payne said.
Bean said the research aims to contribute to the development of safer and more reliable AI systems, particularly as interest in using chatbots for health advice continues to grow.
