AI Chatbots Are Bad at Diagnosing Symptoms For a Surprising Reason, Study Finds

Key Points:

• A recent study found that widely available AI chatbots are not reliable for diagnosing common medical conditions or advising on appropriate care, with users performing worse than those using traditional information sources.
• Although large language models (LLMs) demonstrate strong medical knowledge and can pass licensing exams, their effectiveness declines when interacting with real users due to communication issues and incomplete or misunderstood information.
• The study highlights the complexity of clinical care, which involves human qualities like empathy, judgement, and tailored communication that current AI chatbots cannot replicate.
• AI's most effective role in healthcare today is supportive, such as organizing information, summarizing records, and drafting clinical documents, rather than acting as frontline diagnostic tools.
• Policymakers should carefully evaluate the real-world performance of AI technologies before integrating them into critical healthcare settings, as impressive benchmark results may not translate into practical clinical use.