AI’s shadow medical system| STAT

Key Points:

• Recent studies claim AI outperforms physicians on clinical reasoning tasks, fueling headlines that suggest AI could replace doctors, which has led to a growing consumer-driven medical system outside traditional clinics.
• Millions of Americans use AI chatbots like ChatGPT for health questions daily, often without medical disclaimers or guidance towards appropriate care, raising concerns about patients acting on AI-generated advice without professional oversight.
• While AI models perform well on isolated clinical tasks, they lack the nuanced, hidden clinical reasoning and accountability that experienced physicians provide, and their decision-making processes remain largely opaque to developers.
• A recent study found AI models correctly diagnose cases over 90% of the time with full information but fail to generate comprehensive differentials early in clinical encounters, highlighting risks when AI is used without clinical context or physical exams.
• The authors emphasize AI's potential as a supportive tool to enhance medical care and doctor-patient relationships but warn against unregulated AI systems that mimic medical authority without assuming clinical responsibility or accountability.