AI doctor's assistant swayed to change scrips
theregister.com • • health
Key Points:
- Security researchers at Mindgard found that Doctronic's healthcare AI can be easily manipulated to reveal and alter its system prompts, enabling it to spread misinformation or change behavior during a session.
- Although these manipulations are session-specific and typically do not persist across users, the AI's SOAP notes—used for clinical record-keeping and recommendations—can maintain some altered information, posing risks if reviewed carelessly by clinicians.
- Mindgard highlighted concerns that altered SOAP notes could lead to dangerous prescription changes, such as tripling an OxyContin dosage, potentially going unnoticed by overworked physicians due to the AI's high confidence in its recommendations.
- Doctronic and Utah state officials emphasized that safeguards in the current Utah pilot prevent controlled substance
