You might think it’s a terrible idea to ask AI chatbot for medical advice, but hundreds of millions of people do it every week.
In a near-death instance, now a subject of suit, ChatGPT advised a pastor to ignore the symptoms of pulmonary embolism—a condition with a 30% mortality rate. In fact, the advice he offered probably made things worse…
AI chatbots can be useful in many ways, but using them as a substitute for a doctor’s consultation is not one of them. Given their track record of reliably hallucinating, even using them instead of a Google search is not recommended.
However, even OpenAI’s own records show that hundreds of millions of people ask ChatGPT health questions every week. Similarly, an analysis of searches on Microsoft’s Copilot showed that health-related questions are among the most frequently asked questions on the mobile app.
If you needed any proof of the potential consequences of relying on a chatbot for medical advice, the Florida priest’s lawsuit provides an illuminating example. The New York Times reports that ChatGPT is suing OpenAI after it told him there was no need to worry about pulmonary embolism symptoms.
The lawsuit alleges that ChatGPT convinced Scott Winters that the early warning signs of a health crisis were “nothing dangerous” and discouraged him from seeking medical advice, telling him instead that “God didn’t design your body for endless failure.”
ChatGPT advised him to just rest. Not only was it not advisable to seek medical attention, but prolonged immobility is the worst thing you can do in these conditions – and I say this as someone who suffered a pulmonary embolism after a 12-hour flight with a broken leg at a time when the risks are known to researchers but not yet communicated to patients.
When Winters was finally admitted to the intensive care unit, doctors said his symptoms likely reflected a series of pulmonary embolisms that had been exacerbated by the weeks he spent sitting because he followed ChatGPT’s recommendations.
Winters was reportedly using GPT-4o, an older ChatGPT model noted for its sycophantic responses. This likely played a role in references to trust in God, but whatever AI model you use, please don’t let it give you medical advice.
The author of the photo Ben Weeks about Splash
FTC: We use automatic affiliate links that generate income. More.
