The study was conducted by researchers at the Icahn School of Medicine at Mount Sinai, and the results were published in the scientific journal Nature. Researchers tested OpenAI’s virtual assistant with 60 complex clinical scenarios spanning 21 different branches of medicine.
These scenarios did not consist of simple health problems, but rather cases designed to measure the capacity to distinguish life-threatening situations.
Results: Successful in some cases, risky in others
According to the research results, ChatGPT gave successful results in cases with classic and distinct symptoms. For example, he was able to accurately assess conditions such as heart attacks or severe allergic reactions that present with typical chest pain.
However, in cases where the clinical picture was more uncertain or atypical, the system made serious errors. In about half of true emergency cases, the chatbot did not advise the user to go to the emergency room and instead suggested a more cautious approach.
According to experts, in real life such a suggestion could have dangerous consequences.
“Wait and see” advice may pose risks
The most important problem that researchers point out is that artificial intelligence has difficulty reading the symptoms and gives too many calming and cautious suggestions.
AI has suggested a “wait and observe” approach in many cases. However, because time is critical in some diseases, this approach can mean the difference between life and death.
One of the examples given in the study was an acute asthma attack. Although ChatGPT correctly identified the severe respiratory distress the user described, it recommended monitoring the situation at home rather than calling for emergency medical help.
According to the researchers, this error stems from a structural problem: the algorithm often underestimates the severity of the risk by avoiding alarmist language.
A more complex problem in mental health
The research also revealed an even more worrying picture in the field of mental health. While the AI provided over-alerts in some low-risk psychological situations, it could respond less well in situations where users openly expressed thoughts of self-harm or suicide.
This “reverse alert” situation particularly worries psychiatrists. Because the most vulnerable users may not be adequately protected by the algorithm’s security filters.
Experts: Artificial intelligence cannot replace the doctor
Despite the problems identified, the researchers are not saying that artificial intelligence should be rejected completely. According to experts, this technology can support the healthcare system in the future; For example, it can be used as an assistive tool in doctors’ triage processes.
But for now, the message is pretty clear: ChatGPT is not a doctor and cannot replace human clinical judgment.
Experts emphasize that users should contact healthcare professionals or emergency services, not artificial intelligence, when the following symptoms occur:
Constant chest pain
sudden shortness of breath
Thoughts of self-harm
According to researchers, human life should not be left to the decision of an algorithm that says “let’s wait and see.”
