{"id":6748,"date":"2026-02-27T19:08:48","date_gmt":"2026-02-27T19:08:48","guid":{"rendered":"https:\/\/foreignnewstoday.com\/?p=6748"},"modified":"2026-02-27T19:08:48","modified_gmt":"2026-02-27T19:08:48","slug":"where-chatgpt-health-fails-and-how-it-could-turn-deadly","status":"publish","type":"post","link":"https:\/\/foreignnewstoday.com\/?p=6748","title":{"rendered":"Where ChatGPT Health fails \u2014 and how it could turn deadly"},"content":{"rendered":"<p><br \/>\n<br \/><\/p>\n<div>\n<p>Don\u2019t call it ChatEMT.<\/p>\n<p>OpenAI last month <a href=\"https:\/\/openai.com\/index\/introducing-chatgpt-health\/\" target=\"_blank\" rel=\"noopener\">introduced ChatGPT Health<\/a>, a dedicated space in ChatGPT that allows users to ask health questions, analyze their medical records and connect to wellness apps.<\/p>\n<p>Now, weeks after its launch, researchers from the\u00a0<a href=\"https:\/\/icahn.mssm.edu\/\" target=\"_blank\" rel=\"noopener\">Icahn School of Medicine at Mount Sinai<\/a> are raising concerns that the AI tool often fails to recommend urgent care in emergency cases and sometimes misses suicide-crisis alerts.<\/p>\n<figure class=\"nyp-slideshow-modal-image wp-block-image aligncenter size-large\"><figcaption>With millions of people asking ChatGPT health questions every day, OpenAI recently launched ChatGPT Health. <span class=\"credit\">OpenAI<\/span><\/figcaption><\/figure>\n<p>\u201cChatGPT Health performed well in textbook emergencies such as stroke or severe allergic reactions,\u201d<span style=\", Helvetica, Arial, sans-serif;letter-spacing: normal\">\u00a0Dr. Ashwin Ramaswamy, instructor of urology at the Icahn School of Medicine at Mount Sinai<\/span>, <a href=\"https:\/\/www.mountsinai.org\/about\/newsroom\/2026\/research-identifies-blind-spots-in-ai-medical-triage\" target=\"_blank\" rel=\"noopener\">said in a statement<\/a>.<\/p>\n<p>\u201cBut it struggled in more nuanced situations where the danger is not\u00a0immediately\u00a0obvious, and those are often the cases where clinical judgment matters most.\u201d<\/p>\n<p>OpenAI, the maker of ChatGPT, <a href=\"https:\/\/cdn.openai.com\/pdf\/2cb29276-68cd-4ec6-a5f4-c01c5e7a36e9\/OpenAI-AI-as-a-Healthcare-Ally-Jan-2026.pdf\" target=\"_blank\" rel=\"noopener\">said in January<\/a> that more than 40 million people use ChatGPT every day to address their healthcare concerns.<\/p>\n<p>Thus, ChatGPT Health was born \u2014 it was initially released to a small group of users and piqued the curiosity of Mount Sinai researchers.<\/p>\n<aside class=\"single__inline-module aligncenter wp-block-nypost-editor-primary-tag\">\n<\/aside>\n<p>\u201c<span style=\", Helvetica, Arial, sans-serif;letter-spacing: normal\">We wanted to answer a very basic but critical question: if someone is experiencing a real medical emergency and turns to ChatGPT Health for help, will it clearly tell them to go to the emergency room?<\/span>\u201d <span style=\", Helvetica, Arial, sans-serif;letter-spacing: normal\">Ramaswamy<\/span>\u00a0said.<\/p>\n<p>For his study, published this week in <a href=\"https:\/\/www.nature.com\/articles\/s41591-026-04297-7\" target=\"_blank\" rel=\"noopener\">Nature Medicine<\/a>, <span style=\", Helvetica, Arial, sans-serif;letter-spacing: normal\">Ramaswamy<\/span>\u2018s team devised 60 clinical scenarios spanning 21 medical specialties.<\/p>\n<p>Each scenario was tested 16 times, with conditions such as race, gender and lack of insurance changing each time to see if it led to a different outcome.<\/p>\n<figure class=\"nyp-slideshow-modal-image wp-block-image aligncenter size-large\"><figcaption>ChatGPT Health allows users to connect medical records and wellness apps. <span class=\"credit\">OpenAI<\/span><\/figcaption><\/figure>\n<p>In all, the researchers logged 960 interactions with ChatGPT Health. Its recommendations were compared to physician consensus.<\/p>\n<p>The study found that the tool failed to flag users to seek emergency care in 52% of serious cases.<\/p>\n<p>For example, ChatGPT Health identified\u00a0early warning signs of respiratory failure in one asthma scenario, but suggested waiting instead of getting urgent treatment, <span style=\", Helvetica, Arial, sans-serif;letter-spacing: normal\">Ramaswamy<\/span>\u00a0said.<\/p>\n<aside class=\"single__inline-module alignright\">\n\t<\/aside>\n<p><a href=\"https:\/\/profiles.ucl.ac.uk\/63386-alex-ruani\" target=\"_blank\" rel=\"noopener\">Alex Ruani<\/a>, a doctoral researcher in health misinformation mitigation with University College London, called these inaccurate assessments \u201cunbelievably dangerous.\u201d<\/p>\n<p>\u201cIf you\u2019re experiencing respiratory failure or diabetic ketoacidosis, you have a 50\/50 chance of this AI telling you it\u2019s not a big deal,\u201d <a href=\"https:\/\/www.theguardian.com\/technology\/2026\/feb\/26\/chatgpt-health-fails-recognise-medical-emergencies\" target=\"_blank\" rel=\"noopener\">she told The Guardian<\/a>.<\/p>\n<p>\u201cWhat worries me most is the false sense of security these systems create. If someone is told to wait 48 hours during an asthma attack or diabetic crisis, that reassurance could cost them their life.\u201d<\/p>\n<p>ChatGPT Health also irregularly alerted users to the <a href=\"https:\/\/988lifeline.org\/\" target=\"_blank\" rel=\"noopener\">988 Suicide and Crisis Lifeline<\/a> in high-risk situations, according to the research.<\/p>\n<figure class=\"nyp-slideshow-modal-image wp-block-image aligncenter size-large\"><figcaption>Researchers from the\u00a0Icahn School of Medicine at Mount Sinai found inaccuracies and inconsistencies with ChatGPT Health. <span class=\"credit\">Helayne Seidman<\/span><\/figcaption><\/figure>\n<p>Senior and co-corresponding\u00a0study author\u00a0<a href=\"https:\/\/profiles.mountsinai.org\/girish-n-nadkarni\" target=\"_blank\" rel=\"noreferrer noopener\">Dr. Girish N. Nadkarni<\/a> called this a \u201cparticularly\u00a0surprising and concerning finding.\u201d<\/p>\n<p>\u201cWhile we expected some variability, what we\u00a0observed\u00a0went beyond inconsistency,\u201d said Nadkarni, chief AI officer of the Mount Sinai Health System.<\/p>\n<p>\u201cThe system\u2019s alerts were inverted\u00a0relative\u00a0to clinical risk, appearing more reliably for lower-risk scenarios than for\u00a0cases\u00a0when someone\u00a0shared\u00a0how they intended to hurt\u00a0themselves,\u201d he added. \u201cIn real life, when someone talks about exactly how they would harm themselves,\u00a0that\u2019s\u00a0a sign of more immediate and serious danger, not less.\u201d<\/p>\n<aside class=\"single__inline-module alignright\">\n\t<\/aside>\n<p>ChatGPT and other chatbots have already been blamed in high-profile lawsuits for contributing to <a href=\"https:\/\/nypost.com\/2025\/11\/07\/business\/chatgpt-drove-users-to-suicide-psychosis-and-financial-ruin-california-lawsuits\/\">user suicides<\/a> and <a href=\"https:\/\/nypost.com\/2025\/08\/29\/business\/ex-yahoo-exec-killed-his-mom-after-chatgpt-fed-his-paranoia-report\/\">mental health crises<\/a>.<\/p>\n<p>The Post reached out to OpenAI for comment.<\/p>\n<p>A spokesperson told The Guardian that the study did not reflect real-life use of ChatGPT Health, a platform that\u2019s constantly updated and refined.<\/p>\n<p>The Mount Sinai doctors are not suggesting that users forgo AI health tools altogether, just that these systems should be closely monitored, independently evaluated and updated as needed.<\/p>\n<p>\u201cWe do believe that while there is a need for and a place for consumer-facing AI, there is potential for harm and thus an urgent need for independent evaluation and testing along with ongoing monitoring to establish failure modes and have engineering and human-centered safeguards to prevent adverse effects on people,\u201d Nadkarni and <span style=\", Helvetica, Arial, sans-serif;letter-spacing: normal\">Ramaswamy<\/span>\u00a0told The Post.<\/p>\n<p>They plan to assess consumer-facing AI tools in areas such as pediatric care, medication safety and use by people who don\u2019t speak English.<\/p>\n<p><em><em>If you are struggling with suicidal thoughts or are experiencing a mental health crisis and live in New York City, you can call 1-888-NYC-WELL for free and confidential crisis counseling. If you live outside the five boroughs, you can dial the 24\/7 National Suicide Prevention hotline at 988 or go to <a href=\"http:\/\/SuicidePreventionLifeline.org\" target=\"_blank\" rel=\"noreferrer noopener\">SuicidePreventionLifeline.org<\/a>.<\/em><\/em><\/p>\n<\/p><\/div>\n<p><br \/>\n<br \/><a href=\"https:\/\/nypost.com\/2026\/02\/27\/health\/where-chatgpt-health-fails-and-how-it-could-turn-deadly\/\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Don\u2019t call it ChatEMT. OpenAI last month introduced ChatGPT Health, a dedicated space in ChatGPT that allows users to ask health questions, analyze their medical records&hellip;<\/p>\n","protected":false},"author":1,"featured_media":6749,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_lmt_disableupdate":"","_lmt_disable":"","footnotes":""},"categories":[42],"tags":[],"class_list":["post-6748","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-health-wellness"],"_links":{"self":[{"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=\/wp\/v2\/posts\/6748","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6748"}],"version-history":[{"count":0,"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=\/wp\/v2\/posts\/6748\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=\/wp\/v2\/media\/6749"}],"wp:attachment":[{"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6748"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6748"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/foreignnewstoday.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6748"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}