TL;DR

OpenAI’s ChatGPT Health product launched earlier this month to serve the 230 million people already asking health-related queries weekly. Research suggests LLMs may provide more accurate medical information than traditional web searches, though risks around sycophancy and hallucination remain.

The End of “Dr. Google”?

For two decades, searching symptoms online has been the default first step for health concerns—a practice that earned the moniker “Dr. Google.” Now, OpenAI is positioning ChatGPT Health as a superior alternative, offering access to users’ electronic medical records and fitness data with permission.

The timing proved challenging: the launch came just two days after reports of a teenager’s death following extensive ChatGPT conversations about drug combinations, prompting questions about AI-based health advice.

What Research Shows

Studies comparing LLM responses to web searches present a nuanced picture. Research on GPT-4o found it answered medical questions correctly approximately 85% of the time—notable when human doctors misdiagnose patients 10-15% of the time.

Marc Succi, a Harvard Medical School associate professor, observed a marked improvement in patient enquiries: “You see patients with a college education, a high school education, asking questions at the level of something an early med student might ask.”

However, studies also reveal concerning patterns. GPT-4 and GPT-4o have been shown to accept incorrect drug information and fabricate definitions for non-existent medical conditions—tendencies that could spread medical misinformation.

The Sycophancy Problem

A key concern is LLMs’ tendency to agree with users rather than challenge incorrect assumptions. Reeva Lederman of the University of Melbourne notes that patients who dislike their doctor’s diagnosis might seek validation from an LLM, which could inappropriately encourage rejection of professional medical advice.

OpenAI reports the GPT-5 series shows reduced sycophancy and hallucination, with their HealthBench benchmark testing for appropriate uncertainty expression and unnecessary alarmism avoidance.

Looking Forward

The comparison to autonomous vehicles proves instructive: the question isn’t whether ChatGPT Health is perfect, but whether it causes less harm than existing alternatives. For UK businesses in healthcare or employee wellness, understanding how staff and customers interact with AI health tools will become increasingly important.

The key message: AI health tools may improve information quality, but they remain supplements to—not replacements for—professional medical advice.