Your Doctor is Already Using AI for Healthcare – Should You?
Artificial Intelligence (AI) is no longer a futuristic concept in healthcare; it’s here, and it’s transforming the way we approach medical advice and treatment. But here’s where it gets controversial: while AI-powered chatbots are being hailed as the next big thing in healthcare, their reliability and ethical implications are still hotly debated. Just a couple of years ago, a study revealed that ChatGPT could accurately diagnose only 2 out of 10 pediatric cases. And let’s not forget the bizarre recommendations from Google Gemini, like eating a small rock daily or using glue to keep cheese on pizza. Even more alarming, a nutritionist ended up hospitalized after following ChatGPT’s advice to replace salt with sodium bromide. So, should you trust AI with your health?
The landscape is rapidly evolving. Companies like OpenAI and Anthropic are now launching health-specific chatbots designed for both consumers and healthcare professionals. OpenAI’s ChatGPT Health allows users to connect their medical records for more personalized responses, while ChatGPT for Healthcare is already being used in hospitals nationwide. Anthropic’s Claude for Healthcare aims to assist doctors with tasks like retrieving medical records and improving patient communication. But is this a game-changer or a risky gamble?
According to Torrey Creed, an associate professor of psychiatry at the University of Pennsylvania, health-specific chatbots should be trained exclusively on reliable healthcare data, avoiding sources like social media. Additionally, ensuring HIPAA compliance and robust privacy settings is crucial to protect user data. And this is the part most people miss: while these tools can streamline care, they are not replacements for human clinicians. Their strength lies in guidance, not judgment.
Raina Merchant, executive director of the Center for Health Care Transformation and Innovation at UPenn, emphasizes that AI has immense potential but should be used cautiously. At Penn, tools like Chart Hero—an AI embedded in patient health records—help doctors quickly access information, freeing up time for more meaningful patient interactions. AI is also being used in ambient listening (with patient consent) to generate notes and in messaging interfaces to answer patient queries, always with a human overseeing the process.
But how reliable are these tools for diagnosing conditions? While they offer a wealth of information, their tendency to 'hallucinate' or deviate from medical guidelines remains a concern. Merchant advises patients to cross-verify AI-generated information with trusted sources like the American Heart Association and to trust their instincts. Should you ask ChatGPT Health about a low-grade fever? Maybe for next steps, but not for a diagnosis.
When it comes to data security, Merchant recommends avoiding sharing personal details like names, addresses, or medical record numbers with AI chatbots unless the platform is transparent about data usage and HIPAA-compliant. So, what do you think? Are AI chatbots a revolutionary tool or a risky experiment in healthcare? Let us know in the comments—we’d love to hear your thoughts!
