AI vs. Doctors - Who Is Most Trustworthy with a Diagnosis?
AI vs. Doctors - Who Is Most Trustworthy with a Diagnosis?
In The Upside of Your Dark Side, Dr. Robert Biswas-Diener and I wrote a provocative chapter titled, “Beyond the Obsession with Mindfulness.” In it, we discuss how sometimes the best decisions occur when we engage in a bit of mindless distraction. To be precise, there are four steps:
Mindful contemplation of the situation.
Stop and take a break.
Distract yourself in an unrelated activity.
Make a decision.
You do make better decisions (click here for more details)
When we wrote our book, AI was not part of mainstream culture. Now everyone with internet access can use AI as a co-collaborator (or conspirator, depending on your optimism about technology). Like me, have you ever found yourself in a heated debate about whether AI could replace human psychologists and physicians? Well, grab an Italian rainbow square cookie and get comfy, because we're about to explore a study that adds serious fuel to that fire.
The Ultimate Face-Off
Google, the tech giant, trains its AI to diagnose not just your everyday cold, but rare, complex medical conditions. They then pit this AI against 20 human doctors, asking them to provide diagnoses for 302 such cases. Some doctors are armed with just the patient information, some have the entire Internet at their disposal, and some get to consult AI. And then, the AI takes a shot at diagnosing the same 302 cases. Click here to read the study.
The Unexpected Outcome
Here's where it gets wild: AI didn't just hold its own, it outperformed those humans who added 8 years of medical school on top of their undergraduate education. The AI's Top-10 accuracy was a whopping 59.1%, compared to a mere 33.6% by human physicians. Even when physicians were allowed to Google or consult AI, they still couldn't outsmart the AI.
The Plot Twist
But wait, there's a plot twist: the cases were drawn from New England Journal of Medicine (NEJM) case reports, and it's likely that the AI was trained using the actual material. So, it might have been bit of a cheat: hiding the answer key between lines of code rather than actual diagnostic assessments. But then again, maybe the physicians had the cheat sheet with , access to X-rays and photographs while AI could only rely on text. A bit of an uneven playing field? Then again, how do you even the playing field between two distinctive people other than Kurt Vonneguts’ Harrison Bergeron1 route?
Chew on This
The odds improve when physicians Google or consult AI. (Notice that nobody is mentioning WebMD, the best app for generating hypochondriasis.)2 Yet, with Google and AI assistance, physicians lose to just AI. Apparently, physicians mistrust AI and still choose the wrong diagnosis even if AI gives them the correct answer.
Provocations
If AI is already outperforming human physicians, what should we expect in 10 years?
The Ethical Dilemma: Consider a scenario where an AI makes a correct diagnosis that a human physician disagrees with, leading to a patient receiving incorrect treatment. Who should be held accountable in such a situation?
The Future of Medicine: Imagine it's 2038. AI has advanced significantly and is now the primary diagnostic tool in most hospitals. What new skills might physicians need and why aren’t we developing them now?
The Personal Touch: A big part of healthcare is the human connection between doctor and patient. Can AI ever replicate this? If not, how can we ensure this crucial aspect of healthcare isn't lost as AI becomes more prevalent?
The Learning Curve: AI learns from vast amounts of data. But what happens when it encounters a completely new disease? Can it still outperform human physician who can draw on their intuition and experience? How can we prepare AI for such unknowns?
Provoked is free today. But if you enjoyed this post, let me know this work is valuable by pledging a future subscription.
Dr. Todd B. Kashdan is an author of several books including The Upside of Your Dark Side (Penguin) and The Art of Insubordination: How to Dissent and Defy Effectively (Avery/Penguin) and Professor of Psychology and Leader of The Well-Being Laboratory at George Mason University.
Connect on: Twitter or Facebook or Linkedin or Instagram
Read Past Issues Here Including:
If you haven’t read this short story, this might be the best thing you gain from this issue.
To no surprise, I have yet to be consulted for their marketing campaign.
I think in an ideal world, doctors would leverage AI's diagnostic (and thus treatment) accuracy and then use coaching/motivational interviewing and lifestyle approaches to enhance the therapeutic relationship to help patients achieve wellness and wellbeing. (so easy, right??) That would take some humility and of course refining. This would be more easily achieved by referrals to other specialists/healthcare providers/coaches (wink wink). We definitely need to see studies like this repeated! Fascinating