Tuesday, December 10, 2024

Yeah, but how's its bedside manner

My husband and I used to love watching House, a show in which an eccentric, obnoxious, brilliant doctor - Gregory House (played by the terrific Hugh Laurie) -  sat around with a squad of younger colleagues he was mentoring and put their collective genius minds to the task of solving thorny diagnoses. 

We watched it throughout its long, eight-year run, right up 'til the final episode (which was terrible: the worst in the series), which we saw while Jim was hospitalized after surgery following a devastating, although not especially thorny to figure out, cancer diagnosis. 

Anyway, we always got a kick out of watching the genius that was Greg House at work.

The show went off the air in 2012, well before AI chatbots were playing doctor, and I wonder what genius Dr. House would have thought of using ChatGPT. I suspect he would have scorned it, and likely have pitted himself against it to prove that he was more brilliant. This being TV, and all.

But in real life, ChatGPT is apparently faring somewhat better than real life doctors in diagnosing complex cases. 

In an experiment, 50 doctors (residents and attendings from large healthcare systems) were presented with 6 case histories to diagnose. Half the doctors were given a ChatGPT resource to assist them, the others just stuck to conventional diagnostic info and methods. The doctors using the chatbot received an average score of 76% accuracy; the doctors without the AI assist had an average score of 90% accuracy.

So, no clear win for AI? 

Not so fast.

Dr. ChatGPT, operating on their own, earned a score of 90%.

So why didn't the doctors using AI perform better than the non-AI docs? Seems like doctors, who base their diagnoses not just on book learning but on intuition and experience as well, didn't trust AI when its diagnosis ran counter to their intuition and experience. They just didn't like having their judgement challenged. (Why am I not surprised.)

I don't expect that AI is going to replace real doctors anytime soon, but I think that doctors, once they get the hang of it, are going to get along just fine with their ChatGPT colleagues.

After all, there's just so much information/data out there, you just can't expect a doctor - even a House-ian genius type - to keep up with it all. Let alone a kindly old gaffer like Marcus Welby, MD, who was used to diagnosing appendicitis and heart murmurs in the cosy little office attached to his house back in the 1970's. After all, back in that day, there were fewer diagnostic options to choose from. I bet Marcus Welby never even heard of plaque psoriasis or Peyronie's Disease. 

How could your primary care doctor possibly stay on top of all there is to know about every possible disease out there? 

Even specialists must be hard put to stay up with what's happening in their far narrower fields.

Certainly, within my lifetime - which, realistically, ain't going to be all that long - doctors are going to be deploying AI assistants, rather than the likes of Marcus Welby's hip and happening sidekick, Dr. Steve Kiley. 

And I'm all for it.

A few years ago, a friend from the gym came down with something that, for lack of a more technical description, was exceedingly weird and rare.

The initial diagnosis was insect bite that happened in Peru, where he and his family were climbing Machu Pichu.

After time in the hospital in Peru, my friend came home and saw doctors at Mass General, Dana Farber, and the Mayo Clinic. No one could figure it out. Until finally someone did: an obscure blood cancer that was rarely encountered. 

I don't know whether an earlier diagnosis would have helped any. Survival rate is very dim for this form of cancer. But, sadly my friend - a fit, vigorous, healthy guy in his mid 50's (and truly one of the nicest men I've ever known) - died about a year after he first became ill. Maybe an earlier diagnosis, which I'm guessing that ChatGPT, with its ability to ingest an endless amount of information, would have figured things out before the human experts did.

Anyway, I'm not the biggest AI fan out there, but, in the medical case, bring it. Just bring it along with a human, not on its AI own. 

When it comes to hearing the diagnosis, I want kindly, warm-hearted, hand-holding Marcus Welby. Or even the acerbic Dr. House. I want a human to deliver the news, to tell me what's next. Sure, I'm going to run back home and start googling furiously - with the initial blob of info that comes back to me coming straight out of AI. But I really do want humans involved in my care.

I want bedside manner, dammit, not just beyond-human capacity intelligence. 



Source: NY Times

No comments: