
Peter Lee and the Impact of GPT-4 + Large Language AI Models in Medicine
Om avsnittet
Link to the book: The AI Revolution in Medicine
Link to my review of the book
Link to the Sparks of Artificial General Intelligence preprint we discussed
Link to Peter’s paper on GPT-4 in NEJM
Transcript (with a few highlights in bold of many parts that could be bolded!)
Eric Topol (00:00):
Hello, I'm Eric Topol, and I'm really delighted to have with me Peter Lee, who's the director of Microsoft Research and who is the author, along with a couple of colleagues for an incredible book called The AI Revolution in Medicine, GPT-4 and Beyond. Welcome, Peter.
Peter Lee (00:20):
Hello Eric. And thanks so much for having me on. This is a real honor to be here.
Eric Topol (00:24):
Well, I think you are in the enviable position of having spent now more than seven months looking at GPT-4’s S capability, particularly in the health and medicine space. And it was great that you recorded that in a book for everyone else to learn because you had such a nice head start. I guess what I wanted to start with is, I mean, it's, it's a phenomenal book. I [holding the book up], this prop. I can't resist
Peter Lee (00:52):
Eric Topol (00:53):
When, when I got it, I, I couldn't, I stayed up most of the night because I couldn't put it down. It was, it is so engrossing. But when you, when you first got your hands on this and started testing it, what were, what were your initial thoughts?
Peter Lee (01:09):
Yeah. I, let me first start by saying thank you for the nice words about the book, but really, so much of the credit goes to the co-authors, Carey Goldberg and Zach Kohane and Corey in particular took my overly academic writing. I suspect you have the same kind of writing style as well as Zach's pretty academic writing and helped turn it into something that would be approachable to non-computer scientists and as she put it, as much as possible as a page turner. So I'm glad that her work helped make the, the book an easy read. I,
Eric Topol (01:54):
I want to just say you're very humble because the first three chapters that you wrote yourself were clearly the, the best ones for me. Anyway. I don't mean to interrupt, but it, it, it is an exceptional book, really.
Peter Lee (02:06):
Oh thank you very much. It means a lot. Hearing that from you. You know, my own view is that the, the best writing and the best analyses and the best ideas for applications or not of this type of technology in medicine are yet to come. But you're right that I did benefit from this seven-month head start. And so, you know, I think the timing is, is very good. but I'm hoping that much better books and much better writings and ideas will come, you know, when you start with something like this, I, I suspect, Eric, you had the same thing. you start off with a lot of skepticism and I, in fact, I sort of now made light with this. I talk about the nine stages of grief that you have to go through.
(02:55):
I was extremely skeptical. Of course, I was very aware of GPT 2, GPT 3 and GPT 3.5. I understand, you know, what goes into those models really deeply. and so some of the claims, when I was exposed to the early development, GPT-4 just seemed outlandish and impossible. So I, I was, you know, skeptical, somewhat quietly skeptical. We've all been around the block before and, you know, we've heard lots of AI claims and I was in that state for maybe more than two weeks. And then I started to become in that two weeks annoyed, because I know that some of my colleagues like falling into what I felt was the trap of getting fooled by this technology. And then that turned into frustration and fear. I actually got angry. And one colleague who I won't name I've since had to apologize because then I into the phase of amazement because you start to encounter things that you can't explain that this thing seems to be doing that turns into joy.
(04:04):
I remember the exhilaration of thinking, wow, I did not think I would live long enough to see a technology like this. and then intensity, There was a period of about three days when I didn't sleep, I was just experimenting. Then you run into some limits and some areas of puzzlement and that's a phase of chagrin. And then real dangerous missteps and mistakes that this system can make that you realize might end up really hurting people. and then, you know, ChatGPT gets released and to our surprise it catches fire with people. And we learn directly through communications that some clinicians are using it in clinical settings. And that heightens the concern. And I, I can't say I'm in the ninth stage of enlightenment yet, but you do become very committed to wanting to help the medical community get up to speed and to be in a position to take ownership of the question of whether, when, and how a technology like this should be used. and that was really the motivating force behind the book. And it, it was really that journey. And that journey also has given me patience with everyone else in the world, because I realize everyone else in the world has to go through those same nine, nine stages.
Eric Topol (05:35):
Well, those stages that you went through are actually a great way to articulate this pluripotent technology. I mean, I think you, you touched on that chat. ChatGPT was released November 30th and within 90 days had a billion distinct users, which is beyond anything in history. And then of course, this transcended that quite a bit as you showed in the book coming out in you know, just a very short time in March. right. And I think a lot of people want access to GPT-4 because they know that there is this jump in its capabilities. But the book starts off after Sam Altman's forward, which was also nice because he said, you know, this is just an early, as you pointed out there, there's a lot more to come in the large language model space.
(06:30):
But the grabber to me was this futuristic, this second year medical resident who's using an app on the phone to get to the latest GPT to help manage her patient, and then all the other things that it's doing to check on her patients and do all the things that are the tasks that clinicians don't really want to do, that they need help with. And that just grabs you as to the futuristic potential, which may not be so far away. And I think then you get into the nuts and bolts, but one of the things that I think is a misnomer that you really nailed is how you say it isn't just that it generates, but it really is great at editing and analyzing. And here it's, it's called generative AI. Can you, can you expound on that? And it's unbelievable conversationalist capability.
Peter Lee (07:23):
Yeah. you know, the term Generative AI, I tried for a while to push back on this, but I think it's just caught on and I've given up on that. And I get it. You know, I, I think especially with ChatGPT it's of course reasonable for the public to be, you know infatuated with a thing that can write love letters, write poetry and that generative capability. and of course, you know school children writing their essays and so on this way. But as you say one thing we have discovered through a lot of experimentation is it's actually somewhat of a marginal generator of text. I would not say at all. That is, it is not as good a poet as good human poets. It's not the, you know, people have programmed GPT-4 to try to write whole novels and it can do that,
(08:24):
they aren't great. and it's a challenge, you know within Microsoft, our Nuance division has been integrating GPT-4 to help write clinical and encounter notes. and you can tell it's just hitting at the very limits of the capabilities in and of the intelligence of GPT-4 to be able to do that well. But one area where it really excels is in evaluating or judging or reviewing things. And we've seen that over and over again. in chapter three. You know, I have this example of its analysis of some contemporary poetry which is just stunning in its kind of insights and its use of metaphor and allegory. And but then in other situation…
Get full access to Ground Truths at erictopol.substack.com/subscribe
Fler avsnitt
Visa alla avsnitt av Ground TruthsGround Truths med Eric Topol finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.