In this episode, we are reviewing YouTube presentation by Nigam Shah discussing AI in healthcare data, model and practical application. He emphasizes that AI's effectiveness in healthcare hinges on high-quality data infrastructure and thoughtful implementation, considering ethical and cost-effective aspects. He highlights how AI models can aid in treatment decisions and diagnosis, and makes the important distinction between prediction and classification in computer science and medicine. He also shares examples of AI applications at Stanford, including predicting patient mortality and summarizing patient timelines to promote proper, less burdensome care. The speaker stresses the importance of verifying the benefits of AI, noting that improving efficiency, rather than increasing productivity, may be the most valuable application of this technology. Ultimately, the goal is to ensure AI's responsible and sustainable integration into healthcare for improved patient outcomes and a better quality of life for both patient and provider.

Key Themes and Ideas:

• The Importance of Data & Infrastructure:
• AI's foundation in healthcare is high-quality, well-structured data. The speaker stresses that healthcare data differs significantly from data in other industries due to its temporal nature. "AI comes from data and when people think of data in other Industries you think of files or documents or pictures or sound in healthcare we have to think of a timeline."
• Healthcare data is fragmented across multiple IT systems, posing significant challenges in assembling a complete patient timeline. The speaker notes that "these data are split across about 12200 it systems 1,00 all of that needs to be assembled into a cohesive construct"
• Data is often incomplete, both in terms of continuous measurement over time and the simultaneous measurement of all relevant modalities.
• Stanford's early investment in data infrastructure (since 2005) is a crucial enabler for its AI initiatives.
• AI's Role in Clinical Decision-Making:
• At a fundamental level, AI in healthcare assists in two key decisions: "whether to treat somebody or how to treat somebody."
• Many applications that are labeled predictive are actually classification problems. "about 80 90% of things in medicine that masquerade as predictions are really classifications they're just figuring out what is"
• The "how to treat" decision is computationally challenging, requiring causal inference and the ability to reason about alternative scenarios.
• Translation to Practice is key
• Implementation is key, as evidenced by the anecdote about the green button project where doctors preferred a report over a tool.
• Three Domains of AI Application in Healthcare:
• Advancing the Science of Medicine: Discovering new insights into disease mechanisms (e.g., identifying subtypes of heart failure with preserved ejection fraction).
• Advancing the Practice of Medicine: Developing tests and treatments based on scientific advances, leading to improved guidelines and clinical outcomes.
• Advancing the Delivery of Medicine (Healthcare): Implementing solutions that improve patient outcomes, reduce costs, and enhance the efficiency of healthcare delivery. This is where AI implementation can get the attention of hospital administration. "that's when Rick and David will take notice and that's where the CFO sees the effect of this both CFO of the health system and the provider Network and government officials and people who fund Medicare because now you're doing something that is saving lives and hopefully money at the same time"
• The Importance of Workflow Integration:
• Models should be fair, useful, reliable and sustainable.
• The presentation emphasizes that the success of AI depends not only on the accuracy of the models but also on the capacity to act upon the predictions. "if there's no capacity to follow up we could run a 100 models no problem and have alerts and emails and text messages flying around but if we can't follow up nothing good comes out of it"
• The Hype and Reality of Generative AI (LLMs):
• There's a need to rigorously evaluate the benefits of LLMs in healthcare beyond simply passing medical exams. "we don't really have a good handle on how to evaluate these things like we're evaluating if for answering USMLE exam questions but then we want to use them for something completely different and how can we trust that leap of faith"
• LLMs can be inaccurate when asked to summarize patient information from the EHR. Even at their best, they have an error rate that doctors won’t accept.
• LLMs can provide efficiency gains in that they may reduce cognitive load on physicians, but not in the sense of productivity gains.
• The Potential of Timeline-Trained Language Models:
• The patient timeline has its own language, which can be learned by computers. "for a computer any sequence of tokens is a language amino acids in a protein language atcg in a DNA sequence language these ICD CPT and those codes in a patient timeline language"
• Language models trained on patient timelines can improve prediction accuracy and work across subgroups and sites. "I can train classifiers with 64 examples now that's awesome we can build we can build a 100 classifiers now and it's not going to cost me 10 years and $27 million"
• Ethical Considerations and Responsible AI:
• The presentation stresses that AI models can be used for nefarious reasons if their implementation is not designed carefully, so it is up to us to decide what the responsive action is.
• The Role of the Data Science Team:
• The Stanford Healthcare data science team focuses on thought leadership, assessments, creating infrastructure and business processes, and executing high-value projects.
• Cost and Speed:
• The current model building process is too costly and too slow.

Quotes:

• "Writing a paper is easy like I have 300 of them so it's great uh and then building the delivery science that how can we execute on what the algorithms tell us that's not easy actually in fact that's like 80% of the work."
• "Good AI begins with data and that is the foundation Sanford started this journey laying the foundation for a good data infrastructure in 2005 right when I arrived on campus so that 20year investment 19-year investment is what enables us to be at the Forefront."
• "The key thing that we do different Beyond everybody else is that this is sort of a an odd-looking plot uh the point here is that let's say we make a bunch of predictions we can rank order them based on probability or one minus the probability so that's why it goes from 0 to 100 and then when you take action if your action works there's some benefit hours saved money saved saved happiness accured doesn't matter what your utility function is but it's in some units um and as you take action you start accumulating utility and then it starts going down because you're kind of wasting effort diminishing returns after a while so the key thing is how far down the list can we get because if there's no capacity to follow up we could run a 100 models no problem and have alerts and emails and text messages flying around but if we can't follow up nothing good comes out of it so we pay a lot of attention to that side the workflow side of the house so to speak than just building the model."
• "Essentially in a day you can get a report of what happened to similar patients and of course given Chad jpt now they even have a bot that you can talk to the bot's just interviewing you you know asking like what is the exposure what is the outcome the statistics is actually done by open- Source uh battle tested uh software packages."
• "The model stratifies by risk value comes from that responsive action and it is up to us to decide what the responsive action is...all sorts of Nefarious uses can happen but it's up to us what we do with it."
• "so that language can be learned by a computer using the exact same technology as is used to learn language models in spoken and natural languages"
• "the workflow work Still Remains like what are we going to do about them so we're pushing the envelope on both like we're figuring out how to do this uh sustainably at the technology level and figuring out how can we sustain it from a Workforce level"
• "[regarding generative AI] productivity is not going to be the winner here it's going to be something else"

Conclusion:

Stanford's approach to AI in healthcare is characterized by a focus on data quality, practical application, ethical considerations, and workflow integration. The presentation emphasizes the need to move beyond simply building models to creating sustainable, cost-effective, and reliable AI solutions that demonstrably improve patient care.