Faces of Digital Health

How will health AI assurance labs look like and who will pay for assessments?

Listen Later

Several organizations are thinking about the right way to regulate AI and the idea of assurance labs which would test and validate AI solutions in the US healthcare is taking shape. This was the topic we discussed with Brian Anderson - CEO of the coalition for Health AI or CHAI: how will assurance labs look like, how much will assessments cost, who will pay for them, and how will AI “nutrition labels” look like. 


Summary:

Assurance Labs in Healthcare AI
  • The Coalition for Health AI (CHI) is developing a network of quality assurance labs to evaluate AI models in healthcare.
  • These labs aim to provide independent, transparent assessments of AI models' performance across different populations.
  • By the end of 2024, CHI plans to have two certified labs operational, with more to follow in 2025.
    • Model Cards and Evaluation
    • CHAI has introduced "model cards" or "nutrition labels" for AI models, describing their training data, methodology, indications, and limitations.
    • Model cards are created by developers, while assurance labs provide independent evaluation reports.
    • CHAI is working on technical specifications for model cards to ensure consistency and transparency.
      • Goals and Benefits
      • Assurance labs aim to balance innovation with safety in AI development.
      • They can help identify model performance issues across different populations and accelerate improvements.
      • The process is intended to build trust in AI among healthcare providers and patients.
        • Implementation and Challenges
        • CHAI is creating a competitive marketplace of quality assurance labs to keep costs reasonable.
        • Labs must be free from conflicts of interest with AI vendors.
        • Evaluation reports will be published in a public registry for transparency.
        • The cost of evaluations is expected to be in the range of thousands of dollars, not millions.
          • Future Plans
          • CHAI is exploring partnerships with health systems and NGOs to establish quality assurance labs in the EU.
          • The initiative aims to be scalable and adaptable to different geographic regions and populations.


            • www.facesofdigitalhealth.com

            Newsletter: https://fodh.substack.com/

            ...more
            View all episodesView all episodes
            Download on the App Store
            Faces of Digital HealthBy Tjasa Zajc
            • 5
            • 5
            • 5
            • 5
            • 5

            5

            20 ratings

            More shows like Faces of Digital Health

            View all
            HBR IdeaCast by Harvard Business Review

            HBR IdeaCast

            246 Listeners

            Freakonomics Radio by Freakonomics Radio + Stitcher

            Freakonomics Radio

            32,108 Listeners

            a16z Podcast by Andreessen Horowitz

            a16z Podcast

            999 Listeners

            The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

            The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

            513 Listeners

            The Daily by The New York Times

            The Daily

            111,470 Listeners

            Masters of Scale by WaitWhat

            Masters of Scale

            3,964 Listeners

            The Diary Of A CEO with Steven Bartlett by DOAC

            The Diary Of A CEO with Steven Bartlett

            6,373 Listeners

            The Readout Loud by STAT

            The Readout Loud

            317 Listeners

            The Journal. by The Wall Street Journal & Gimlet

            The Journal.

            5,886 Listeners

            Tradeoffs by Tradeoffs

            Tradeoffs

            391 Listeners

            The Prof G Pod with Scott Galloway by Vox Media Podcast Network

            The Prof G Pod with Scott Galloway

            5,090 Listeners

            All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

            All-In with Chamath, Jason, Sacks & Friedberg

            8,734 Listeners

            Raising Health by Andreessen Horowitz, a16z Bio + Health

            Raising Health

            141 Listeners

            The Mel Robbins Podcast by Mel Robbins

            The Mel Robbins Podcast

            20,972 Listeners

            The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

            The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

            421 Listeners