Faces of Digital Health

How will health AI assurance labs look like and who will pay for assessments?

Listen Later

Several organizations are thinking about the right way to regulate AI and the idea of assurance labs which would test and validate AI solutions in the US healthcare is taking shape. This was the topic we discussed with Brian Anderson - CEO of the coalition for Health AI or CHAI: how will assurance labs look like, how much will assessments cost, who will pay for them, and how will AI “nutrition labels” look like. 


Summary:

Assurance Labs in Healthcare AI
  • The Coalition for Health AI (CHI) is developing a network of quality assurance labs to evaluate AI models in healthcare.
  • These labs aim to provide independent, transparent assessments of AI models' performance across different populations.
  • By the end of 2024, CHI plans to have two certified labs operational, with more to follow in 2025.
    • Model Cards and Evaluation
    • CHAI has introduced "model cards" or "nutrition labels" for AI models, describing their training data, methodology, indications, and limitations.
    • Model cards are created by developers, while assurance labs provide independent evaluation reports.
    • CHAI is working on technical specifications for model cards to ensure consistency and transparency.
      • Goals and Benefits
      • Assurance labs aim to balance innovation with safety in AI development.
      • They can help identify model performance issues across different populations and accelerate improvements.
      • The process is intended to build trust in AI among healthcare providers and patients.
        • Implementation and Challenges
        • CHAI is creating a competitive marketplace of quality assurance labs to keep costs reasonable.
        • Labs must be free from conflicts of interest with AI vendors.
        • Evaluation reports will be published in a public registry for transparency.
        • The cost of evaluations is expected to be in the range of thousands of dollars, not millions.
          • Future Plans
          • CHAI is exploring partnerships with health systems and NGOs to establish quality assurance labs in the EU.
          • The initiative aims to be scalable and adaptable to different geographic regions and populations.


            • www.facesofdigitalhealth.com

            Newsletter: https://fodh.substack.com/

            ...more
            View all episodesView all episodes
            Download on the App Store
            Faces of Digital HealthBy Tjasa Zajc
            • 5
            • 5
            • 5
            • 5
            • 5

            5

            20 ratings

            More shows like Faces of Digital Health

            View all
            WSJ Tech News Briefing by The Wall Street Journal

            WSJ Tech News Briefing

            1,639 Listeners

            Freakonomics Radio by Freakonomics Radio + Stitcher

            Freakonomics Radio

            32,174 Listeners

            The Human Upgrade with Dave Asprey by Dave Asprey

            The Human Upgrade with Dave Asprey

            7,185 Listeners

            Hidden Brain by Hidden Brain, Shankar Vedantam

            Hidden Brain

            43,397 Listeners

            How I Built This with Guy Raz by Guy Raz | Wondery

            How I Built This with Guy Raz

            30,231 Listeners

            The Daily by The New York Times

            The Daily

            111,294 Listeners

            The Diary Of A CEO with Steven Bartlett by DOAC

            The Diary Of A CEO with Steven Bartlett

            6,911 Listeners

            Today, Explained by Vox

            Today, Explained

            10,101 Listeners

            HIMSSCast by HIMSS Media

            HIMSSCast

            9 Listeners

            Hard Fork by The New York Times

            Hard Fork

            5,425 Listeners

            Coaching Real Leaders by Harvard Business Review / Muriel Wilkins

            Coaching Real Leaders

            649 Listeners

            Huberman Lab by Scicomm Media

            Huberman Lab

            28,272 Listeners

            The Ezra Klein Show by New York Times Opinion

            The Ezra Klein Show

            15,206 Listeners

            News Connect あなたと経済をつなぐ5分間 #ニュースコネクト by Chronicle

            News Connect あなたと経済をつなぐ5分間 #ニュースコネクト

            24 Listeners

            The Economics of Everyday Things by Freakonomics Network & Zachary Crockett

            The Economics of Everyday Things

            1,585 Listeners