O'Reilly Data Show Podcast

Understanding deep neural networks


Listen Later

In this episode of the Data Show, I speak with Michael Mahoney, a member of RISELab, the International Computer Science Institute, and the Department of Statistics at UC Berkeley. A physicist by training, Mahoney has been at the forefront of many important problems in large-scale data analysis. On the theoretical side, his works spans algorithmic and statistical methods for matrices, graphs, regression, optimization, and related problems. On the applications side, he has contributed to systems used for internet and social media analysis, social network analysis, as well as for a host of applications in the physical and life sciences. Most recently, he has been working on deep neural networks, specifically developing theoretical methods and practical diagnostic tools that should be helpful to practitioners who use deep learning.

Analyzing deep neural networks with WeightWatcher. Image by Michael Mahoney and Charles Martin, used with permission.

We had a great conversation spanning many topics, including:

  • The class of problems in big data, machine learning, and data analysis that he has worked on at Yahoo, Stanford, and Berkeley.
  • The new UC Berkeley FODA (Foundations of Data Analysis) Institute.
  • HAWQ (Hessian AWare Quantization of Neural Networks with Mixed-Precision), a new framework for addressing problems pertaining to model size and inference speed/power in deep learning.
  • WeightWatcher: a new open source project for predicting the accuracy of deep neural networks. WeightWatcher stems from a recent series of papers with Charles Martin, of Calculation Consulting.
  • Related resources:

    • “Deep learning at scale: Tools and solutions” – a new tutorial at the Artificial Intelligence Conference in San Jose
    • Ameet Talwalker on “How to train and deploy deep learning at scale”
    • Greg Diamos on “How big compute is powering the deep learning rocket ship”
    • “RISELab’s AutoPandas hints at automation tech that will change the nature of software development”
    • Reza Zadeh on “Scaling machine learning”
    • “Becoming a machine learning company means investing in foundational technologies”
    • “Managing risk in machine learning”
    • “What are model governance and model operations?”
    • “Product management in the machine learning era”: a tutorial at the Artificial Intelligence Conference in San Jose
    • ...more
      View all episodesView all episodes
      Download on the App Store

      O'Reilly Data Show PodcastBy O'Reilly Media

      • 4
      • 4
      • 4
      • 4
      • 4

      4

      63 ratings


      More shows like O'Reilly Data Show Podcast

      View all
      The Changelog: Software Development, Open Source by Changelog Media

      The Changelog: Software Development, Open Source

      285 Listeners

      O'Reilly Radar Podcast - O'Reilly Media Podcast by O'Reilly Media

      O'Reilly Radar Podcast - O'Reilly Media Podcast

      35 Listeners

      Data Skeptic by Kyle Polich

      Data Skeptic

      475 Listeners

      Talk Python To Me by Michael Kennedy

      Talk Python To Me

      580 Listeners

      Software Engineering Daily by Software Engineering Daily

      Software Engineering Daily

      624 Listeners

      O'Reilly Design Podcast - O'Reilly Media Podcast by O'Reilly Media

      O'Reilly Design Podcast - O'Reilly Media Podcast

      8 Listeners

      AWS Podcast by Amazon Web Services

      AWS Podcast

      203 Listeners

      Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

      Super Data Science: ML & AI Podcast with Jon Krohn

      295 Listeners

      Python Bytes by Michael Kennedy and Brian Okken

      Python Bytes

      214 Listeners

      Data Engineering Podcast by Tobias Macey

      Data Engineering Podcast

      139 Listeners

      DataFramed by DataCamp

      DataFramed

      266 Listeners

      Practical AI by Practical AI LLC

      Practical AI

      196 Listeners

      Google DeepMind: The Podcast by Hannah Fry

      Google DeepMind: The Podcast

      188 Listeners

      Me, Myself, and AI by MIT Sloan Management Review and Boston Consulting Group (BCG)

      Me, Myself, and AI

      99 Listeners

      AI Chat: ChatGPT & AI News, Artificial Intelligence, OpenAI, Machine Learning by Jaeden Schafer

      AI Chat: ChatGPT & AI News, Artificial Intelligence, OpenAI, Machine Learning

      139 Listeners

      This Day in AI Podcast by Michael Sharkey, Chris Sharkey

      This Day in AI Podcast

      178 Listeners

      The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

      The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

      397 Listeners