The Real Python Podcast

Docker + Python for Data Science and Machine Learning

Listen Later

Docker is a common tool for Python developers creating and deploying applications, but what do you need to know if you want to use Docker for data science and machine learning? What are the best practices if you want to start using containers for your scientific projects? This week we have Tania Allard on the show. She is a Sr. Developer Advocate at Microsoft focusing on Machine Learning, scientific computing, research and open source.

Tania has created a talk for the PyCon US 2020 which is now online. The talk is titled “Docker and Python: Making them Play Nicely and Securely for Data Science and ML.” Her talk draws on her expertise in the improvement of processes, reproducibility and transparency in research and data science. We discuss a variety of tools for making your containers more secure and results reproducible.

Tania is passionate about mentoring, open-source, and its community. She is an organizer for Mentored Sprints for Diverse Beginners, and she talks about the upcoming online sprints for PyCon US 2020. We also discuss her plans to start a podcast.

Topics:

  • 00:00:00 – Introduction
  • 00:01:43 – Microsoft Senior Developer Advocate Role
  • 00:04:07 – PyCon 2020 Talk - Docker and Python: making them play nicely
  • 00:05:34 – What is Docker?
  • 00:10:08 – Reproducibility of project results
  • 00:12:03 – What are the challenges of using Docker for machine learning?
  • 00:15:06 – Getting started suggestions
  • 00:16:26 – What metadata should be included?
  • 00:17:48 – Creating images through stages
  • 00:21:16 – What about your data?
  • 00:22:40 – Kubernetes: Orchestrating containers
  • 00:24:37 – Continuing stages into testing
  • 00:25:37 – What are tools for testing security?
  • 00:27:07 – Challenges in using containers for ML
  • 00:28:52 – What types of databases?
  • 00:29:39 – Are you doing initial research on a local machine?
  • 00:30:59 – An example of a recent ML project
  • 00:32:16 – Papermill: parameterizing and executing notebooks
  • 00:33:16 – NLP: Natural Language Processing
  • 00:33:58 – Kaggle: Help us better understand COVID-19
  • 00:34:42 – What are other best practices for data intensive projects?
  • 00:39:13 – Resources to get started in machine learning?
  • 00:40:30 – Mentored Sprints for Diverse Beginners
  • 00:45:34 – Tania’s upcoming podcast
  • 00:48:38 – A visiting fellow at the Alan Turing Institute
  • 00:49:08 – Weight lifting
  • 00:50:16 – Craft beer
  • 00:52:09 – What is something you thought you knew in Python but were wrong about?
  • 00:53:50 – What are excited about in the world of Python?
  • 00:54:42 – Thank you and Goodbye

    • Show links:

    • Tania Allard: Personal site
    • Docker and Python: making them play nicely and securely for Data Science and ML - Tania Allard
    • Slides for Docker and Python Talk
    • Docker
    • XKCD: Python Superfund Site
    • Best practices for writing Dockerfiles
    • Run Python Versions in Docker: How to Try the Latest Python Release
    • Kubernetes: Production-Grade Container Orchestration
    • Snyk: Securing open source and containers
    • papermill: A tool for parameterizing and executing Jupyter Notebooks
    • Natural Language Processing: Wikipedia article
    • Natural Language Processing With spaCy in Python: Real Python article
    • Kaggle: Help us better understand COVID-19
    • datree.io: Scale Engineering organization
    • repo2docker: Build, Run, and Push Docker Images from Source Code Repositories
    • Jupyter Docker Stacks: A set of ready-to-run Docker images
    • binder: Turn a Git Repo into a Collection of Interactive Notebooks
    • Hands-On Machine Learning with Scikit-Learn and TensorFlow: O’Reilly
    • Data Science from Scratch: O’Reilly
    • Python for Data Analysis: Wes McKinney - Creator of Pandas
    • Mentored Sprints for Diverse Beginners
    • The Alan Turing Institute
    • Easy Data Processing With Azure Fun - Tania Allard - PyCon 2020
    • PEP 581 – Using GitHub Issues for CPython
    • Python’s migration to GitHub - Request for Project Manager Resumes

      • Level up your Python skills with our expert-led courses:

      • Using Jupyter Notebooks
      • Histogram Plotting in Python: NumPy, Matplotlib, Pandas & Seaborn
      • Idiomatic pandas: Tricks & Features You May Not Know

        • Support the podcast & join our community of Pythonistas

        ...more
        View all episodesView all episodes
        Download on the App Store
        The Real Python PodcastBy Real Python
        • 4.7
        • 4.7
        • 4.7
        • 4.7
        • 4.7

        4.7

        136 ratings

        More shows like The Real Python Podcast

        View all
        Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

        Software Engineering Radio - the podcast for professional software developers

        272 Listeners

        The Changelog: Software Development, Open Source by Changelog Media

        The Changelog: Software Development, Open Source

        283 Listeners

        Thoughtworks Technology Podcast by Thoughtworks

        Thoughtworks Technology Podcast

        41 Listeners

        Talk Python To Me by Michael Kennedy

        Talk Python To Me

        592 Listeners

        Software Engineering Daily by Software Engineering Daily

        Software Engineering Daily

        625 Listeners

        Soft Skills Engineering by Jamison Dance and Dave Smith

        Soft Skills Engineering

        269 Listeners

        Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

        Super Data Science: ML & AI Podcast with Jon Krohn

        298 Listeners

        Python Bytes by Michael Kennedy and Brian Okken

        Python Bytes

        213 Listeners

        Data Engineering Podcast by Tobias Macey

        Data Engineering Podcast

        142 Listeners

        Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

        Syntax - Tasty Web Development Treats

        981 Listeners

        DataFramed by DataCamp

        DataFramed

        266 Listeners

        Kubernetes Podcast from Google by Abdel Sghiouar, Kaslin Fields

        Kubernetes Podcast from Google

        181 Listeners

        Practical AI by Practical AI LLC

        Practical AI

        190 Listeners

        The Stack Overflow Podcast by The Stack Overflow Podcast

        The Stack Overflow Podcast

        64 Listeners

        The Pragmatic Engineer by Gergely Orosz

        The Pragmatic Engineer

        52 Listeners