The Python Podcast.__init__

Scaling Deep Learning Using Polyaxon with Mourad Mourafiq

Listen Later
Summary

With libraries such as Tensorflow, PyTorch, scikit-learn, and MXNet being released it is easier than ever to start a deep learning project. Unfortunately, it is still difficult to manage scaling and reproduction of training for these projects. Mourad Mourafiq built Polyaxon on top of Kubernetes to address this shortcoming. In this episode he shares his reasons for starting the project, how it works, and how you can start using it today.

Preface
  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Finding a bug in production is never a fun experience, especially when your users find it first. Airbrake error monitoring ensures that you will always be the first to know so you can deploy a fix before anyone is impacted. With open source agents for Python 2 and 3 it’s easy to get started, and the automatic aggregations, contextual information, and deployment tracking ensure that you don’t waste time pinpointing what went wrong. Go to podcastinit.com/airbrake today to sign up and get your first 30 days free, and 50% off 3 months of the Startup plan.
  • To get worry-free releases download GoCD, the open source continous delivery server built by Thoughworks. You can use their pipeline modeling and value stream map to build, control and monitor every step from commit to deployment in one place. And with their new Kubernetes integration it’s even easier to deploy and scale your build agents. Go to podcastinit.com/gocd to learn more about their professional support services and enterprise add-ons.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • Your host as usual is Tobias Macey and today I’m interviewing Mourad Mourafiq about Polyaxon, a platform for building, training and monitoring large scale deep learning applications.
    • Interview
    • Introductions
    • How did you get introduced to Python?
    • Can you give a quick overview of what Polyaxon is and your motivation for creating it?
    • What is a typical workflow for building and testing a deep learning application?
    • How is Polyaxon implemented?
      • How has the internal architecture evolved since you first started working on it?
      • What is unique to deep learning workloads that makes it necessary to have a dedicated tool for deploying them?
      • What does Polyaxon add on top of the existing functionality in Kubernetes?


      • It can be difficult to build a docker container that holds all of the necessary components for a complex application. What are some tips or best practices for creating containers to be used with Polyaxon?

      • What are the relative tradeoffs of the various deep learning frameworks that you support?

      • For someone who is getting started with Polyaxon what does the workflow look like?

        • What is involved in migrating existing projects to run on Polyaxon?


        • What have been the most challenging aspects of building Polyaxon?

        • What are your plans for the future of Polyaxon?

          • Keep In Touch
          • Website
          • @mmourafiq on Twitter
          • mouradmourafiq on GitHub
            • Picks
            • Tobias
              • Kubernetes
              • Kubernetes Up And Running
              • Kelsey Hightower
              • Food Fight Show With Kelsey Hightower


              • Mourad

                • Schopenhauer


                  • Links
                  • Polyaxon
                  • Investment Banking
                  • Luxembourg
                  • Matlab
                  • Text Mining
                  • Tensorflow
                  • Docker
                  • Kubernetes
                  • Deep Learning
                    • Free Deep Learning Textbook


                    • Machine Learning Engineer

                    • Hyperparameters

                    • Continuous Integration

                    • PyTorch

                    • MXNet

                    • Scikit-Learn

                    • Helm

                    • Mesos

                    • Spark

                    • SparkML

                      • The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

                      ...more
                      View all episodesView all episodes
                      Download on the App Store
                      The Python Podcast.__init__By Tobias Macey
                      • 4.4
                      • 4.4
                      • 4.4
                      • 4.4
                      • 4.4

                      4.4

                      100 ratings

                      More shows like The Python Podcast.__init__

                      View all
                      The Changelog: Software Development, Open Source by Changelog Media

                      The Changelog: Software Development, Open Source

                      283 Listeners

                      Data Skeptic by Kyle Polich

                      Data Skeptic

                      481 Listeners

                      Chat With Traders by Tessa Dao

                      Chat With Traders

                      1,979 Listeners

                      Talk Python To Me by Michael Kennedy

                      Talk Python To Me

                      593 Listeners

                      Software Engineering Daily by Software Engineering Daily

                      Software Engineering Daily

                      623 Listeners

                      The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

                      The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

                      446 Listeners

                      Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

                      Super Data Science: ML & AI Podcast with Jon Krohn

                      297 Listeners

                      Python Bytes by Michael Kennedy and Brian Okken

                      Python Bytes

                      215 Listeners

                      Data Engineering Podcast by Tobias Macey

                      Data Engineering Podcast

                      142 Listeners

                      Machine Learning Guide by OCDevel

                      Machine Learning Guide

                      764 Listeners

                      Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

                      Syntax - Tasty Web Development Treats

                      981 Listeners

                      DataFramed by DataCamp

                      DataFramed

                      267 Listeners

                      Practical AI by Practical AI LLC

                      Practical AI

                      190 Listeners

                      The Real Python Podcast by Real Python

                      The Real Python Podcast

                      140 Listeners

                      Hard Fork by The New York Times

                      Hard Fork

                      5,427 Listeners