Docker is a common tool for Python developers creating and deploying applications, but what do you need to know if you want to use Docker for data science and machine learning? What are the best practices if you want to start using containers for your scientific projects? This week we have Tania Allard on the show. She is a Sr. Developer Advocate at Microsoft focusing on Machine Learning, scientific computing, research and open source.
Tania has created a talk for the PyCon US 2020 which is now online. The talk is titled “Docker and Python: Making them Play Nicely and Securely for Data Science and ML.” Her talk draws on her expertise in the improvement of processes, reproducibility and transparency in research and data science. We discuss a variety of tools for making your containers more secure and results reproducible.
Tania is passionate about mentoring, open-source, and its community. She is an organizer for Mentored Sprints for Diverse Beginners, and she talks about the upcoming online sprints for PyCon US 2020. We also discuss her plans to start a podcast.
00:00:00 – Introduction00:01:43 – Microsoft Senior Developer Advocate Role00:04:07 – PyCon 2020 Talk - Docker and Python: making them play nicely00:05:34 – What is Docker?00:10:08 – Reproducibility of project results00:12:03 – What are the challenges of using Docker for machine learning?00:15:06 – Getting started suggestions00:16:26 – What metadata should be included?00:17:48 – Creating images through stages00:21:16 – What about your data?00:22:40 – Kubernetes: Orchestrating containers00:24:37 – Continuing stages into testing00:25:37 – What are tools for testing security?00:27:07 – Challenges in using containers for ML00:28:52 – What types of databases?00:29:39 – Are you doing initial research on a local machine?00:30:59 – An example of a recent ML project00:32:16 – Papermill: parameterizing and executing notebooks00:33:16 – NLP: Natural Language Processing00:33:58 – Kaggle: Help us better understand COVID-1900:34:42 – What are other best practices for data intensive projects?00:39:13 – Resources to get started in machine learning?00:40:30 – Mentored Sprints for Diverse Beginners 00:45:34 – Tania’s upcoming podcast00:48:38 – A visiting fellow at the Alan Turing Institute00:49:08 – Weight lifting00:50:16 – Craft beer00:52:09 – What is something you thought you knew in Python but were wrong about?00:53:50 – What are excited about in the world of Python?00:54:42 – Thank you and GoodbyeTania Allard: Personal siteDocker and Python: making them play nicely and securely for Data Science and ML - Tania AllardSlides for Docker and Python TalkDockerXKCD: Python Superfund SiteBest practices for writing DockerfilesRun Python Versions in Docker: How to Try the Latest Python ReleaseKubernetes: Production-Grade Container OrchestrationSnyk: Securing open source and containerspapermill: A tool for parameterizing and executing Jupyter NotebooksNatural Language Processing: Wikipedia articleNatural Language Processing With spaCy in Python: Real Python articleKaggle: Help us better understand COVID-19datree.io: Scale Engineering organizationrepo2docker: Build, Run, and Push Docker Images from Source Code RepositoriesJupyter Docker Stacks: A set of ready-to-run Docker imagesbinder: Turn a Git Repo into a Collection of Interactive NotebooksHands-On Machine Learning with Scikit-Learn and TensorFlow: O’ReillyData Science from Scratch: O’ReillyPython for Data Analysis: Wes McKinney - Creator of PandasMentored Sprints for Diverse BeginnersThe Alan Turing InstituteEasy Data Processing With Azure Fun - Tania Allard - PyCon 2020PEP 581 – Using GitHub Issues for CPythonPython’s migration to GitHub - Request for Project Manager ResumesLevel up your Python skills with our expert-led courses:
Using Jupyter NotebooksHistogram Plotting in Python: NumPy, Matplotlib, Pandas & SeabornIdiomatic pandas: Tricks & Features You May Not Know Support the podcast & join our community of Pythonistas