Datacast

Datacast

Download on the App Store

Datacast episodes

  • Episode 39: Serverless Machine Learning In Action with Carl Osipov

    Show Notes

    • (2:22) Carl talked about his early exposure to programming and his Bachelor’s degree in Computer Science at the University of Rochester in the late 90s.
    • (5:12) Carl implemented his first fully connected, two-hidden-layer artificial neural network using the C programming language back in 2000 when using neural networks wasn’t nearly as cool as it is today.
    • (8:00) Carl started his career as a software engineer at IBM, writing software for large-scale distributed systems and voice-dialog management system.
    • (13:31) The first production machine learning system that Carl worked on is called Conversational Interaction Manager, which is a dialog management system for conversational mixed-initiative natural language applications. He brought up the challenges in DevOps and data quality.
    • (20:05) The second production machine learning system that you worked on is called Smarter Campus, which is a project that enables staffing recommendations based on social networking, optimization, and text analytics.
    • (27:16) Carl unpacked the evolution of his career at IBM, working on various leadership roles. In particular, he worked on IBM Bluemix, IBM’s cloud platform-as-a-service, with over 1 million registered users. He emphasized the importance of talking to customers and finding product-market fit.
    • (33:01) Carl discussed his decision to pursue a Master’s degree in Computer Science at the University of Florida in the mid of his career.
    • (35:24) Carl explained his research paper, which combines game theory and machine learning called “AmalgaCloud: Social Network Adaptation for Human and Computational Agent Team Formation.” The paper focuses on the relationship between network adaptation for candidate group participants and the performance of problem-solving groups.
    • (40:50) Carl discussed his patent on learning ontologies for machine learning - which maps ontologies from data warehouses to computer systems.
    • (47:00) Carl unpacked his 4-part blog series dated in 2016 that discusses server-less computing via tools such as Docker and Apache OpenWhisk.
    • (52:58) Carl emphasized the importance of learning Docker to be productive as a Machine Learning practitioner.
    • (55:02) Carl became a program manager at Google Cloud and helped manage the company’s efforts to democratize machine learning via the Advanced Solutions Lab in 2017.
    • (59:07) Carl recalled his experience as an instructor at various machine learning boot camps.
    • (01:01:44) Carl went over the growing popularity of semi-structured data, referring to his talk at Google’s 2018 Data Cloud Next event.
    • (01:06:29) Currently, Carl is the CTO of CounterFactual AI, which works with various clients using tools such as PyTorch and AWS. He brought up an example of a food delivery application.
    • (01:09:13) Carl went over his experience leading a workshop on Server-less Machine Learning with TensorFlow at the Reinforce AI Conference in Budapest last year.
    • (01:10:52) Carl is writing a book with Manning called Server-less Machine Learning In Action. He explained that server-less tools help minimize the efforts to do MLOps.
    • (01:13:47) Carl talked about the rise of PyTorch as a production-ready deep learning framework, as well as his preference for the PyTorch’s language design philosophy.
    • (01:17:10) Carl shared his opinions on choosing different cloud platforms to host and run the server-less ML pipeline.
    • (01:19:37) Carl described the data and tech community in Orlando, Florida.
    • (01:21:53) Closing segment.

    His Contact Information

    • Website
    • LinkedIn
    • Twitter
    • GitHub
    • Google Scholar

    His Recommended Resources

    • “An Introduction to Natural Computation” by Dana Ballard
    • IBM’s semiconductor facility FAB
    • IBM Bluemix
    • “Pattern Recognition and Machine Learning” by Christopher Bishop
    • Docker
    • PyTorch
    • PyTorch Lightning
    • Jurgen Schmidhuber (the father of LSTM)
    • Solomon Hykes (Founder, CTO, and chief architect of Docker)
    • Dana Ballard (professor of Computer Science at UT-Austin)
    • Gang of Four Design Patterns (engineering book with object-oriented design theory and practice)

    Serverless Machine Learning In Action

    • Check out the book at this link: https://www.manning.com/books/serverless-machine-learning-in-action?a_aid=khanhnamle1994&a_bid=fa913283
    • Here are the 5 free Ebook codes: smldcr-0BE0, smldcr-9F05, smldcr-F807, smldcr-CBB2, smldcr-52D1
    • Here is a 40% discount code: poddcast19


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 30 min
  • Episode 38: Designing For Analytics with Brian O'Neill

    Show Notes

    • (2:40) Brian discussed his career as a musician and his mission as a consultant to bring design principles into the analytics world.
    • (5:25) Brian talked about his career inception as a UX designer and his interest in human-centered design.
    • (9:48) Brian shared the backstory behind starting Designing for Analytics and his advice for anyone interested in becoming a consultant (hint: finding the minimum viable audience for your craft!).
    • (20:44) Brian shared the common problems that his clients ask him to solve - citing that many of the solutions in engineering-driven organizations are “Technically Right, Effectively Wrong” (listen to Brian’s podcast with David Stephenson).
    • (27:20) Brian explained why data product design goes well beyond user interfaces and helps define what is required to enable the desired user and business outcomes, referring to his post “Does your data product enable surgery or healing?”
    • (33:14) Brian revealed the tactical tips for designing an effective prototype for data products, as shared in his post “Designing MVPs for Data Products and Decision Support Tools.”
    • (40:31) Brian talked about the importance of using human-centered design to measure meaningful engagement in the context of data products, as shared in his post “Why Low Engagement May Not be the Problem with Your Data Product or Analytics Service” (Hint: Think about the last mile and use design to make deliberate choices to improve user engagement).
    • (47:46) Brian unpacked the design framework CED (which stands for Conclusion, Evidence, and Data), which helps build customer trust, engagement, and indispensability around advanced analytics.
    • (54:55) Brian shared his take on how to structure a quad team, including software engineers, UX designers, data scientists, and product managers to build machine learning-powered products.
    • (01:03:25) Brian emphasized the importance of trust in modern data products, after countless conversations with leaders in his podcast Experiencing Data.
    • (01:06:34) Brian unveiled his seminar called Designing Human-Centered Data Products for data scientists, technical product managers, and analytics practitioners.
    • (01:09:47) Closing segment.


    His Contact Info

    • Designing For Analytics
    • Twitter
    • LinkedIn
    • Experiencing Data Podcast
    • Insights Newsletter

    His Recommended Resources

    • Seth Godin’s podcast Akimbo
    • Minimum Viable Product
    • Wizard of Oz Testing
    • CED Framework
    • Chris Do
    • Scott Berkun (his book “How Design Makes The World”)
    • Juhan Sonin (Involution Studios)
    • Amanda Cox (NYT’s The Upshot)
    • “Good Charts” (by Scott Berinato)
    • “Change By Design” (by Tim Brown)
    • “Infonomics” (by Douglas Laney)
    • “Competing In The Age of AI” (by Marco Iansiti and Karim Lakhani)


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 16 min
  • Episode 37: Machine Learning In Production with Luigi Patruno

    Show Notes

    • (2:19) Luigi got his Bachelor’s in Mathematics and Master’s in Computer Science from Fordham University, with a break working as a Data Analyst in between.
    • (5:41) Luigi worked as a Research Engineer at Fordham's Wireless Sensor Data Mining Lab for a year during his Master’s program and got exposed to Machine Learning.
    • (9:13) Luigi’s first role out of graduate school is a Data Engineering position at Namely, a Human Resources platform for thousands of mid-sized companies.
    • (14:33) Luigi then worked as a Machine Learning Engineer at CTRL-Labs - a startup (acquired by Facebook) pioneering the development of non-invasive neural interfaces that reimagine how humans and machines collaborate.
    • (20:45) Luigi discusses the skills he picked up during his transition from Data Engineering to Machine Learning Engineering, such as data analysis, data visualization, dimensionality reduction, and domain expertise.
    • (25:38) Luigi went over his time teaching graduate courses in Applied Statistics & Probability and Big Data Programming at Fordham’s Department of Computer Science.
    • (28:37) Luigi talked about his next role working as a Data Scientist at 2U - an edTech SaaS platform providing schools with the comprehensive operating infrastructure they need to attract, enroll, educate, support, and graduate students globally.
    • (31:12) Luigi emphasized the importance of being good at data science and picking up skills from other functional domains for anyone looking into management roles.
    • (33:47) Luigi shared brief thoughts on the role of ed-tech in the current environment with remote education.
    • (35:47) Luigi unpacked his blog post called How I Hire Data Scientists that shares advice for both the hiring managers and the job applicants.
    • (42:05) Luigi shared his anecdotal journey of starting ML In Production - which provides content on the best practices of doing machine learning in production. Check out this article for more detail!
    • (47:16) Luigi discussed the nuts and bolts of setting up the weekly newsletter for his website.
    • (51:21) Luigi unpacked the 4-part series "Docker for Machine Learning” that discusses the benefits of using Docker with machine learning, how to build custom Docker images, how to perform batch inference using Docker containers, and how to perform online inference using Docker and Flask REST API.
    • (54:26) Luigi’s next post, "Batch Inference vs. Online Inference," discusses the differences between using batch inference or online inference for model serving.
    • (56:51) Luigi’s next post, "Storing Metadata from ML Experiments," reveals the importance of storing metadata during the machine learning process as well as the types of metadata to capture.
    • (01:00:49) Luigi’s following post "How Data Leakage Impacts ML Models" goes over the issues of data leakage, which occurs when data used at training time is unavailable at inference time.
    • (01:04:14) Luigi unpacked his 6-part series that first introduces Kubernetes and then goes deeper into its components, including Pods, Jobs, CronJobs, Deployment, and Services.
    • (01:07:06) Luigi reflected on his talk “Productionizing ML Models at scale with Kubernetes” at the TWIML conference last year.
    • (01:10:50) Luigi dug into his popular post "The Ultimate Guide to Model Retraining," which covers the problem of model drift as well as the necessary steps to retrain models already in production.
    • (01:14:36) Luigi laid out the benefits of using AWS SageMaker for model deployment. Check out his concise description of SageMaker’s architecture as well as his video tutorial on how to train scikit-learn models on SageMaker.
    • (01:17:50) Luigi unpacked his multi-part series on model deployment. So far, he has covered deployment in the machine learning context, software interfaces, batch inference, online inference, model registries, test-driven development, and A/B testing.
    • (01:22:12) Luigi encouraged every software engineer to learn about running ML systems in production, given the gradual shift to Software 2.0, as indicated in his post "Machine Learning is Forcing Software Development to Evolve."
    • (01:26:48) Luigi reveals what differentiates successful industry ML projects from unsuccessful ones, based on his interview series with other ML practitioners. Hint: (1) don’t focus on the hype, instead focus on the business outcomes + (2) start small.
    • (01:30:10) Luigi distinguished the skills required for the three roles: data engineers, data scientists, and machine learning engineers.
    • (01:33:46) Luigi shared his opinions on the data science community in New York City.
    • (01:35:41) Closing segment.

    His Contact Info

    • Website
    • Twitter
    • LinkedIn

    His Recommended Resources

    • MLinProduction.com
    • Monitor! Stop Being a Blind Data Scientist (by Ori Cohen)
    • Andrej Karpathy
    • Lex Fridman
    • Xavier Amatriain
    • “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow” by Aurelien Geron

    A New Course From Luigi

    Luigi just launched his first online course, Build, Deploy, and Monitor Machine Learning Models with Amazon SageMaker! I had a look at the course content, and I’m convinced that the course will be super valuable to any ML engineer or data scientist who wants to level up and learn how to productionize their machine learning models.

    You can take the course on your own, but Luigi is also teaming up with TWiML to offer a version with virtual Study Group sessions for people who want a more interactive experience. Right now, Luigi is offering an early bird discount on the course until August 1st!

    I know the course will be precious for a lot of you within my community, so Luigi created a coupon code DATACAST to save an additional 10% off the course!

    Head over to the Teachable course page to learn more about AmazonSageMaker and take advantage of the discount!



    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 43 min
  • Episode 36: Machine Learning Bookcamp with Alexey Grigorev

    Show Notes

    • (2:00) Alexey studied Information Systems and Technologies from a local university in his hometown in eastern Russia.
    • (4:54) Alexey commented on his experience working as a Java developer in the first three years after college in Russia and Poland, along with his initial exposure to Machine Learning thanks to Coursera.
    • (7:55) Alexey talked about his decision to pursue the IT4BI Master Program specializing in Large-Scale Business Intelligence in 2013.
    • (9:42) Alexey discussed his time working as a Research Assistant on Apache Flink at the DIMA Group at TU Berlin.
    • (12:28) Alexey’s Master Thesis is called Semantification of Identifiers in Mathematics for Better Math Information Retrieval, which was later presented at the SIGIR conference on R&D in Information Retrieval in 2016.
    • (14:35) Alexey discussed his first job as a Data Scientist at Searchmetrics - working on projects to help content marketers improve SEO ranking for their articles.
    • (18:54) Alexey’s next role was with the ad-tech company Simplaex. There, he designed, developed, and maintained the ML infrastructure for processing 3+ billion events per day with 100+ million unique daily users - working with tools like Spark for data engineering tasks.
    • (22:17) Alexey reflected on his journey participating in Kaggle competitions.
    • (25:35) Alexey also participated in other competitions at academic conferences: winning 2nd place at the Web Search and Data Mining 2017 challenge on Vandalism Detection and winning 1st place at the NIPS 2017 challenge on Ad Placement.
    • (29:59) Alexey authored his first book called Mastering Java for Data Science, which teaches readers how to create data science applications with Java.
    • (31:40) Alexey then transitioned to a Data Scientist role at OLX Group, a global marketplace for online classified advertisements.
    • (33:23) Alexey explained the ML system that detects duplicates of images submitted to the OLX marketplace, which he presented at PyData Berlin 2019. Read his two-part blog series: The first post presents a two-step framework for duplicate detection, and the second post explains how his team served and deployed this framework at scale.
    • (38:12) Alexey was recently involved in building an infrastructure for serving image models at OLX. Read his two-part blog series on this evolution of image model serving at OLX, including the transition from AWS SageMaker to Kubernetes for model deployment, as well as the utilization of AWS Athena and MXNet for design simplification.
    • (42:39) Alexey is in the process of writing a technical book called Machine Learning Bookcamp - which encourages readers to learn machine learning by doing projects.
    • (46:17) Alexey discussed common struggles during data science interviews, referring to his talk on Getting a Data Science Job.
    • (48:32) Alexey has put together a neat GitHub page that includes both theoretical and technical questions for people who are preparing for interviews.
    • (52:19) Alexey extrapolated on the steps needed to become a better data scientist, in conjunction to his LinkedIn post a while back.
    • (56:40) Alexey gave his advice for software engineers looking to transition into data science.
    • (58:32) Alexey shared his opinion on the data science community in Berlin.
    • (01:01:53) Closing segment.

    His Contact Info

    • Website
    • Twitter
    • LinkedIn
    • GitHub
    • Kaggle
    • Quora
    • Google Scholar
    • Medium

    His Recommended Resources

    • Apache Flink
    • Kubeflow
    • Data Science Interviews GitHub Repo
    • PyData Berlin
    • Berlin Buzzwords
    • Andrew Ng
    • Designing Data-Intensive Applications by Martin Kleppmann

    Machine Learning Bookcamp

    • Permanent 40$ discount code: poddcast19
    • 5 free eBook codes (each good for one sample of the book): mlbdrt-D452, mlbdrt-5922, mlbdrt-2C4D, mlbdrt-3034, mlbdrt-1DD1


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 6 min
  • Episode 35: Data Science For Food Discovery with Ankit Jain

    Show Notes

    • (2:27) Ankit studied Electrical Engineering with a focus on Communication and Signal Processing at the Indian Institute of Technology, Bombay.
    • (3:27) Ankit then worked for three years as a Senior Field Engineer at Schlumberger, an international oilfield services company.
    • (4:23) Ankit then went to the US to pursue a Masters in Financial Engineering from the Walter Hass School of Business at UC Berkeley.
    • (6:13) Ankit had an opportunity to intern as a data scientist at Facebook during his Masters and worked on detecting spam for Facebook pages.
    • (8:27) Ankit worked full-time as a Quantitative Finance Analyst at Bank of America after finishing his degree, with projects such as building models to identify risk in bank portfolio and analyzing relevance opportunities for strategic investment.
    • (9:46) Ankit discussed his transition to a Data Scientist role at ClearSlide, a B2B platform for Sales Enablement + Engagement.
    • (11:32) Ankit discussed his work on sales forecasting algorithms at ClearSlide.
    • (15:06) In 2015, Ankit moved to Bangalore to become the Head of Data Science and Analytics at Ruunr, a B2B platform that offers hyper-local logistics services that partners with merchants in India.
    • (16:18) Ankit unpacked his thorough post “How Food Delivery Can Be a Sustainable Business” that reflects his experience at Ruunr.
    • (18:35) Ankit talked about the similarities and differences of tech culture in Bangalore and San Francisco.
    • (19:39) Ankit came back to the US and started working as a Data Scientist at Uber in early 2017.
    • (20:34) Ankit discussed his work at Uber on user-level forecasting.
    • (23:12) Ankit talked about the different types of problems that researchers at Uber AI Labs work on.
    • (24:49) Ankit unpacked his in-depth technical post on Uber’s Engineering blog “Food Discovery with Uber Eats: Using Graph Learning to Power Recommendations” — including graph neural networks for food recommendations, the design of the data and training pipeline, and ways to incorporate more data for further improvement.
    • (28:55) Ankit discussed the challenges with building the Uber Eats recommendation system in production.
    • (32:15) Ankit has written a technical book called TensorFlow Machine Learning Projects — which teaches how to exploit the benefits (simplicity, efficiency, and flexibility) of using TensorFlow in various real-world projects.
    • (34:43) Ankit gave his two cents on the battle of frameworks between TensorFlow and PyTorch.
    • (36:26) Ankit shared his advice for academics looking to work in the industry: building end-to-end projects, learning how to build scalable pipelines, and keeping up with important research topics.
    • (38:33) Ankit reflected on the benefits of his electrical engineering and financial analysis education towards his career in data science.
    • (40:11) Closing segment.

    His Contact Info

    • LinkedIn
    • Twitter
    • GitHub
    • Quora

    His Recommended Resources

    • GraphSAGE
    • Meta-Graph: Few-Shot Link Prediction via Meta-Learning
    • Ankit’s book "TensorFlow Machine Learning Projects” published with Packt
    • Andrew Ng
    • Geoffrey Hinton
    • Jeff Dean
    • "Elements of Statistical Learning" by Trevor Hastie, Robert Tibshirani, and Jerome Friedman


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    45 min
  • Episode 34: Deep Learning Generalization, Representation, and Abstraction with Ari Morcos

    Show Notes

    • (2:32) Ari discussed his undergraduate studying Physiology and Neuroscience at UC San Diego, while doing neuroscience research on adult neurogenesis at the Gage Lab.
    • (4:39) Ari discussed his decision to pursue a Ph.D. in Neurobiology at Harvard after college and extracted the importance of communication in research, thanks to his advisor Chris Harvey.
    • (7:16) Ari explained his Ph.D. thesis titled “Population dynamics in parietal cortex during evidence accumulation for decision-making” - in which he developed methods to understand how neuronal circuits perform the computations necessary for complex behavior.
    • (12:59) Ari talked about his process of learning machine learning and using that to analyze massive neuroscience datasets in his research.
    • (15:22) Ari recounted attending NIPS 2015 and serendipitously meeting people from DeepMind, which he lated joined as a Research Scientist in their London office.
    • (18:59) Ari’s research focuses on the generalization of neural networks, and shared his work called "On the Importance of Single Directions for Generalization” presented at ICLR 2018 (inspired by Chiyuan Zhang’s paper and Quoc Le’s paper previously).
    • (28:51) Ari explained the differences between generalizing networks and memorizing networks, citing the results from his work "Insights on Representational Similarity in Neural Networks with Canonical Correlation” with Maithra Raghu and Samy Bengio presented at NeurIPS 2018 (Read Maithra’s paper on SVCCA that inspired it).
    • (35:16) Another topic that Ari focuses on is representation learning and abstraction for intelligent systems. His team at DeepMind proposes a dataset and a challenge designed to probe abstract reasoning, as explained in “Measuring Abstract Reasoning in Neural Networks" presented at ICML 2018 (learn more about the IQ test Raven’s Progressive Matrices and take the challenge here).
    • (42:21) An extension from the work above is "Learning to Make Analogies by Contrasting Abstract Relational Structure" - presented at ICLR 2019. With the same authors (led by Felix Hill along with David Barrett, Adam Santoro, Tim Lillicrap), Ari showed that while architecture choice can influence generalization performance, the choice of data and the manner in which it is presented to the model is even more critical.
    • (48:18) Ari discussed "Neural Scene Representation and Rendering” (led by Ali Eslami and Danilo Rezende) that introduces Generative Query Network (GQN), a framework within which machines learn to represent scenes using only their own sensors (watch the video and check out the data).
    • (55:09) Ari explained the findings in "Analyzing Biological and Artificial Neural Networks: Challenges with Opportunities for Synergy?” published at the Current Opinion in Neurobiology (joint work with David Barrett and Jakob Macke).
    • (57:04) Ari shared the properties of pruning algorithms that influence stability and generalization, as claimed in “The Generalization-Stability Tradeoff in Neural Network Pruning” led by Brian Bartoldson.
    • (01:00:56) Ari went over the generalization of lottery tickets in neural networks, which is inspired by the lottery ticket hypothesis from Jonathan Frankle and Michael Carbin at MIT. The two papers mentioned are collaboration with Haonan Yu, Yuandong Tian, Michela Paganini, and Sergey Edunov (Check out his talk at REWORK Deep Learning Summit in Montreal 2019).
    • (01:09:00) Ari investigated "Training BatchNorm and Only BatchNorm” which looks at the performance of neural networks when trained only with the Batch Normalization parameters (joint work with Jonathan Frankle and David Schwab).
    • (01:12:12) Ari mentioned "The Early Phase of Neural Network Training” (presented at ICML 2020) that uses the lottery ticket framework to rigorously examine the early part of the training (joint work with Jonathan Frankle and David Schwab).
    •  (01:16:25) Ari discussed at length “Representation Learning Through Latent Canonicalizations" (presented at ICLR 2020). This work seeks to learn representations in which semantically meaningful factors of variation (like color or shape) can be independently manipulated by learned linear transformations in latent space, termed “latent canonicalizes” (joint work with Or Litany, Srinath Sridhar, Leonidas Guibas, and Judy Hoffman).
    • (01:22:15) Ari summarized "Selectivity Considered Harmful: Evaluating the Causal Impact of Class Selectivity in DNNs" - which investigates the causal impact of class selectivity on network function (led by Matthew Leavitt).
    • (01:25:26) Ari reflected on his career and shared advice for individuals who want to make a dent in AI research.
    • (01:28:10) Ari shared his excitement on self-supervised learning, which addresses the need of neural networks to require expensive labeled data.
    • (01:29:47) Closing segment.

    His Contact Information

    • Website
    • Google Scholar
    • LinkedIn
    • Twitter
    • GitHub

    His Recommended Resources

    • “Understanding Deep Learning Requires Rethinking Generalization” by Chiyuan Zhang
    • “Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability” by Maithra Raghu
    • Raven’s Progressive Matrices IQ test
    • "The Lottery Ticket Hypothesis” by Jonathan Frankle and Michael Carbin (Open-Source Framework)
    • “Random Features for Large-Scale Kernel Machines” by Ali Rahimi and Ben Recht (NIPS 2017 Test Of Time Award)
    • “beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework” by DeepMind
    • Samy Bengio (Research Scientist at Google AI)
    • Aleksander Madry (Professor of Computer Science at MIT)
    • Jason Yosinski (Founding Member of Uber AI Labs)
    • “The Idea Factory: Bell Labs and The Great Age of American Innovation" by Jon Gertner


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 38 min
  • Episode 33: Domain Randomization in Robotics with Josh Tobin

    Show Notes:

    • (2:02) Josh studied Mathematics at Columbia University during his undergraduate and explained why he was not set out for a career as a mathematician.
    • (3:55) Josh then worked for two years as a Management Consultant at McKinsey.
    • (6:05) Josh explained his decision to go back to graduate school and pursue a Ph.D. in Mathematics at UC Berkeley.
    • (7:23) Josh shared the anecdote of taking a robotics class with professor Pieter Abbeel and switching to a Ph.D. in the Computer Science department at UC Berkeley.
    • (8:50) Josh described the period where he learned programming to make the transition from Math to Computer Science.
    • (10:46) Josh talked about the opportunity to collaborate and then work full-time as a Research Scientist at OpenAI - all during his Ph.D.
    • (12:40) Josh discussed the sim2real problem, as well as the experiments conducted in his first major work "Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model".
    • (17:43) Josh discussed his paper "Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World", which has been cited more than 600 times up until now.
    • (20:51) Josh unpacked the OpenAI’s robotics system that was trained entirely in simulation and deployed on a physical robot, which can learn a new task after seeing it done once (Read the blog post “Robots That Learn” and watch the corresponding video).
    • (24:01) Josh went over his work on Hindsight Experience Replay - a novel technique that can deal with sparse and binary rewards in Reinforcement Learning (Read the blog post “Generalizing From Simulation").
    • (28:41) Josh talked about the paper "Domain Randomization and Generative Models for Robotic Grasping”, which (1) explores a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis; and (2) proposes an autoregressive grasp planning model that maps sensor inputs of a scene to a probability distribution over possible grasps.
    • (32:27) Josh unpacked the design of OpenAI's Dactyl - a reinforcement learning system that can manipulate objects using a Shadow Dexterous Hand (Read the paper “Learning Dexterous In-Hand Manipulation” and watch the corresponding video).
    • (35:31) Josh reflected on his time at OpenAI.
    • (36:05) Josh investigated his most recent work called “Geometry-Aware Neural Rendering” - which tackles the neural rendering problem of understanding the 3D structure of the world implicitly.
    • (28:21) Check out Josh's talk "Synthetic Data Will Help Computer Vision Make the Jump to the Real World" at the 2018 LDV Vision Summit in New York.
    • (28:55) Josh summarized the mental decision tree to debug and improve the performance of neural networks, as a reference to his talk "Troubleshooting Deep Neural Networks” at Reinforce Conf 2019 in Budapest.
    • (41:25) Josh discussed the limitations of domain randomization and what the solutions could look like, as a reference to his talk "Beyond Domain Randomization” at the 2019 Sim2Real workshop in Freiburg.
    • (44:52) Josh emphasized the importance of working on the right problems and focusing on the core principles in machine learning for junior researchers who want to make a dent in the AI research community.
    • (48:30) Josh is a co-organizer of Full-Stack Deep Learning, a training program for engineers to learn about production-ready deep learning.
    • (50:40) Closing segment.

    His Contact Information:

    • Website
    • LinkedIn
    • Twitter
    • GitHub
    • Google Scholar

    His Recommended Resources:

    • Full-Stack Deep Learning
    • Pieter Abbeel
    • Ilya Sutskever
    • Lukas Biewald
    • “Thinking Fast and Slow” by Daniel Kahneman


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    54 min
  • Episode 32: Economics, Data For Good and AI Research with Sara Hooker

    Show Notes:

    • (2:20) Sara shared her childhood growing up in Africa.
    • (4:05) Sara talked about her undergraduate experience at Carleton College studying Economics and International Relations.
    • (9:07) Sara discussed her first job working as an Economics Analyst at Compass Lexecon in the Bay Area.
    • (12:20) Sara then joined Udemy as a data analyst, then transitioned to the engineering team to work on spam detection and recommendation algorithms.
    • (14:58) Sara dig deep into the “hustling period” of her career and how she brute-forced her way to grow as an engineer.
    • (17:24) Sara founded Delta Analytics - a local Bay Area non-profit community of data scientists, engineers, and economists in 2014 that believes in using data for good.
    • (20:53) Sara shared Delta’s collaboration with Eneza Education to empower students to access quizzes by mobile texting in Kenya (check out her presentation at the ODSC West 2016).
    • (25:16) Sara shared Delta’s partnership with Rainforest Connection to identify illegal de-forestation using steamed audio from the rainforest (check out her presentation at MLconf Seattle 2017).
    • (28:22) Sara unpacked her blog post Why “data for good” lacks precision, in which she described 4 key criteria frequently used to qualify an initiative as “data for good” and discussed some open challenges associated with each.
    • (36:34) Sara unpacked her blog post Slow learning, in which she revealed her journey to get accepted into the AI Residency program at Google AI.
    • (41:03) Sara discussed her initial research interest on model interpretability for deep neural networks and her work done at Google called The (Un)reliability of Saliency Methods - which argues that saliency methods are not reliable enough to explain model prediction.
    • (45:55) Sara pushed the research above further with A Benchmark for Interpretability Methods in Deep Neural Networks, which proposes an empirical measure of the approximate accuracy of feature importance estimates in deep neural networks called RemOve And Retrain.
    • (48:46) Sara explained why model interpretability is not always required (check out her talks at PyBay 2018, REWORK Toronto 2018, and REWORK San Francisco 2019).
    • (52:10) Sara explained the typical measurements of model reliability and the limitations of them, such as localization methods and points of failure.
    • (59:04) Sara explained why model compression is an interesting research direction and her work The State of Sparsity in Deep Neural Networks - which highlights the need for large-scale benchmarks in the field of model compression.
    • (01:02:49) Sara discussed her paper Selective Brain Damage: Measuring the Disparate Impact of Model Pruning - which explores the impact of pruning techniques for neural networks trained for computer vision tasks. Check out the paper website!
    • (01:05:08) Sara shared her future research directions on efficient pruning, sparse network training, and local gradient updates.
    • (01:06:56) Sara explained the premise behind her talk Gradual Learning at the Future of Finance Summit in 2019, in which she shared the three fundamental approaches to machine learning impact.
    • (01:12:20) Sara described the AI community in Africa as well as the issues the community is currently facing: both from the investment landscape and the infrastructure ecosystem.
    • (01:18:00) Sara and her brother recently started a podcast called Underrated ML which pitches the underrated ideas in machine learning.
    • (01:20:15) Sara reflected how her background in economics influences her career outlook in machine learning.
    • (01:25:42) Sara reflected on the differences between applied ML and research ML, and shared her advice for people contemplating between these career paths.
    • (01:29:49) Closing segment.

    Her Contact Information:

    • Website
    • LinkedIn
    • GitHub
    • Google Scholar
    • Twitter
    • Medium

    Her Recommended Resources :

    • Deep Learning Indaba
    • Southeast Asia Machine Learning School
    • MILA - AI For Humanity
    • Why “data for good” lacks precision (Sara's take on "Data for Good" initiatives)
    • Slow learning (Sara's journey to Google AI)
    • fast.ai 
    • Sanity Check for Saliency Maps by Julius Adebayo et al.
    • Focal Loss for Dense Object Detection by Tsung-Yi Lin et al.
    • MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications by Andrew Howard et al.
    • Underrated ML (Sara’s new podcast)
    • Dumitru Erhan (Research Scientist at Google AI)
    • Samy Bengio (Research Scientist at Google AI)
    • Andrea Frome (Ex-Research Engineer at Google AI)
    • Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani, and Jerome Friedman


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 34 min
  • Episode 31: From Quantum Computing to Epidemic Modeling with Colleen Farrelly

    Show Notes:

    • (1:58) Colleen gave a brief overview of her professional background and her path to data science.
    • (3:27) Colleen explained how her background in medicine and social science contributes to her success as a data scientist working in different domains.
    • (5:20) Colleen share her thoughts on how data science varies by sector.
    • (7:02) Referring to her consulting company Staticlysm LLC, Colleen discussed a current medical technology project that she is working on.
    • (8:19) Colleen shared applications of quantum machine learning in the wild, referring to her work at Quantopo - where she is the cofounder and chief mathematician.
    • (12:33) Colleen discussed a new project leveraging quantum and quantum-inspired algorithms for nuclear reactor optimization at Quantopo.
    • (15:17) Colleen gave advice for data scientists who want to start a business and get into consulting.
    • (16:58) Colleen discussed topological data analysis in machine learning.
    • (23:02) Colleen discussed epidemic modeling and engaging foreign aid organizations, given her experience with the Ebola outbreak.
    • (26:01) Colleen discussed the different approaches to model the spread of diseases in epidemics, which follow a differential equation framework.
    • (30:22) Colleen explained why buy-in and corporation from those in power are critical to combat epidemics.
    • (32:20) Colleen explained the difference between the current Coronavirus pandemic and the previous Ebola epidemic (note that this conversation is recorded in mid-March 2020, so information about COVID19 may have dated).
    • (37:06) Colleen shared resources to get up-skilled in data science.
    • (39:40) Colleen talked about the benefits of writing on Quora, where she has written more than 13,000 answers.
    • (41:31) Colleen shared the traits of an excellent technical communicator and/or data translator.
    • (43:26) Colleen shared her process of writing a technical book that focuses on the use-cases of topology, geometry, and graph theory in machine learning and data science.
    • (50:32) Colleen talked about the growth of the data science community in Miami.
    • (54:30) Colleen discussed her involvement in data science within the African sectors.
    • (57:55) Colleen shared her thoughts on how the data science field will evolve in the next few years: the access to big data platforms, the dominance of Python, and the rise of quantum computing.
    • (01:01:12) Closing segment.

    Her Contact Info:

    • LinkedIn
    • Quora
    • ResearchGate
    • KDNuggets

    Her Recommended Resources: 

    • IBM Quantum Computing
    • Xanadu (Quantum Computing Hardware)
    • D-Wave Systems (Quantum Computing Hardware)
    • Ayasdi (Topological Data Analysis-Focused Startup)
    • The SIR Model for Spread of Disease
    • Google Scholar
    • arXiv
    • LinkedIn Learning
    • Coursera
    • Genetic Algorithms and Adaptation paper by John Holland
    • Random Forests paper by Leo Breiman
    • Andrew Ng (Founder of Coursera, DeepLearning.AI, and Landing.AI)
    • Dover Series on Mathematics


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 8 min
  • Episode 30: Data Science Evangelism with Parul Pandey

    Show Notes:

    • (2:12) Parul talked about her educational background, studying Electrical Engineering at the National Institute of Technology, Hamirpur.
    • (3:18) Parul worked as a Business Analyst at Tata Power India for 7 years.
    • (4:29) Parul talked about her initial interests in writing about data science and machine learning on Medium.
    • (6:30) Parul discussed her first blog series “A Guide to Machine Learning in R for Beginners” - which covers the Fundamentals of ML, Intro to R, Distributions and EDA in R, Linear Regression, Logistic Regression, and Decision Trees.
    • (8:02) Reference to her articles on data visualization, Parul talked about matplotlib, seaborn, and plotly as the main visualization libraries she practices, in addition to Tableau for building dashboard.
    • (10:11) Parul shared her thoughts on the state of Machine Learning interpretability, in reference to her articles on this topic.
    • (13:54) Parul discussed the advantages of using Jupyter Lab over Jupyter Notebook.
    • (17:30) Parul discussed the common challenges of bringing recommendation systems from prototype into production (Read her two articles about recommendation systems: (1) an overview of different approaches and (2) an overview of the process of designing and building a recommendation system pipeline)
    • (21:00) Parul went in depth into her NLP project called "Building a Simple Chatbot from Scratch in Python (using NLTK).”
    • (23:26) Parul continued this chatbot project with a 2-part series on building a conversational chatbot with Rasa stack and Python and deploying it on Slack.
    • (28:15) Parul went over her Satellite Imagery Analysis with Python piece, which examines the vegetation cover of a region with the help of satellite data.
    • (32:22) Parul talked about the process of Recreating Gapminder in Tableau: A Humble Tribute to Hans Rosling.
    • (35:17) Parul discussed her project Music Genre Classification, which shows how to analyze an audio/music signal in Python.
    • (39:20) Parul went over her tutorials on Computer Vision: (1) Face Detection with Python using OpenCV and (2) Image Segmentation with Python’s scikit-image module.
    • (42:01) Parul unpacked her tutorial "Predicting the Future with Facebook’s Prophet” - a forecasting model to predict the number of views for her Medium articles.
    • (44:58) Parul have been working as a Data Science Evangelist at H2O.AI since July 2019.
    • (47:04) Parul described Flow - H2O's web-based interface (Read her tutorial here).
    • (49:23) Parul described Driverless AI - H2O’s product that automates the challenging and repetitive tasks in applied data science (Read her tutorial here).
    • (52:39) Parul described AutoML - H2O's automation of the end-to-end process of applying ML to real-world problems (Read her tutorial here).
    • (57:07) Parul shared her secret sauce for effective data visualization and storytelling, as illustrated in her analysis of the 2019 Kaggle Survey to figure out women’s representation in machine learning and data science.
    • (01:02:02) Parul described the data science community in Hyderabad, from her lens as an organizer for the Hyderabad Chapter of the Women in Machine Learning and Data Science.
    • (01:05:45) Parul was recognized as a LinkedIn’s Top Voices 2019 in the Software Development category.
    • (01:10:30) Closing segment.

    Her Contact Info:

    • Medium
    • GitHub
    • Twitter
    • LinkedIn
    • Website
    • Kaggle

    Her Recommended Resources:

    • Interpretable Machine Learning post
    • "Interpretable Machine Learning: A Guide for Making Black Box Models Explainable" by Chris Molnar
    • “Towards A Rigorous Science of Interpretable Machine Learning” by Finale Doshi-Velez and Been Kim
    • Parul’s Compilation of Data Visualization articles
    • Parul’s Programming with Python articles
    • Women in Machine Learning and Data Science
    • Rachel Thomas
    • Andreas Mueller
    • HuggingFace
    • “Factfulness” by Hans Rosling


    This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe
    1 hr 15 min

About Datacast

From the publisher's feed

Datacast follows the narrative journey of data practitioners and researchers to unpack the career lessons they learned along the way. James Le hosts the show.