Gradient Dissent: Conversations on AI

Stephan Fabel is Senior Director of Infrastructure Systems & Software at NVIDIA, where he works on Base Command, a software platform to coordinate access to NVIDIA's DGX SuperPOD infrastructure.

Lukas and Stephan talk about why having a supercomputer is one thing but using it effectively is another, why a deeper understanding of hardware on the practitioner level is becoming more advantageous, and which areas of the ML tech stack NVIDIA is looking to expand into.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-stephan-fabel

---

Timestamps:

0:00 Intro

1:09 NVIDIA Base Command and DGX SuperPOD

10:33 The challenges of multi-node processing at scale

18:35 Why it's hard to use a supercomputer effectively

25:14 The advantages of de-abstracting hardware

29:09 Understanding Base Command's product-market fit

36:59 Data center infrastructure as a value center

42:13 Base Command's role in tech stacks

47:16 Why crowdsourcing is underrated

49:24 The challenges of scaling beyond a POC

51:39 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts​​

👉 Google Podcasts: http://wandb.me/google-podcasts​

👉 Spotify: http://wandb.me/spotify​

Chris Padwick is Director of Computer Vision Machine Learning at Blue River Technology, a subsidiary of John Deere. Their core product, See & Spray, is a weeding robot that identifies crops and weeds in order to spray only the weeds with herbicide.

Chris and Lukas dive into the challenges of bringing See & Spray to life, from the hard computer vision problem of classifying weeds from crops, to the engineering feat of building and updating embedded systems that can survive on a farming machine in the field. Chris also explains why user feedback is crucial, and shares some of the surprising product insights he's gained from working with farmers.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-chris-padwick

---

Connect with Chris:

📍 LinkedIn: https://www.linkedin.com/in/chris-padwick-75b5761/

📍 Blue River on Twitter: https://twitter.com/BlueRiverTech

---

Timestamps:

0:00 Intro

1:09 How does See & Spray reduce herbicide usage?

9:15 Classifying weeds and crops in real time

17:45 Insights from deployment and user feedback

29:08 Why weed and crop classification is surprisingly hard

37:33 Improving and updating models in the field

40:55 Blue River's ML stack

44:55 Autonomous tractors and upcoming directions

48:05 Why data pipelines are underrated

52:10 The challenges of scaling software & hardware

54:44 Outro

55:55 Bonus: Transporters and the singularity

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts​​

👉 Google Podcasts: http://wandb.me/google-podcasts​

👉 Spotify: http://wandb.me/spotify​

Kathryn Hume is Vice President Digital Investments Technology at the Royal Bank of Canada (RBC). At the time of recording, she was Interim Head of Borealis AI, RBC's research institute for machine learning.

Kathryn and Lukas talk about ML applications in finance, from building a personal finance forecasting model to applying reinforcement learning to trade execution, and take a philosophical detour into the 17th century as they speculate on what Newton and Descartes would have thought about machine learning.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-kathryn-hume

---

Connect with Kathryn:

📍 Twitter: https://twitter.com/humekathryn

📍 Website: https://quamproxime.com/

---

Timestamps:

0:00 Intro

0:54 Building a personal finance forecasting model

10:54 Applying RL to trade execution

18:55 Transparent financial models and fairness

26:20 Semantic parsing and building a text-to-SQL interface

29:20 From comparative literature and math to product

37:33 What would Newton and Descartes think about ML?

44:15 On sentient AI and transporters

47:33 Why casual inference is under-appreciated

49:25 The challenges of integrating models into the business

51:45 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts​​

👉 Google Podcasts: http://wandb.me/google-podcasts​

👉 Spotify: http://wandb.me/spotify​

Sean McClain is the founder and CEO, and Gregory Hannum is the VP of AI Research at Absci, a biotech company that's using deep learning to expedite drug discovery and development.

Lukas, Sean, and Greg talk about why Absci started investing so heavily in ML research (it all comes back to the data), what it'll take to build the GPT-3 of DNA, and where the future of pharma is headed. Sean and Greg also share some of the challenges of building cross-functional teams and combining two highly specialized fields like biology and ML.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-sean-and-greg

---

Connect with Sean and Greg:

📍 Sean's Twitter: https://twitter.com/seanrmcclain

📍 Greg's Twitter: https://twitter.com/gregory_hannum

📍 Absci's Twitter: https://twitter.com/abscibio

---

Timestamps:

0:00 Intro

0:53 How Absci merges biology and AI

11:24 Why Absci started investing in ML

19:00 Creating the GPT-3 of DNA

25:34 Investing in data collection and in ML teams

33:14 Clinical trials and Absci's revenue structure

38:17 Combining knowledge from different domains

45:22 The potential of multitask learning

50:43 Why biological data is tricky to work with

55:00 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts​​

👉 Google Podcasts: http://wandb.me/google-podcasts​

👉 Spotify: http://wandb.me/spotify​

You might know him as the host of Gradient Dissent, but Lukas is also the CEO of Weights & Biases, a developer-first ML tools platform!

In this special episode, the three W&B co-founders — Chris (CVP), Shawn (CTO), and Lukas (CEO) — sit down to tell the company's origin stories, reflect on the highs and lows, and give advice to engineers looking to start their own business.

Chris reveals the W&B server architecture (tl;dr - React + GraphQL), Shawn shares his favorite product feature (it's a hidden frontend layer), and Lukas explains why it's so important to work with customers that inspire you.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-wandb-cofounders

---

Connect with us:

📍 Chris' Twitter: https://twitter.com/vanpelt

📍 Shawn's Twitter: https://twitter.com/shawnup

📍 Lukas' Twitter: https://twitter.com/l2k

📍 W&B's Twitter: https://twitter.com/weights_biases

---

Timestamps:

0:00 Intro

1:29 The stories behind Weights & Biases

7:45 The W&B tech stack

9:28 Looking back at the beginning

11:42 Hallmark moments

14:49 Favorite product features

16:49 Rewriting the W&B backend

18:21 The importance of customer feedback

21:18 How Chris and Shawn have changed

22:35 How the ML space has changed

28:24 Staying positive when things look bleak

32:19 Lukas' advice to new entrepreneurs

35:29 Hopes for the next five years

38:09 Making a paintbot & model understanding

41:30 Biggest bottlenecks in deployment

44:08 Outro

44:38 Bonus: Under- vs overrated technologies

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts​​

👉 Google Podcasts: http://wandb.me/google-podcasts​

👉 Spotify: http://wandb.me/spotify​

Pete is the Technical Lead of the TensorFlow Micro team, which works on deep learning for mobile and embedded devices.

Lukas and Pete talk about hacking a Raspberry Pi to run AlexNet, the power and size constraints of embedded devices, and techniques to reduce model size. Pete also explains real world applications of TensorFlow Lite Micro and shares what it's been like to work on TensorFlow from the beginning.

The complete show notes (transcript and links) can be found here: http://wandb.me/gd-pete-warden

---

Connect with Pete:

📍 Twitter: https://twitter.com/petewarden

📍 Website: https://petewarden.com/

---

Timestamps:

0:00 Intro

1:23 Hacking a Raspberry Pi to run neural nets

13:50 Model and hardware architectures

18:56 Training a magic wand

21:47 Raspberry Pi vs Arduino

27:51 Reducing model size

33:29 Training on the edge

39:47 What it's like to work on TensorFlow

47:45 Improving datasets and model deployment

53:05 Outro

---

Subscribe and listen to our podcast today!

👉 Apple Podcasts: http://wandb.me/apple-podcasts​​

👉 Google Podcasts: http://wandb.me/google-podcasts​

👉 Spotify: http://wandb.me/spotify​

Pieter is the Chief Scientist and Co-founder at Covariant, where his team is building universal AI for robotic manipulation. Pieter also hosts The Robot Brains Podcast, in which he explores how far humanity has come in its mission to create conscious computers, mindful machines, and rational robots.

Lukas and Pieter explore the state of affairs of robotics in 2021, the challenges of achieving consistency and reliability, and what it'll take to make robotics more ubiquitous. Pieter also shares some perspective on entrepreneurship, from how he knew it was time to commercialize Gradescope to what he looks for in co-founders to why he started Covariant.

Show notes: http://wandb.me/gd-pieter-abbeel

---

Connect with Pieter:

📍 Twitter: https://twitter.com/pabbeel

📍 Website: https://people.eecs.berkeley.edu/~pabbeel/

📍 The Robot Brains Podcast: https://www.therobotbrains.ai/

---

Timestamps:

0:00 Intro

1:15 The challenges of robotics

8:10 Progress in robotics

13:34 Imitation learning and reinforcement learning

21:37 Simulated data, real data, and reliability

27:53 The increasing capabilities of robotics

36:23 Entrepreneurship and co-founding Gradescope

44:35 The story behind Covariant

47:50 Pieter's communication tips

52:13 What Pieter's currently excited about

55:08 Focusing on good UI and high reliability

57:01 Outro

In this episode we're joined by Chris Albon, Director of Machine Learning at the Wikimedia Foundation.

Lukas and Chris talk about Wikimedia's approach to content moderation, what it's like to work in a place so transparent that even internal chats are public, how Wikimedia uses machine learning (spoiler: they do a lot of models to help editors), and why they're switching to Kubeflow and Docker. Chris also shares how his focus on outcomes has shaped his career and his approach to technical interviews.

Show notes: http://wandb.me/gd-chris-albon

---

Connect with Chris:

- Twitter: https://twitter.com/chrisalbon

- Website: https://chrisalbon.com/

---

Timestamps:

0:00 Intro

1:08 How Wikimedia approaches moderation

9:55 Working in the open and embracing humility

16:08 Going down Wikipedia rabbit holes

20:03 How Wikimedia uses machine learning

27:38 Wikimedia's ML infrastructure

42:56 How Chris got into machine learning

46:43 Machine Learning Flashcards and technical interviews

52:10 Low-power models and MLOps

55:58 Outro

In this episode, Emily and Lukas dive into the problems with bigger and bigger language models, the difference between form and meaning, the limits of benchmarks, and why it's important to name the languages we study.

Show notes (links to papers and transcript): http://wandb.me/gd-emily-m-bender

---

Emily M. Bender is a Professor of Linguistics at and Faculty Director of the Master's Program in Computational Linguistics at University of Washington. Her research areas include multilingual grammar engineering, variation (within and across languages), the relationship between linguistics and computational linguistics, and societal issues in NLP.

---

Timestamps:

0:00 Sneak peek, intro

1:03 Stochastic Parrots

9:57 The societal impact of big language models

16:49 How language models can be harmful

26:00 The important difference between linguistic form and meaning

34:40 The octopus thought experiment

42:11 Language acquisition and the future of language models

49:47 Why benchmarks are limited

54:38 Ways of complementing benchmarks

1:01:20 The #BenderRule

1:03:50 Language diversity and linguistics

1:12:49 Outro

Jeff talks about building Facebook's early data team, founding Cloudera, and transitioning into biomedicine with Hammer Lab and Related Sciences.

(Read more: http://wandb.me/gd-jeff-hammerbacher)

---

Jeff Hammerbacher is a scientist, software developer, entrepreneur, and investor. Jeff's current work focuses on drug discovery at Related Sciences, a biotech venture creation firm that he co-founded in 2020.

Prior to his work at Related Sciences, Jeff was the Principal Investigator of Hammer Lab, a founder and the Chief Scientist of Cloudera, an Entrepreneur-in-Residence at Accel, and the manager of the Data team at Facebook.

---

Follow Gradient Dissent on Twitter: https://twitter.com/weights_biases

---

0:00 Sneak peek, intro

1:13 The start of Facebook's data science team

6:53 Facebook's early tech stack

14:20 Early growth strategies at Facebook

17:37 The origin story of Cloudera

24:51 Cloudera's success, in retrospect

31:05 Jeff's transition into biomedicine

38:38 Immune checkpoint blockade in cancer therapy

48:55 Data and techniques for biomedicine

53:00 Why Jeff created Related Sciences

56:32 Outro