Linear Digressions

Optimized Optimized Web Crawling


Listen Later

Last week’s episode, about methods for optimized web crawling logic, left off on a bit of a cliffhanger: the data scientists had found a solution to the problem, but it wasn’t something that the engineers (who own the search codebase, remember) liked very much. It was black-boxy, hard to parallelize, and introduced a lot of complexity to their code. This episode takes a second crack, where we formulate the problem a little differently and end up with a different, arguably more elegant solution.
Relevant links:
http://www.unofficialgoogledatascience.com/2018/07/by-bill-richoux-critical-decisions-are.html
http://www.csc.kth.se/utbildning/kth/kurser/DD3364/Lectures/KKT.pdf
...more
View all episodesView all episodes
Download on the App Store

Linear DigressionsBy Ben Jaffe and Katie Malone

  • 4.8
  • 4.8
  • 4.8
  • 4.8
  • 4.8

4.8

352 ratings


More shows like Linear Digressions

View all
Global News Podcast by BBC World Service

Global News Podcast

7,654 Listeners

Data Skeptic by Kyle Polich

Data Skeptic

476 Listeners

The Daily by The New York Times

The Daily

110,759 Listeners

Up First from NPR by NPR

Up First from NPR

55,948 Listeners

What's That Rash? by ABC listen

What's That Rash?

248 Listeners

The Ezra Klein Show by New York Times Opinion

The Ezra Klein Show

15,481 Listeners

Prof G Markets by Vox Media Podcast Network

Prof G Markets

1,166 Listeners