Developer Voices

Reimplementing Apache Kafka with Golang and S3


Listen Later

This week on Developer Voices we’re talking to Ryan Worl, whose career in big data engineering has taken him from DataDog to Co-Founding WarpStream, an Apache Kafka-compatible streaming system that uses Golang for the brains and S3 for the storage.

Ryan tells us about his time at DataDog, along with the things he learnt from doing large-scale systems migration bit-by-bit, before we discuss how and why he started WarpStream. Why re-implement Kafka? What are the practical challenges and cost benefits of moving all your storage to S3? And would he choose Go a second time around?

--

WarpStream: https://www.warpstream.com/

DataDog: https://www.datadoghq.com/

Ryan on Twitter: https://x.com/ryanworl 

Kris on Mastodon: http://mastodon.social/@krisajenkins

Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Kris on Twitter: https://twitter.com/krisajenkins

...more
View all episodesView all episodes
Download on the App Store

Developer VoicesBy Kris Jenkins

  • 5
  • 5
  • 5
  • 5
  • 5

5

30 ratings


More shows like Developer Voices

View all
Software Engineering Radio - the podcast for professional software developers by team@se-radio.net (SE-Radio Team)

Software Engineering Radio - the podcast for professional software developers

273 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

289 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

623 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

582 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

988 Listeners

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

CoRecursive: Coding Stories

188 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

98 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

522 Listeners

Two's Complement by Ben Rady and Matt Godbolt

Two's Complement

32 Listeners

devtools.fm: Developer Tools, Open Source, Software Development by Andrew Lisowski, Justin Bennett

devtools.fm: Developer Tools, Open Source, Software Development

25 Listeners

Software Unscripted by Richard Feldman

Software Unscripted

29 Listeners

Oxide and Friends by Oxide Computer Company

Oxide and Friends

67 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

98 Listeners

Rust in Production by Matthias Endler

Rust in Production

25 Listeners

Fallthrough by Fallthrough Media

Fallthrough

13 Listeners