The GeekNarrator

Can Math simplify incremental compute?


Listen Later

In this episode of The Geek Narrator podcast, Lalit Suresh, CEO of Feldera, joins us to share insights on incremental view maintenance and its significance in modern data processing.We have discussed the challenges posed by distributed systems, the mathematical foundation of DBSP, and how Feldera's architecture addresses these challenges. Performance optimization, handling late events, and the future of stream processing, the importance of SQL in creating efficient data workflows - its all in here.Chapters00:00 Introduction to Incremental View Maintenance06:30 Challenges in Distributed Systems11:46 Batch Processing vs Stream Processing16:27 Understanding DBSP: The Mathematical Foundation27:46 Architecture of Feldera and Data Flow39:23 Partitioning and Storage Layer in Feldera42:51 Understanding Co-Design Storage Layers45:52 Foreground and Background Workers in DBSP49:16 Tuning Background Workers for Performance49:41 Synchronous Compute Model and View Propagation51:35 Zsets and Batch Processing in Stream Workloads54:00 Data Model Optimization in Feldera57:22 Handling Late Events and Lateness in Feldera01:01:18 Watermarks and Lateness Annotations01:04:20 Error Handling and Idempotency in Feldera01:11:05 Feldera's Differentiators and Future Roadmap

...more
View all episodesView all episodes
Download on the App Store

The GeekNarratorBy Kaivalya Apte

  • 5
  • 5
  • 5
  • 5
  • 5

5

3 ratings


More shows like The GeekNarrator

View all
Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

Software Engineering Radio - the podcast for professional software developers

272 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

284 Listeners

Thoughtworks Technology Podcast by Thoughtworks

Thoughtworks Technology Podcast

40 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

590 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

621 Listeners

Odd Lots by Bloomberg

Odd Lots

1,784 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

140 Listeners

Practical AI by Practical AI LLC

Practical AI

192 Listeners

The Stack Overflow Podcast by The Stack Overflow Podcast

The Stack Overflow Podcast

62 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

139 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

408 Listeners

Oxide and Friends by Oxide Computer Company

Oxide and Friends

47 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

461 Listeners

Money Stuff: The Podcast by Bloomberg

Money Stuff: The Podcast

371 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

63 Listeners