In this episode, we talk to Seth Wiseman the Director of Field Engineering at Materialize and Apache Flink Committer about streaming databases. He is not a fan of the term streaming database - "Streaming is just an implementation of a database."

Enter Materialize. The name itself carries the promise of a new era. The market is growing hungry to understand the potential of materialized views. While the perception is that streaming might be complex, it's worth noting that companies like Uber have invested significantly in technologies like Apache Flink, boasting a team of 30 engineers solely dedicated to this endeavor. The market demand for such solutions has never been more apparent, but curiously, the "streaming engineer" title remains relatively niche.

As we delve deeper, we uncover the internal distinctions between Flink and Materialize, tracing the evolution of streaming databases. Flink plays the role of the compute, while Materialize aims to build out the database itself. The journey from Spark to Flink reveals unique internals, showcasing the distinction between pipeline architectures and the streaming paradigm. Seth provides an exciting perspective on streaming and streaming databases in general and how Materialize differs from Flink and Spark.

Hubert’s Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Expense it brah.

Get full access to SUP! Hubert’s Substack at hubertdulay.substack.com/subscribe