Sign up to save your podcasts
Or
Share Messaging and Streaming with Apache Pulsar - with Matteo Merli
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Messaging and Streaming with Apache Pulsar - with Matteo Merli
In this video I talk about Apache Pulsar with Matteo Merli, CTO at StreamNative. This episode will provide you good insight about how Apache Pulsar works and more importantly differs with the most popular Pub/Sub and streaming platform Apache Kafka. Things like, what enables possibility of 1 million topics? Why is rebalancing not required? How does decoupled storage and compute architecture works? How it uses the concept of Subscriptions to avoid retaining data unnecessarily?
And much more...
Chapters:
00:00 Introduction and Guest Introduction
00:08 Understanding Apache Pulsar and its Origin
01:22 The Problem Apache Pulsar was Designed to Solve
02:35 The Evolution of Apache Pulsar
05:15 Understanding Basic Concepts of Apache Pulsar
09:27 Deep Dive into Apache Pulsar's Architecture
21:16 Understanding the Flow of Data in Apache Pulsar
28:54 Understanding Subscriptions in Apache Pulsar
31:57 Understanding End-to-End Latency and Subscription Creation
32:32 Broker's Role and Handling Metadata
33:05 Memory Management and Consumer Handling
34:07 Message Processing and Flow Control
34:32 Message Storage and Retrieval
36:00 Comparing Pulsar with Kafka
43:52 Understanding Multi-Tenancy in Pulsar
49:17 Exploring Tiered Storage and Future Developments
Important links:
StreamNative: https://streamnative.io/
Apache Pulsar: https://pulsar.apache.org/
Matteo Merli: https://twitter.com/merlimat
===============================================================================
For discount on the below courses:
Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
===============================================================================
Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
View all episodes
By Kaivalya Apte
5
33 ratings
In this video I talk about Apache Pulsar with Matteo Merli, CTO at StreamNative. This episode will provide you good insight about how Apache Pulsar works and more importantly differs with the most popular Pub/Sub and streaming platform Apache Kafka. Things like, what enables possibility of 1 million topics? Why is rebalancing not required? How does decoupled storage and compute architecture works? How it uses the concept of Subscriptions to avoid retaining data unnecessarily?
And much more...
Chapters:
00:00 Introduction and Guest Introduction
00:08 Understanding Apache Pulsar and its Origin
01:22 The Problem Apache Pulsar was Designed to Solve
02:35 The Evolution of Apache Pulsar
05:15 Understanding Basic Concepts of Apache Pulsar
09:27 Deep Dive into Apache Pulsar's Architecture
21:16 Understanding the Flow of Data in Apache Pulsar
28:54 Understanding Subscriptions in Apache Pulsar
31:57 Understanding End-to-End Latency and Subscription Creation
32:32 Broker's Role and Handling Metadata
33:05 Memory Management and Consumer Handling
34:07 Message Processing and Flow Control
34:32 Message Storage and Retrieval
36:00 Comparing Pulsar with Kafka
43:52 Understanding Multi-Tenancy in Pulsar
49:17 Exploring Tiered Storage and Future Developments
Important links:
StreamNative: https://streamnative.io/
Apache Pulsar: https://pulsar.apache.org/
Matteo Merli: https://twitter.com/merlimat
===============================================================================
For discount on the below courses:
Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
===============================================================================
Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
More shows like The GeekNarrator
View all
Software Engineering Daily
622 Listeners
TED Tech
396 Listeners
Up First from NPR
56,524 Listeners
Dwarkesh Podcast
517 Listeners
The Daily Brief
14 Listeners
The Pragmatic Engineer
64 Listeners