In this episode we chat with Frederic from Polar Signals. We dive deep into the intricacies of building a continuous profiler, the challenges faced, and the unique solutions developed by Polar Signals. Frederic shares insights from his background in observability and discusses the innovations in FrostDB, a custom columnar database designed for high-performance query and storage of profiling data.

Chapters:

00:00 Introduction

00:29 Frederic's Background

03:40 What is Continuous Profiling?

06:56 Challenges in Data Collection

18:22 Profiling Data Ingestion and Storage Architecture

27:23 Querying Data

28:52 High Cardinality Data and Cost Optimization

23:39 Tenant Isolation and Load Management

41:24 Performance Optimizations

46:02 Testing & Deterministic Simulation

50:33 Technical and Organizational Learnings

54:32 Future of Polar Signals

56:21 Conclusion

You can check more about Polar Signals here: https://www.polarsignals.com/

Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me.

Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.

https://app.codecrafters.io/join?via=geeknarrator

If you like this episode, please hit the like button and share it with your network.

Also please subscribe if you haven't yet.

Database internals series: https://youtu.be/yV_Zp0Mi3xs

Popular playlists:

Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-

Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17

Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d

Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN

Stay Curios! Keep Learning!

#distributedsystems #systemdesign