The GeekNarrator

The GeekNarrator

By Kaivalya ApteTechnology
Download on the App Store

The GeekNarrator episodes

  • Building a continuous profiler with Frederic from Polar Signals

    Building a Continuous Profiler with Frederic from Polar Signals | Geek Narrator Podcast

    In this episode we chat with Frederic from Polar Signals. We dive deep into the intricacies of building a continuous profiler, the challenges faced, and the unique solutions developed by Polar Signals. Frederic shares insights from his background in observability and discusses the innovations in FrostDB, a custom columnar database designed for high-performance query and storage of profiling data.
    Chapters:
    00:00 Introduction
    00:29 Frederic's Background
    03:40 What is Continuous Profiling?
    06:56 Challenges in Data Collection
    18:22 Profiling Data Ingestion and Storage Architecture
    27:23 Querying Data
    28:52 High Cardinality Data and Cost Optimization
    23:39 Tenant Isolation and Load Management
    41:24 Performance Optimizations
    46:02 Testing & Deterministic Simulation
    50:33 Technical and Organizational Learnings
    54:32 Future of Polar Signals
    56:21 Conclusion
    You can check more about Polar Signals here: https://www.polarsignals.com/
    Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me.
    Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
    https://app.codecrafters.io/join?via=geeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #systemdesign

    58 min
  • Database Internals - SlateDB with Chris Riccomini

    Welcome back to another episode! Today, I have a special guest, Chris Riccomini, joining me to delve into the exciting world of databases. In this episode, we focus on SlateDB, a new and innovative database that's making waves in the tech community. We'll cover a wide range of topics, including the architecture of SlateDB, its internals, design decisions, and some fascinating use cases. Chris, a seasoned software engineer with a background at LinkedIn and WePay, shares his journey and the motivations behind creating SlateDB. 🎙️

    Chatpers:
    00:00 Introduction to the Topic and Guest
    01:58 Chris Riccomini's Background and Experience
    04:19 The Genesis of SlateDB
    04:54 Understanding SlateDB's Architecture
    10:22 The Rise of Object Storage in Databases
    13:43 Exploring SlateDB's Features and Trade-offs
    32:54 Understanding Latency Trade-offs
    34:12 Exploring Storage Formats and Manifest Files
    37:25 Caching Strategies and Optimizations in SlateDB
    50:21 Consistency Guarantees and Transactionality
    52:36 Integration and Resource Management in SlateDB
    56:04 Future Prospects and Use Cases for SlateDB
    SlateDB: https://slatedb.io/
    More about Chris: https://cnr.sh/
    Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
    https://app.codecrafters.io/join?via=geeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #systemdesign #formalmethods

    1 hr 2 min
  • System Design the formal way with FizzBee

    In this video I talk to Jayaprabhakar Kadarkarai aka JP who is the founder of FizzBee. FizzBee is a design specification language and model checker to help developers verify their design before writing even a single line of implementation code.

    We have discussed where it is applicable, what are the benefits, how does it work and many other interesting challenges with examples.
    Chapters:
    00:00 Introduction
    01:13 Challenges in Designing Distributed Systems
    03:13 Understanding Design Specification Languages
    04:00 The Value of Structured Design Documents
    09:00 When to Use Design Specification Languages
    21:27 Modeling a Travel Booking System
    22:51 Ensuring Atomicity in Distributed Systems
    26:09 Handling Failures and Consistency
    34:45 Refinement in System Design
    35:38 Balancing Abstraction and Implementation
    37:53 Common Pitfalls in Modeling and Implementation
    40:02 Challenges in System Design and Implementation
    40:12 Two-Way Feedback in System Design
    41:01 Performance Considerations in Implementation
    41:36 Importance of Solid Design Blueprints
    41:56 Model-Based Testing and Continuous Integration
    43:27 Updating Design Documentation
    44:38 Simulation Testing vs. Model Checking
    45:32 Design Issues and Formal Verification
    49:51 Applying Formal Verification to Existing Systems
    55:35 Common Design Problems and Solutions
    01:07:57 Future Enhancements in Design Specification Tools
    01:12:50 Getting Started with FizzBee
    FizzBee : https://fizzbee.io/
    Get in touch with JP: https://www.linkedin.com/in/jayaprabhakar
    Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
    https://app.codecrafters.io/join?via=geeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #systemdesign #formalmethods

    1 hr 17 min
  • Learnings from building Open Source Distributed Systems with Kishore Gopalakrishna

    In this episode of The Geek Narrator podcast, hosted by Kaivalya Apte, we welcome a special guest, Kishore Gopalakrishna from StarTree, co-author of Apache Pinot and other notable projects. Kishore shares his extensive experience in building real-time analytics and streaming systems, including Apache Pino, Espresso, Apache Helix, and Third Eye. The episode delves into the motivations and challenges behind creating these systems, the innovations they brought to distributed systems, and the impact of community on open-source projects. Kishore also discusses the evolution of testing methodologies, cost optimizations in transactional and analytical systems, and key considerations for companies evaluating real-time analytics solutions.

    Don't miss this in-depth conversation packed with valuable insights for both seasoned developers and tech enthusiasts!
    Chapters:
    00:00 Introduction
    03:13 Building Distributed Systems at LinkedIn
    08:57 Testing and Challenges in Distributed Systems
    30:50 Advantages of Columnar Storage
    33:04 The Importance of Upserts
    34:24 Building a Strong Open Source Community
    41:10 Challenges and Lessons in System Design
    51:35 Real-Time Analytics: Do You Need It?
    StarTree: https://startree.ai/
    Apache Pinot: https://pinot.apache.org/
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #kafka #s3 #streaming #realtimeanalytics #database #pinot #startree

    1 hr 1 min
  • WarpStream: A drop-in replacement for Kafka

    In this episode of The GeekNarrator podcast, host Kaivalya Apte interviews Ryan and Richie, the founders of WarpStream. They discuss the architecture, benefits, and core functionalities of WarpStream, a drop-in replacement for Apache Kafka. The conversation covers their experience with Kafka, the design decisions behind WarpStream, and the operational challenges it addresses. They also delve into the seamless migration process, the scalability, and cost benefits, the integration with the Kafka ecosystem, and potential future features. This episode is a must-watch for developers and tech enthusiasts interested in modern, distributed data streaming solutions.

    Chapters:
    00:00 Introduction
    02:27 Introducing Warpstream: A Kafka Replacement
    11:07 Deep Dive into Warpstream's Architecture
    35:42 Exploring Kafka's Ordering Guarantees
    36:52 Handling Buffering and Compaction
    38:44 Efficient Data Reading and File Caching
    44:06 WarpStream's Flexibility and Cost Efficiency
    01:06:59 Future Features
    Links:
    WarpStream : https://www.warpstream.com/
    Blog: https://www.warpstream.com/blog
    X:
    Ryan: https://x.com/ryanworl
    Richard Artoul: https://x.com/richardartoul
    Kaivalya Apte: https://x.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #kafka #s3 #streaming

    1 hr 13 min
  • XTDB - An Immutable SQL Database

    Exploring XTDB with Jeremy Taylor & Malcolm Sparks: An In-Depth Dive into Immutability and Database Internals

    In this episode of the Geek Narrator Podcast, host Kaivalya is joined by Jeremy Taylor and Malcolm Sparks from Juxt to explore XTDB, an immutable database designed to handle complex historical and financial data with precision. They delve into the architecture, internal mechanics, and use cases while discussing the importance of immutability.
    This episode covers everything you need to know about XTDB and its capabilities. Whether you're a developer interested in databases or someone curious about data management and history tracking, this discussion offers invaluable insights.
    Chapters:
    00:00 Introduction
    02:51 Challenges with General Purpose Databases
    11:50 XTDB: A New Approach to Databases
    31:56 Understanding Kafka and XTDB Integration
    36:06 Querying and Indexing in XTDB
    40:31 Temporal Data Management and Use Cases
    54:52 Deployment and User Experience
    XTDB: https://xtdb.com/
    XTDB Github: https://github.com/xtdb/xtdb
    Juxt: https://www.juxt.pro/
    Juxt Github: https://github.com/juxt
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #sql #kafka #datastorage #immutable

    1 hr 7 min
  • Testing Distributed Systems the right way ft. Will Wilson

    In this episode of The GeekNarrator podcast, host Kaivalya Apte dives into the complexities of testing distributed systems with Will Wilson from Antithesis. If you’re grappling with the challenges of testing databases, micro-services, and distributed systems, this episode is a must-watch. Will Wilson demystifies the concept of deterministic simulation testing, shares insights about its advantages over conventional testing methods, and explains how Antithesis helps developers ensure software reliability. Learn about the various strategies and techniques used to identify and resolve bugs, and explore how deterministic simulation can transform your software testing approach. Perfect for developers, engineers, and tech enthusiasts who are keen on improving their testing methodologies for complex systems.

    Chapters:
    00:00 Introduction
    03:04 Limitations of Conventional Testing Methods
    04:09 Understanding Deterministic Simulation Testing
    08:07 Implementing Deterministic Simulation Testing
    14:30 Real-World Example: Chat Application
    19:56 Antithesis Hypervisor and Determinism
    27:06 Defining Properties and Assertions
    38:34 Optimizing Snapshot Efficiency
    40:44 Understanding Isolation in CI/CD Pipelines
    43:39 Strategies for Effective Bug Detection
    47:59 Exploring Program State Trees
    51:17 Heuristics and Fuzzing Techniques
    01:01:56 Mocking Third-Party APIs
    01:05:54 Handling Long-Running Tests
    01:09:06 Classifying and Prioritizing Bugs
    01:15:35 Future Plans and Closing Remarks
    References:
    Hypervisor: https://antithesis.com/blog/deterministic_hypervisor/
    AFL : https://github.com/google/AFL
    Antithesis website: https://antithesis.com/
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #databases #microservices #antithesis #fuzzer #testing

    1 hr 18 min
  • Turso - SQLite for production

    Exploring Turso with Glauber Costa: Insights on SQLite for Production

    In this episode of The GeekNarrator podcast, host Kaivalya Apte interviews Glauber Costa, founder and CEO of TursoDB. They discuss the inception of TursoDB, Glauber's background in Linux kernel development, and the journey from unikernel projects to founding a database company. Glauber explains TursoDB's enhancements to SQLite for production use, including native replication, schema management, and vector search capabilities. The conversation dives deep into use cases, architecture, and the benefits of a multi-tenant database design. Learn about TursoDB’s future plans and essential insights for developers.
    Chapters:
    00:00 Introduction
    05:05 The Birth of Turso
    08:02 Challenges and Pivot to libSQL
    17:12 SQLite for Production: Enhancements and Features
    22:02 Replication and Backup Solutions
    23:38 Enterprise-Level Features and Multi-Tenancy
    25:55 User Experience and Simplicity of TursoDB
    33:14 Handling Network Failures and Monitoring
    36:35 Native Replication in SQLite
    37:52 Virtualizing the Write-Ahead Log
    39:20 Replication Mechanisms
    41:31 Primary and Replica Dynamics
    46:51 Multi-Tenancy and Scalability
    53:33 Schema Changes and Migrations
    58:51 Vector Search Capabilities
    01:02:13 Future Roadmap and Features
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #distributedsystems #databases #sqlite #sql

    1 hr 5 min
  • Taking Postgres to the next level with Neon

    Deep Dive into Serverless Databases with Neon: Featuring Heikki Linnakangas

    In this episode of the Geek Narrator podcast, host Kaivalya Apte is joined by Heikki Linnakangas, co-founder of Neon, to explore the innovative world of serverless databases. They discuss Neon's unique approach to separating compute and storage, the benefits of serverless architecture for modern applications, and dive into various compelling use cases. They also cover Neon's architectural features like branching, auto-scaling, and auto-suspend, making it a powerful tool for both developers and enterprises. Whether you're curious about multi-tenancy, fault tolerance, or developer productivity, this episode offers insightful knowledge about leveraging Neon's capabilities for your next project.
    00:00 Introduction
    00:53 The Birth of Neon: Why It Was Created
    02:16 Understanding Serverless Databases
    07:06 Neon's Architecture: Separation of Compute and Storage
    09:59 Exploring Branching in Neon
    18:21 Auto Scaling and Handling Spikes in Traffic
    20:17 The Challenge of Multiple Writers in Distributed Systems
    22:51 Auto Suspend: Cost-Effective Database Management
    26:02 Optimizing Cold Start Times
    27:14 Balancing Cost and Performance
    28:52 Replication and Durability
    30:32 Understanding the Storage Layer
    34:02 Custom LSM Tree Implementation
    36:21 Fault Tolerance and Failover
    07:00 Developer Productivity and Use Cases
    42:56 Migration and Tooling
    48:35 Future Roadmap and User Experience
    50:28 Conclusion and Final Thoughts
    Neon website: https://neon.tech/
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #PostgreSQL #SQL #RDBMS #NEON

    51 min
  • Scaling Derived Data for Planet-Scale Applications at Linkedin

    In this video I speak with Felix GV, who is a Principal Staff Engineer at Linkedin, and has done major contributions to the data infrastructure and Linkedin, including VeniceDB.

    This episode will give you a good understanding of why we need a new database for storing "Derived Data" in a low latency, high performance manner, which is very important for Machine Learning workloads.
    Chapters:
    00:00 Introduction
    01:42 The Evolution of LinkedIn's Databases
    03:15 Challenges with Voldemort and the Birth of VeniceDB
    08:42 Understanding Derived Data
    13:33 Planet-Scale Applications and Multi-Region Support
    17:40 Writing Data into VeniceDB
    22:53 Merging Data in VeniceDB
    40:31 Understanding the Architecture
    40:47 Components of the Write Path
    41:56 Leader and Follower Architecture
    43:58 Partitioning and DaVinci Client
    47:57 Read Patterns and Client Options
    54:25 Fault Tolerance and Recommender Systems
    01:01:19 Kafka Integration and Deployment
    01:06:56 Roadmap and Future Improvements
    Important links:
    VeniceDB blog: https://www.linkedin.com/blog/engineering/open-source/open-sourcing-venice-linkedin-s-derived-data-platform
    VeniceDB docs: https://venicedb.org/
    Qcon: https://youtu.be/pJeg4V3JgYo?si=vblGUxp5fNdKPHoC
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #kafka #linkedin #venicedb #Rocksdb

    1 hr 13 min

About The GeekNarrator

From the publisher's feed

The GeekNarrator podcast is a show hosted by Kaivalya Apte who is a Software Engineer and loves to talk about Technology, Technical Interviews, Self Improvement, Best Practices and Hustle.