The GeekNarrator

The GeekNarrator

By Kaivalya ApteTechnology
Download on the App Store

The GeekNarrator episodes

  • Volt Active Data: Low Latency Stream processing

    In this episode of The GeekNarrator podcast, our host Kaivalya talks to Seeta Somagani from Volt Active Data, a low latency stream processing platform. They discuss fascinating topics about what low latency stream processing means, the different guarantees that Volt Active Data provides, and the various problems it can solve. They delve into the evolution of VoltDB to Volt Active Data, real-time data processing use cases, the high-level architecture, and how the platform effectively addresses high-concurrency challenges. This is a must-listen for anyone interested in understanding latency critical applications, data processing, and high performance computing.

    Chapters:
    00:00 Welcome to The GeekNarrator Podcast with Special Guest from Volt Active Data
    00:41 Introduction
    01:45 The Evolution of VoltDB to Volt Active Data
    06:13 Exploring Real-Time Data Processing and Use Cases
    08:25 Addressing High-Concurrency Challenges in Various Industries
    12:57 High-Level Architecture of Volt Active Data
    19:26 Understanding Stored Procedures and Data Processing in Volt
    22:48 Practical Application: Tracking Data Usage with Volt Active Data
    25:16 Diving into Replicated and Partitioned Tables
    25:44 Exploring Event Processing and Exporting
    26:57 Understanding Stored Procedures and Performance
    29:03 Partitioning Strategies and Recommendations
    31:39 Ensuring Determinism in Stored Procedures
    35:02 Handling Complex Requirements with Compound Procedures
    37:25 Fault Tolerance and Data Replication Strategies
    40:44 Exploring Use Cases for VoltActiveData
    43:30 The Future of Streaming and VoltActiveData's Role
    47:05 Closing Remarks and How to Learn More
    Volt Active Data: https://www.voltactivedata.com/use-cases/activesd-streaming-data/
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #sql #streamprocessing #java #acid

    49 min
  • TigerBeetle: World’s Fastest Financial Transactions Database

    In an enlightening episode of the GeekNarrator Podcast, host Kaivalya Apte and TigerBeetle's CEO, Joran, delve deep into the world of online transaction processing (OLTP). They discuss the origin, unique architecture, and innovative methodologies behind TigerBeetle, a database tailored to efficiently handle high-volume transaction systems. The podcast explores the system's key features such as efficient scalability, performance-oriented design, and optimized memory usage, demonstrating its robustness in handling business transactions and accounting. It also elucidates TigerBeetle’s adaptability to various domains beyond finance, like energy management and gaming, while highlighting the rigorous testing it undergoes for impeccable quality assurance.

    Chapters:
    00:00 Introduction
    01:19 Joran's Journey into Databases
    03:59 Understanding Financial Transaction Databases
    07:41 The Evolution of OLTP and OLAP
    16:13 The Need for a New Database: TigerBeetle
    16:53 Performance and Safety Features of TigerBeetle
    28:49 The Importance of Safety in Financial Transactions
    36:49 Changing Developer Experience with TigerBeetle
    41:43 Understanding the CPU and Memory Bandwidth
    42:12 The Importance of Data Format Language
    43:27 The Concept of Serialization and its Impact
    46:23 The Architecture of TigerBeetle
    46:29 The Role of Replicated State Machine
    48:18 The Importance of Consensus in Replication
    50:20 The Structure of TigerBeetle
    50:37 The Importance of Log in Systems
    50:51 Understanding the State in Replicated State Machine
    52:55 The Role of LSM in TigerBeetle
    53:55 The Impact of Compaction Process on Performance
    57:06 The Importance of Predictability in Software
    01:06:15 The Read and Write Path in TigerBeetle
    01:14:46 Potential Use Cases for TigerBeetle
    01:17:09 Understanding the Limitations of TigerBeetle
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    #tigerbeetledb #databases #acid #olap #oltp #postgres #mysql

    1 hr 20 min
  • Clean Code Adventures with Uncle Bob

    In this episode, we dive deep into the world of clean coding with none other than the master and pioneer of the field, Uncle Bob. We explore the nuances and the art behind writing effective and efficient scripts. This conversation covers the nitty-gritty of writing and editing scripts, from understanding how to break down large functions, to discussing principles like 'Single Responsibility Principle', 'Dependency Inversion Principle' and how to balance the 'DRY' (Don't Repeat Yourself) principles. Uncle Bob also shares valuable insights on testing, handling errors, naming conventions and how to work with different types of duplication in coding. He shares recommended resources and books that every coder should read.

    Chapters:
    00:00 Introduction and Welcome
    00:06 The Importance of Code Quality
    00:29 Introducing Robert Martin (Uncle Bob)
    01:39 Uncle Bob's Journey in Programming
    02:34 Discussion on Functional Design and New Book
    03:52 The Evolution of Software Development
    04:28 Revisiting the Clean Code Book
    04:49 The Impact of Hardware Changes on Software
    06:13 The Evolution of Programming Languages
    07:33 The Importance of Code Structure and Organization
    09:07 The Impact of Microservices and Open Source
    11:14 The Role of Modular Programming
    22:07 The Importance of Naming in Code
    26:31 The Role of Functions in Code
    34:12 The Role of Switch Statements in Code
    42:36 The Importance of Immutability
    51:00 Dealing with Complex Steps in Programming
    51:21 Implementing State Machines in Programming
    51:46 The Pragmatic Approach to Programming
    53:01 Understanding Error Handling in Programming
    54:08 The Challenge of Exception Handling
    57:27 The Importance of Log Messages in Debugging
    01:03:05 The Dilemma of Code Duplication
    01:05:51 The Intricacies of Error Handling
    01:07:40 The Role of Abstraction in Programming
    01:13:55 The Importance of Testing in Programming
    01:19:43 The Challenges of Mocking in Testing
    01:25:11 The Essence of Programming: Discipline, Ethics, and Standards
    Book Recommendations:
    Tidy First: https://www.oreilly.com/library/view/tidy-first/9781098151232/
    Design Patterns: https://www.amazon.de/-/en/Erich-Gamma/dp/0201633612
    Analysis Pattern: https://martinfowler.com/books/ap.html
    Structured Analysis and System Specification: https://www.amazon.de/-/en/Tom-Demarco/dp/0138543801
    Fundamental Algorithms: https://www.amazon.com/Art-Computer-Programming-Vol-Fundamental/dp/0201896834
    Sorting and Searching: https://www.amazon.de/-/en/Donald-Knuth/dp/0201896850
    Structure and Interpretation of Computer Programs: https://web.mit.edu/6.001/6.037/sicp.pdf
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!

    1 hr 35 min
  • Durable async/await with Dominik Tornow

    In this episode of the Geek Narrator Podcast, Kaivalya Apte engages in an invigorating discussion with Dominik Tornow, the founder and CEO of Resonate. They explore Durable Async-Await, an interesting concept in distributed systems world, along with other nuances of distributed programming. Dominik also talks about the development and execution of Resonate to simplify distributed systems with a focus on observability, usability, and it's future direction. The conversation concludes with a discussion of different concurrency models and the future of distributed systems

    Chapters:
    00:00 Introduction and Guest Background
    02:44 Understanding Async Await
    10:25 Challenges with Current Async Await Model
    12:53 Introducing Resonate: A Solution for Distributed Async Await
    13:34 Practical Application: E-commerce Example
    24:57 Understanding the Role of the Platform in Distributed Systems
    30:12 Dealing with Partial Failures in Distributed Systems
    39:44 Getting Started with Resonate
    40:40 Introduction to Resonate and its Simplicity
    41:09 Getting Started with Resonate: Installation and Setup
    42:22 Understanding the Durability Aspect of Resonate
    42:49 Exploring the Resonate Durable Promise Server
    44:10 Scaling Up: Introducing Workers into the System
    48:35 The Importance of Open Standards in Resonate
    50:17 Exploring the Integration Capabilities of Durable Promises
    01:04:31 Understanding the Role of Timeouts in Durable Promises
    01:07:29 The Future of Resonate: Challenges and Upcoming Features
    01:13:04 Understanding the Limitations of Durable Promises
    01:14:51 Wrapping Up: Final Thoughts on Resonate and Durable Promises
    References:
    A note on Distributed Systems: https://scholar.harvard.edu/files/waldo/files/waldo-94.pdf
    Thinking in Distributed Systems: https://dtornow.gumroad.com/l/distributed-systems
    McCarthy's paper: https://www-formal.stanford.edu/jmc/recursive/recursive.html
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!

    1 hr 16 min
  • Observability Engineering with Liz Fong-Jones

    Join host Kaivalya Apte in this episode of The Geek Narrator Podcast as he discusses observability engineering with field CTO at Honeycomb, Liz Fong-Jones. They delve into the importance of observability for software engineers, the role of Honeycomb in popularizing this concept, and how observability has evolved over the years. Liz shares her experiences transitioning from being an SRE at Google to advocating for observability at Honeycomb and walking the journey from developer advocate to Field CTO. They discuss the definitions and misconceptions surrounding observability and elucidate on Service-Level Objectives (SLOs) & indicators (SLIs) and challenges they solve. Tune in for an informative and in-depth conversation on observability engineering.

    Chapters:
    00:00 Introduction
    00:08 Understanding Observability Engineering
    00:37 Guest Introduction: Liz Fong Jones
    00:53 Liz's Journey to Field CTO at Honeycomb
    27:38 Understanding Site Reliability Workbook Materials
    27:57 Identifying Critical User Journeys
    29:49 Different Types of Services and Their SLOs
    33:05 Setting Up SLOs: Granularity and Number
    42:42 Understanding Service Level Indicators (SLIs)
    50:26 Common Mistakes in Setting Up SLOs
    52:09 Cultivating an Observability-Driven Development Culture
    References:
    Observability Engineering: https://www.oreilly.com/library/view/observability-engineering/9781492076438/
     @Google  SRE book: https://sre.google/books/
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!

    55 min
  • Messaging and Streaming with Apache Pulsar - with Matteo Merli

    In this video I talk about Apache Pulsar with Matteo Merli, CTO at StreamNative. This episode will provide you good insight about how Apache Pulsar works and more importantly differs with the most popular Pub/Sub and streaming platform Apache Kafka. Things like, what enables possibility of 1 million topics? Why is rebalancing not required? How does decoupled storage and compute architecture works? How it uses the concept of Subscriptions to avoid retaining data unnecessarily?

    And much more...
    Chapters:
    00:00 Introduction and Guest Introduction
    00:08 Understanding Apache Pulsar and its Origin
    01:22 The Problem Apache Pulsar was Designed to Solve
    02:35 The Evolution of Apache Pulsar
    05:15 Understanding Basic Concepts of Apache Pulsar
    09:27 Deep Dive into Apache Pulsar's Architecture
    21:16 Understanding the Flow of Data in Apache Pulsar
    28:54 Understanding Subscriptions in Apache Pulsar
    31:57 Understanding End-to-End Latency and Subscription Creation
    32:32 Broker's Role and Handling Metadata
    33:05 Memory Management and Consumer Handling
    34:07 Message Processing and Flow Control
    34:32 Message Storage and Retrieval
    36:00 Comparing Pulsar with Kafka
    43:52 Understanding Multi-Tenancy in Pulsar
    49:17 Exploring Tiered Storage and Future Developments
    Important links:
    StreamNative: https://streamnative.io/
    Apache Pulsar: https://pulsar.apache.org/
    Matteo Merli: https://twitter.com/merlimat
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!

    1 hr 4 min
  • VictoriaMetrics internals - Making monitoring simple and reliable at massive scale

    Deep Dive into Victoria Metrics with Alex and Roman

    Join the insightful discussion with Vitoriametrics creators, Alex and Roman, in the Geekneritor podcast hosted by Kaivalya Apte. This episode explores the internals of Victoria Metrics - a highly scalable monitoring solution and time series database. Discover the origins of Victoria Metrics, understand how it evolved, and learn about its unique architecture and functionality. From the concept of time series, the usage of consistent hashing in data distribution to real-world applications, it's all packed into this engaging conversation.
    00:00 Introduction
    01:52 The Genesis of VictoriaMetrics
    02:18 The Journey from Postgres to Clickhouse
    03:19 The Transition from Prometheus to Victoria Metrics
    05:08 The Birth and Evolution of Victoria Metrics
    13:01 The Architecture of Victoria Metrics
    20:10 Data Ingestion and Integration in Victoria Metrics
    29:15 Understanding the Vector Metric Architecture
    30:30 Comparing Shared Storage and Object Store
    31:00 Designing the VictoriaMetrics Architecture
    32:01 The Role of Object Storage
    36:15 The Importance of Indexing
    43:19 Understanding the Ingestion Process
    45:46 Exploring the Select Process
    55:55 Future Plans for Victoria Metrics
    Important Links:
    1. Architecture Overview: https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#architecture-overview
    2. How ClickHouse Inspired Us to Build a High Performance Time Series Database
    https://altinity.com/wp-content/uploads/2021/11/How-ClickHouse-Inspired-Us-to-Build-a-High-Performance-Time-Series-Database.pdf
    3. Frequently asked questions.
    https://docs.victoriametrics.com/FAQ.html
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!

    1 hr 3 min
  • TiDB Internals with Li Shen

    Join us on a deep dive into the intricacies of TiDB with Li Shen from PingCap. In this episode, Li Shen provides a comprehensive exploration of TiDB, its unique features, and how it tackles scalability and reliability issues commonly associated with MySQL.

    If you're dealing with struggles in your MySQL cluster and seeking a more dependable and scalable system, TiDB might be the solution for you. This conversation touches on various aspects of this cutting-edge database, its operational mechanism, use case scenarios, and how it's optimized for different workloads.
    Key topics include: the architecture of TiDB, the journey of data from API to storage node, embracing analytical use cases, the importance of database reliability, and the process of migrating to TiDB. Dive in now!
    00:00 Introduction and Welcome
    02:47 Defining TIDB: A Disputed SQL Database
    04:55 The Role of MySQL Compatibility in TIDB
    05:54 Primary Use Cases for TIDB
    09:38 Understanding the Data Ingestion Process in TIDB
    16:52 Understanding Indexing in TIDB
    23:01 Pushing Down Table Scans and Partial Aggregation
    24:39 Introduction to Columnary Extension: Flash
    24:54 Understanding Data Replication and Learner Nodes
    26:23 Ensuring Strong Consistency in Data
    27:12 Balancing Transactional and Analytical Use Cases
    27:57 Understanding Data Replication and Consistency Model
    28:42 Exploring Ty Flash Storage Layer
    28:54 Understanding High Concurrency Insert and Update
    32:09 Exploring the Read Path and Caching Mechanism
    37:50 Understanding the Importance of High Reliability
    43:01 Exploring Migration from Other Databases
    48:01 Comparing TiDB with Other Distributed SQL Databases
    52:21 Identifying Use Cases Where TiDB Might Not Be the Best Choice
    Stay Curios! Keep Learning!

    55 min
  • AI Powered Database optimisation with Andy Pavlo, Ottertune

    In this video I discuss Database tuning and Optimisation with Andy Pavlo, OtterTune.

    Andy is an Associate Professor with Indefinite Tenure of Databaseology in the Computer Science Department at Carnegie Mellon University. My research interest is in database management systems, specifically main memory systems, self-driving / autonomous architectures, transaction processing systems, and large-scale data analytics.
    00:00 Introduction and Welcome
    01:31 Understanding Database Optimization
    05:48 Understanding When Database Tuning is Needed
    08:45 Understanding Database Optimization Difficulties
    16:16 Understanding Default Settings in Databases
    22:35 Role of Machine Learning in Database Tuning
    22:38 Introduction to Ottertune
    28:36 Data Collection for Machine Learning Model
    35:25 Deployment and Data Collection Process
    38:03 Admitting the Limitations of Current Model
    38:53 Challenges in Predicting Performance Improvements
    39:28 The Importance of Data Collection Over Time
    39:52 Avoiding Weekend and Holiday Tuning
    40:05 Introducing New Features for Database Comparison
    42:09 Provisioning Recommendations and Performance Predictions
    43:03 The Importance of Telemetry in Understanding Database Performance
    44:01 Handling Dramatic Changes in Database Workloads
    44:48 Preparing for Predictable Traffic Spikes
    48:13 The Importance of Testing in Database Optimization
    53:33 The Future of Database Optimization
    55:50 Common Mistakes in Database Management
    01:09:15 The Future of Holistic Database Tuning
    Links:
    Ottertune: https://ottertune.com/
    Andy Pavlo: https://www.cs.cmu.edu/~pavlo/
    CMU youtube: https://www.youtube.com/@UCHnBsf2rH-K7pn09rb3qvkA
    Resources:
    CMU: https://15799.courses.cs.cmu.edu/spring2022/schedule.html
    Ottertune blog: https://ottertune.com/blog
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Stay Curios! Keep Learning!

    1 hr 15 min
  • Duckdb Internals with Mark Raasveldt

    Deep Dive into DuckDB with CTO Mark Raasveldt

    Decode the insights of databases with Geek Narrator podcast. In this episode, host Kaivalya Apte converses with Mark Raasveldt, the CTO of DuckDB labs, discussing his journey from being a database enthusiast to creating DuckDB. They delve into how DuckDB, an analytical database, differs from other databases, the design decisions, its internal mechanisms, and much more. The episode also highlights the advantages of DuckDB in analytics, the motivation behind its ACID compliance, and how DuckDB handles ingestion, transaction isolation, mutations, and queries. Join in to learn how your data workloads can benefit from DuckDB.
    00:00 Introduction and Guest Introduction
    00:44 Guest's Journey into Databases
    03:40 The Birth of DuckDB
    04:30 Challenges with Existing Databases
    05:15 Technical Difficulties
    05:16 Why Existing Databases Fall Short for Data Scientists
    09:16 The Role of SQLite and Its Limitations
    13:59 Defining DuckDB
    16:48 Comparing DuckDB with Other Analytical Databases
    19:50 Deployment Models for DuckDB
    22:47 Data ingestion into DuckDB
    22:51 Data Ingestion in DuckDB
    30:24 How DuckDB Handles Updates and Mutations
    35:35 Understanding Column Granularity and Rewrites
    35:58 Implications of Compression on Data Updates
    36:38 Trade-offs in Row Group Size
    37:32 Benefits of Column Storage Model
    38:15 Row Groups and Parallelism
    39:02 Choosing Row Group Size: An Experimental Approach
    40:00 Handling Data Type Changes in Columns
    41:00 Internal Data Structures in DuckDB
    42:21 Reading Data: Point Lookups, Aggregations, and Joins
    47:22 Optimization for Full Table Scans
    53:49 Understanding ACID Compliance in DuckDB
    55:49 Multi-Version Concurrency Control (MVCC) in DuckDB
    59:50 Use Cases and Applications of DuckDB
    01:01:42 The Story Behind DuckDB's Name
    01:02:34 Future Vision for DuckDB
    References:
    DuckDB: https://duckdb.org/
    Mark's blog: https://mytherin.github.io/
    ===============================================================================
    For discount on the below courses:
    Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003
    Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
    ===============================================================================
    Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator
    If you like this episode, please hit the like button and share it with your network.
    Also please subscribe if you haven't yet.
    Database internals series: https://youtu.be/yV_Zp0Mi3xs
    Popular playlists:
    Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
    Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
    Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
    Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
    Stay Curios! Keep Learning!
    Cheers,
    The GeekNarrator

    1 hr 5 min

About The GeekNarrator

From the publisher's feed

The GeekNarrator podcast is a show hosted by Kaivalya Apte who is a Software Engineer and loves to talk about Technology, Technical Interviews, Self Improvement, Best Practices and Hustle.