Data Engineering Podcast

Running Your Database On Kubernetes With KubeDB

Listen Later
Summary

Kubernetes is a driving force in the renaissance around deploying and running applications. However, managing the database layer is still a separate concern. The KubeDB project was created as a way of providing a simple mechanism for running your storage system in the same platform as your application. In this episode Tamal Saha explains how the KubeDB project got started, why you might want to run your database with Kubernetes, and how to get started. He also covers some of the challenges of managing stateful services in Kubernetes and how the fast pace of the community has contributed to the evolution of KubeDB. If you are at any stage of a Kubernetes implementation, or just thinking about it, this is definitely worth a listen to get some perspective on how to leverage it for your entire application stack.

Announcements
  • Hello and welcome to the Data Engineering Podcast, the show about modern data management
  • When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. If you need global distribution, they’ve got that covered too with world-wide datacenters including new ones in Toronto and Mumbai. And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • Alluxio is an open source, distributed data orchestration layer that makes it easier to scale your compute and your storage independently. By transparently pulling data from underlying silos, Alluxio unlocks the value of your data and allows for modern computation-intensive workloads to become truly elastic and flexible for the cloud. With Alluxio, companies like Barclays, JD.com, Tencent, and Two Sigma can manage data efficiently, accelerate business analytics, and ease the adoption of any cloud. Go to dataengineeringpodcast.com/alluxio today to learn more and thank them for their support.
  • Understanding how your customers are using your product is critical for businesses of any size. To make it easier for startups to focus on delivering useful features Segment offers a flexible and reliable data infrastructure for your customer analytics and custom events. You only need to maintain one integration to instrument your code and get a future-proof way to send data to over 250 services with the flip of a switch. Not only does it free up your engineers’ time, it lets your business users decide what data they want where. Go to dataengineeringpodcast.com/segmentio today to sign up for their startup plan and get $25,000 in Segment credits and $1 million in free software from marketing and analytics companies like AWS, Google, and Intercom. On top of that you’ll get access to Analytics Academy for the educational resources you need to become an expert in data analytics for measuring product-market fit.
  • You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference. Go to dataengineeringpodcast.com/conferences to learn more and take advantage of our partner discounts when you register.
  • Go to dataengineeringpodcast.com to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat
  • Your host is Tobias Macey and today I’m interviewing Tamal Saha about KubeDB, a project focused on making running production-grade databases easy on Kubernetes
    • Interview
    • Introduction
    • How did you get involved in the area of data management?
    • Can you start by explaining what KubeDB is and how the project got started?
    • What are the main challenges associated with running a stateful system on top of Kubernetes?
      • Why would someone want to run their database on a container platform rather than on a dedicated instance or with a hosted service?
      • Can you describe how KubeDB is implemented and how that has evolved since you first started working on it?
      • Can you talk through how KubeDB simplifies the process of deploying and maintaining databases?
      • What is involved in adding support for a new database?
        • How do the requirements change for systems that are natively clustered?
        • How does KubeDB help with maintenance processes around upgrading existing databases to newer versions?
        • How does the work that you are doing on KubeDB compare to what is available in StorageOS?
          • Are there any other projects that are targeting similar goals?
          • What have you found to be the most interesting/challenging/unexpected aspects of building KubeDB?
          • What do you have planned for the future of the project?
            • Contact Info
            • LinkedIn
            • @tsaha on Twitter
            • Email
            • tamalsaha on GitHub
              • Parting Question
              • From your perspective, what is the biggest gap in the tooling or technology for data management today?
                • Links
                • KubeDB
                • AppsCode
                • Kubernetes
                • Kubernetes CRD (Custom Resource Definition)
                • Kubernetes Operator
                • Kubernetes Stateful Sets
                • PostgreSQL
                  • Podcast Interview
                  • Hashicorp Vault
                  • Redis
                  • Elasticsearch
                    • Podcast Interview
                    • MySQL
                    • Memcached
                    • MongoDB
                    • Docker
                    • Rook Storage Orchestration for Kubernetes
                    • Ceph
                      • Podcast Interview
                      • EBS
                      • StorageOS
                      • GlusterFS
                      • OpenEBS
                      • CloudFoundry
                      • AppsCode Service Broker

                        • The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

                        Support Data Engineering Podcast

                        ...more
                        View all episodesView all episodes
                        Download on the App Store
                        Data Engineering PodcastBy Tobias Macey
                        • 4.6
                        • 4.6
                        • 4.6
                        • 4.6
                        • 4.6

                        4.6

                        135 ratings

                        More shows like Data Engineering Podcast

                        View all
                        Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

                        Software Engineering Radio - the podcast for professional software developers

                        272 Listeners

                        The Changelog: Software Development, Open Source by Changelog Media

                        The Changelog: Software Development, Open Source

                        283 Listeners

                        The Cloudcast by Massive Studios

                        The Cloudcast

                        152 Listeners

                        Thoughtworks Technology Podcast by Thoughtworks

                        Thoughtworks Technology Podcast

                        41 Listeners

                        Data Skeptic by Kyle Polich

                        Data Skeptic

                        482 Listeners

                        Talk Python To Me by Michael Kennedy

                        Talk Python To Me

                        592 Listeners

                        Software Engineering Daily by Software Engineering Daily

                        Software Engineering Daily

                        624 Listeners

                        The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

                        The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

                        443 Listeners

                        Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

                        Super Data Science: ML & AI Podcast with Jon Krohn

                        298 Listeners

                        Python Bytes by Michael Kennedy and Brian Okken

                        Python Bytes

                        213 Listeners

                        DataFramed by DataCamp

                        DataFramed

                        266 Listeners

                        Practical AI by Practical AI LLC

                        Practical AI

                        189 Listeners

                        The Stack Overflow Podcast by The Stack Overflow Podcast

                        The Stack Overflow Podcast

                        64 Listeners

                        The Real Python Podcast by Real Python

                        The Real Python Podcast

                        140 Listeners

                        Latent Space: The AI Engineer Podcast by swyx + Alessio

                        Latent Space: The AI Engineer Podcast

                        77 Listeners