Data on Kubernetes Community

Data on Kubernetes Community

By Data on Kubernetes CommunityTechnology
Download on the App Store

Data on Kubernetes Community episodes

  • Databases on Kubernetes: Why are they important? // With Bhavin Shah, Xing Yang, Gabriele Bartolini & Patrick McFadin (DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


    ABSTRACT

    Kubernetes has crossed the chasm, but what about stateful applications and databases? Join us for this panel discussion and learn more about how organizations are deploying different databases like PostgreSQL and Cassandra on Kubernetes, what are the benefits of running databases on Kubernetes, and how the ecosystem is working towards making these things boring, so you can focus on your applications! We will have an interactive discussion with the hosts and guests of the Kubernetes Bytes podcast, and open it up to the audience to ask questions and learn more about the what, why, and how about Databases on Kubernetes!

    35 min
  • Architecting Your First Event Driven Serverless Streaming Applications on K8 // Timothy Spann (DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


    ABSTRACT

    Once you have built a topic in Apache Pulsar, you will quickly see the need to build event-driven applications. This can require a lot of decisions on what framework to use, where to run it, how to deploy it, and how to manage these applications on Kubernetes cloud natively.

    I will walk you through step-by-step in building Pulsar Functions which is the easy way to design, test, develop, integrate, deploy, monitor, and manage serverless streaming applications in Java and Python.

    Together we will build a full application as an Apache Pulsar function and enjoy the power of running it in the cloud for IoT events and add any routing, transformation, or machine learning that we need to accomplish our business requirements.

    Through FunctionMesh we run on Kubernetes natively.

    In this talk, you will deploy ML functions to transform real-time data on Kubernetes.

    14 min
  • Fybrik - A Kubernetes based platform for governed data use // Flora Gilboa-Solomon, Alexey Roytman, Maryna Strelchuk & Barry Hijkoop (DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


    ABSTRACT

    Data is the foundation for business value. However, in many enterprises, it is spread across different data stores, public/private clouds, and on-premises. The use of data is governed by regulatory requirements and enterprise policies and enterprises face dynamic data residency challenges.
    Fybrik (https://fybrik.io) is an open-source, Kubernetes-based infrastructure that enables easier access to data distributed across hybrid and multi-cloud environments, while enforcing data governance policy according to the business’s needs.

    In this talk, we will share use cases from a pilot project carried out with ING (a multinational banking corporation). Fybrik enabled ING the use of their data across geographical borders, across clusters in a secure, automated, and compliant manner.

    21 min
  • The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


    ABSTRACT

    This talk will go through both the improvements that have been made in Kubernetes for batch analytic workloads as well as some of the current pain experienced by users and developers moving their workloads to Kube. In this talk you will learn about how we “cheated” back in the YARN and Mesos days to make things go fast, why Kubernetes doesn’t like those cheats, and what some alternatives are.

    21 min
  • Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


    ABSTRACT

    Sourcegraph is a code intelligence platform that helps our customers to understand their code better. As we have scaled up, we are starting to run hundreds of instances for our customers in separate kubernetes clusters.

    Running dozens of distinct clusters with a stateful application that scales horizontally and vertically has been challenging. Sourcegraph's inherently stateful nature requires thoughtful planning to achieve resilience while making tradeoffs to keep costs down. This talk will discuss our methodologies to allow our cluster to scale while minimizing drift across multiple customer deployments. I will also cover our use of Terraform, ArgoCD, and Helm to manage the application and infrastructure. Finally, this talk will discuss our disaster recovery strategies that allowed our organization to achieve SOCII compliance while iterating quickly.

    If you are looking to scale out a managed offering at your organization, learn from our mistakes and come to this talk!

    22 min
  • Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  


    Abstract

    We at OpsVerse provide a DevOps tools platform with fully-managed open source-based tools. One of our key offerings is a holistic observability platform. Metrics and logs are straightforward to aggregate, however traces – which are collected using CNCF Jaeger – were left with some holes in advanced insights. 

     In this talk, I will explain why we decided to switch our Jaeger storage backend to ClickHouse, and why we decided to run it on Kubernetes. I will also share tips and tricks to migrate and run ClickHouse on Kubernetes.

    14 min
  • Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  


    Abstract

    We develop systems to digitize the sheet metal industry with the belief that they should cooperate with each other in an open way. We are convinced that the future lies in creating a software ecosystem that interconnects all levels of the company and even manages to communicate with supplier and customer systems, making for more agile management throughout the entire value chain.  

    One of our newer projects is providing a multi-tenant cloud service to our clients. The chosen database for the service we are developing is MongoDB. After testing with docker containers on virtual machines, we decided to finally host the service in Kubernetes because we wanted something to be able to scale at large for our clients being a service that will be consumed by a multitude of our clients, so to the database in order to reduce latency.

    21 min
  • How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022)

    From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  


    Abstract 

    Working with Terabytes of data is a major challenge for organizations both in terms of architecture and cost. In recent years, a new paradigm has emerged in the world of Big Data, that is, implementing the entire architecture for processing massive data from a microservices approach. This architecture proved to be scalable, efficient, with excellent performance, failsafe, portable between different cloud providers and at a lower cost than conventional architectures. In this talk, I will present how my team managed to implement this architecture based on Kubernetes and what were the main gains we had with the innovation.

    16 min
  • Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández

    https://go.dok.community/slack

    https://dok.community

    We are going to speak about CRDs, and discuss considering them as higher level entities that we normally consider them.

    CRDs normally are kind of a byproduct of an operator. But in reality, they can be considered as the user-facing API of the operator surface. And as such, we would like to introduce conversations about even standardizing some of them, or creating "specifications" out of some CRDs.

    Take this for example: there are some CRDs that could be, potentially, common across multiple operators. Think of a CRD to represent an S3 (or any other storage object) bucket. For operators that store data in object stores, each and every one of them are creating their own versions. But shouldn't we have some "standardized" `ObjectStorage` CRD, without a controller, that all could use?

    We'll be joined by John Sanda of DataStax who works on the K8ssandra project, Alvaro Hernandez who is the CEO of Ongres and Postgres superhero, and Eyar Zilberman Co-Founder and Chief Product Officer at Datree.

    59 min

About Data on Kubernetes Community

From the publisher's feed

The Data on Kubernetes Community (DoKC) is where users go to run data on Kubernetes. We facilitate the creation and sharing of best practices to help users advance in their DoK journey. Here you can…

More shows like Data on Kubernetes Community

All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

All-In with Chamath, Jason, Sacks & Friedberg

10,141 Listeners