Data on Kubernetes Community

Tecton is building a data platform for machine learning. This talk shares some of the adventures and lessons learned while introducing Kafka into our data pipelines.

Engineering Manager @ Tecton. Previously: Staff SWE @ Google working on Search, Platforms, and Photos.

We are going to be looking at and comparing Airflow (the established) versus Argo Workflows (The new kid on the block) and see how they measure up. What you would use each for, why you would want to use one or the other and who would win in a battle for data workflow management supremacy.

DataOps and Data Engineering enthousiast. I get enthousiastic about data and data technologies and mostly about how we can move all that data around and have people smarter than me do smart things with it. Made the switch unknowingly into a Data engineering role some 2 years ago and haven't looked back since Also loves making taco's, anything food related basically and playing the guitar again.

What is Airflow What is Argo Workflows What are their pro's and con's Why you would pick either one

Kya hota hai Kubernetes? Shuruwat kahan se kare? Community ka hissa kaise bane? Kya aap ke mann mein bhi ye sawaal aate hain? Join kariye hume iss meetup mein jahan hum baat karengey har cheez Data on K8s ke baare mein (Hindi mein)! May 3rd ko hum charcha karengey ki kaise aap community ka hissa ban sakte hain, CNCF kya hai, ek SRE ka kaam kya hota hai, and bahot kuch! But yehi nahi! Bhaag lijiye meetup ke end ki quiz mein jisse aap jeet sakte hain kuch special SWAGS DoK ki taraf se!

India se ek student jo hai Open Source and Communities ke liye utsahi! Kunal ne DevOps, and aur kaafi fields mein kaam kiya hai. Ye students ko apne YouTube channel ke dwara padhate and Open Source ke liye sahi rah bhi dikhate hain.

In the talk, we shall dive deep into the latest open-source tools like Prometheus and Jaeger and our journey in using them and ultimately building our own open-source observability tool, SigNoz. We shall discuss: - What is Observability? The 3 pillars of Observability - Metrics, Traces, and Logs - How is monitoring different than observability? - The hard things about Prometheus? - Why Distributed Tracing became so important? - Running both Prometheus and Jaeger to get metrics + traces. How complex can it go? - Pros and cons of using SaaS vs OSS solutions. Why self-host in the 21st century? - Why we built SigNoz? - What is OpenTelemetry? How to instrument a sample app using OpenTelemetry? - Demo of SigNoz to get detailed insights into your applications

Ankit is co-founder and maintainer at SigNoz which is a part of the Y Combinator W21 batch. Ankit loves to discuss new technologies and the problems they solve. He believes technology will be more profoundly used in the future to scale business needs. He is now passionate about the application performance monitoring space and loves talking to developers about how they do it today. When not working he loves to play badminton and go on adventurous trips. He has done many Himalayan trips earlier and is a philanthropist.

- Hard things about Prometheus - How is monitoring different than Observability - How to get started with OpenTelemetry

Within its 7 years of existence, Kubernetes has been the gravitational center of the Cloud Native landscape, elevating a pluggable system that contributed to the diversification of the entire ecosystem. Wider adaptability of the tool prompted the diversification of the end-user base, and a consistent DX for cluster interaction became essential for Kubernetes. The community channeled herculean efforts towards the enhancement of the developer experience by extending the cluster CLI, building portals, and highly-responsive UIs.

Katie is the Ecosystem Technical Advocate at CNCF or Cloud Native Computing Foundation. Katie's focus within this role is to grow and lead the End User Community while bridging the gap with other ecosystem units, such as project maintainers, TOCs, and SIGs. In the past roles as a cloud platform engineer, Katie has build platforms that gravitate towards cloud-native technologies and principles, with Kubernetes as the focal point. These projects started with the automation of application delivery on OpenStack-based infrastructure, which transitioned into the creation of a centralized, globally distributed platform at Condé Nast and American Express. Additionally, Katie is the creator/instructor of the Cloud Native Fundamentals Udacity course and is on the advisory board for Keptn, a CNCF sandbox project.

This talk will focus on the cluster DX chronicles, showcasing tools that contributed to wider adoption for Kubernetes. An emphasis will be placed on cluster CLI and how it can be extended using kubectl plugins and wrappers. This will be followed by an introduction of widespread cluster state managers, covering mechanisms such as GitOps and cloud native IDEs, such as GitPod.

The talk will cover the various aspects of DataOps, why DataOps is important. It will also talk about some of the client experiences and how DataOps strategy is helping addresses some of the challenges. The talk will also cover the DataOps implementations, tools and technologies.

IBM Distinguished Engineer, Master Inventor with 60+ patents, 23 years in IBM. Architect, Developer, Blogger, Author

Good understanding of DataOps, Why we need DataOps, Some understanding of the Technology advancements in the DataOps Area

Are you a Cloud Architect, DevOps Engineer or SRE who is developing cloud-native applications, managing complex app migration projects or needs infrastructure resiliency? Cloud-native applications present extraordinary performance, scale and compliance challenges in hybrid- and multi-cloud environments that legacy tools simply cannot support. In this session and demo, we’ll take you thru a case study for a large aerospace and defense company who is managing and migrating Kubernetes applications and databases in a multi-cloud environment. You’ll also learn how to handle common cloud-native development challenges like recovering from accidental namespace deletions during test/dv or migrating your application to another cloud for scale and performance testing.

Prashanto Kochavara is Director of Product at Trilio, a leader in cloud-native data protection. Kochavara is responsible for the product strategy, development and roadmap of the company's backup and recovery platform for Kubernetes.

Measuring costs in Kubernetes environments is complex. Applications and their resources needs are often dynamic. Teams share resources without transparent prices attached to workloads while organizations are increasingly running resources on a range of machine types and even cloud providers. Kubecost provides an approach built on open source for ensuring consistent and accurate visibility across all your workloads. This discussion will talk about practical examples for implementing cost monitoring & optimization and managing the data that is generated from these efforts.

Webb Brown is a Co-founder and CEO at Kubecost. He is based in San Francisco and was previously a Product Manager at Google working on infrastructure monitoring.

Apache Spark natively runs on top of Kubernetes (instead of Hadoop YARN) since 2018, but it's only since Spark 3.1 (released in March 2021) that the integration is now officially generally available & production-ready. What is the high-level architecture of Spark on Kubernetes, how does it compare to alternatives, what does the migration look like? These are some of the questions we will answer together. We will first introduce the core concepts, then go through the stories of customers who migrated, and then give you concrete technical tips to help you be successful with Spark (on Kubernetes). If time permits, I may do a risky live demo. This will be a technical talk with very fresh content - I hope you will like it. I plan to make it short enough to make room for Q&A and improvisations based on your request. So let me know if there's something specific you're interested in.

I'm one of the co-founders at Data Mechanics (https://www.datamechanics.co), a Cloud-Native Spark Platform for Data Engineers. We're a YCombinator backed startup. We strive to finally make Apache Spark as developer friendly and cost-effective as it should be.. by automating the infrastructure management side (autoscaling, automated sizing of containers, autotuning of Spark configurations) and building intuitive dashboards to help monitor your data pipelines. Prior to Data Mechanics, I was a software engineer at Databricks, where I led their Spark infrastructure team.

Talk in Portuguese

Capacitando arquitetos, desenvolvedores e SREs em todo o mundo para fornecer aplicativos escaláveis e arquiteturas de sistemas otimizados para os recursos exclusivos da nuvem; E as organizações devem navegar em sua jornada nativa da nuvem. Apaixonado por Arquitetura de Sistemas e Engenharia de Software, me dedico a aprender e compartilhar conhecimentos com foco em Cloud, Java e outras tecnologias emergentes. Fui reconhecido como um embaixador da Cloud Native Computing Foundation, sou apaixonado por ajudar a desenvolver a comunidade como um palestrante ativo sobre projetos CNCF e alavancar a liderança local. Além disso, sou Hyperledger Foundation, uma organização de código aberto de blockchain, Líder de Capítulo e membro do conselho da SouJava, um dos maiores JUGs do mundo.