Data on Kubernetes Community

https://go.dok.community/slack

In this panel with Sylvain Kalache, Head of Content at the DoK Community, drives a conversation featuring Nic Vermandé- Principal Developer Advocate at Ondat, Julian Fischer- CEO at anynines, and Sergey Pronin- Group Product Manager at Percona.

https://go.dok.community/slack

When providing data analysis as a service, one must tackle several problems. Data privacy and protection by design are crucial when working on sensitive data. Performance and scalability are fundamental for compute-intensive workloads, e.g. training Deep Neural Networks. User-friendly interfaces and fast prototyping tools are essential to allow domain experts to experiment with new techniques. Portability and reproducibility are necessary to assess the actual value of results.

Kubernetes is the best platform to provide reliable, elastic, and maintainable services. However, Kubernetes alone is not enough to achieve large-scale multi-tenant reproducible data analysis. OOTB support for multi-tenancy is too rough, with only two levels of segregation (i.e. the single namespace or the entire cluster). Offloading computation to off-cluster resources is non-trivial and requires the user's manual configuration. Also, Jupyter Notebooks per se cannot provide much scalability (they execute locally and sequentially) and reproducibility (users can run cells in any order and any number of times).

The Dossier platform allows system administrators to manage multi-tenant distributed Jupyter Notebooks at the cluster level in the Kubernetes way, i.e. through CRDs. Namespaces are aggregated in Tenants, and all security and accountability aspects are managed at that level. Each Notebook spawns into a user-dedicated namespace, subject to all Tenant-level constraints. Users can rely on provisioned resources, either in-cluster worker nodes or external resources like HPC facilities. Plus, they can plug their computing nodes in a BYOD fashion. Notebooks are interpreted as distributed workflows, where each cell is a task that one can offload to a different location in charge of its execution.

BIO  

Iacopo Colonnelli is a Computer Science research fellow. He received his Ph.D. with honours in Modeling and Data Science at Università di Torino with a thesis on novel workflow models for heterogeneous distributed systems, and his master’s degree in Computer Engineering from Politecnico di Torino with a thesis on a high-performance parallel tracking algorithm for the ALICE experiment at CERN. His research focuses on both statistical and computational aspects of data analysis at large scale and on workflow modeling and management in heterogeneous distributed architectures.

Dario is an SWE that turned DevOps, and he's regretting this choice day by day. Besides making memes on Twitter that gain more reactions than technical discussions, leading the development of Open Source projects at CLASTIX, an Open Source-based start-up focusing on Multi-Tenancy in Kubernetes.

KEY TAKE-AWAYS FROM THE TALK

From this talk, people will learn:

All the software presented in the talk is OpenSource, so attendees can directly play with them and include them in their experiments with no additional restrictions.

https://go.dok.community/slack

ABSTRACT OF THE TALK

More and more we see stateful workloads pop up in Kubernetes clusters. These workloads generate data that is unique and is ephemeral. During this talk we will discuss the challenges of stateful workloads and how you can successfully protect

BIO

Working over a decade in IT as a technical expert for Veeam Software. Specializing in backup for the modern hybrid cloud. Passion for scripting and programming. Husband of Lena, father of Lev.

KEY TAKE-AWAYS FROM THE TALK

Data backup of Kubernetes. DR for Kubernetes

https://go.dok.community/slack

ABSTRACT OF THE TALK

BIO

Product and technology leader. Worked in various fields: internet service providers, financial sector and merge & acquisition business. Currently leads product @ Percona focusing on cloud native technologies for open source databases

KEY TAKE-AWAYS FROM THE TALK

Learn how Percona Monitoring and Management (PMM) allows you to enable developers to deploy and manage databases anywhere with private Database-as-a-service capability backed by Operators.

You will get high level overview of Percona's Operators structure and how private DBaaS can boost the productivity of your engineering and IT teams.

https://go.dok.community/slack

ABSTRACT OF THE TALK

The first part of this talk will explain the Apache Kafka ecosystem and how the different components can be used to load data from social networks and use stream processing and machine learning to analyze them.

The second part will show a demo running on Kubernetes which will use Kafka Connect to load data from Twitter and analyze them using the Kafka Streams API.

After this talk, the attendees should be able to better understand the full advantages of the Apache Kafka ecosystem especially with focus on Kafka Connect and Kafka Streams API. And they should be also able to use these components on top of Kubernetes.

BIO

Jakub works at Red Hat as Senior Principal Software Engineer. He has long-term experience with messaging and currently focuses mainly on Apache Kafka and its integration with Kubernetes. He is one of the maintainers of the Strimzi project which provides tooling for running Apache Kafka on Kubernetes. Before joining Red Hat he worked as messaging and solution architect in the financial industry.

KEY TAKE-AWAYS FROM THE TALK

The key takeaway of this talk is that Apache Kafka is more than just a messaging broker. It is a platform and ecosystem of different components which can be used to solve complex tasks when dealing with events or processing data. The talk demonstrates this on loading tweets from Twitter and processing them using the different parts of the Kafka ecosystem. The whole talk and its demos are running on Kubernetes using the Strimzi project. So it also shows how to easily run all the different components on top of Kubernetes with the help of few simple YAML files.

https://go.dok.community/slack

https://dok.community/ 

ABSTRACT OF THE TALK  

The entire app is built in microservices, running on k8s pods and uses k8s-native message broker called memphis  

WORKSHOP

 Learn how to build a cloud-native, distributed, event-driven food delivery application using a message broker, and deploy it to a kubernetes cluster. 

We will also discuss the importance of using a message broker within our app, and what are the benefits it provides to a more resilient application.  

The workshop is based on this article 

- https://medium.com/memphis-dev/how-to-build-your-own-wolt-app-b220d738bb71 

Strongly advise to install before 

- memphis{dev} broker 

- https://github.com/memphisdev/memphis-brokerminikube 

- https://minikube.sigs.k8s.io/docs/start/node.js 

- https://nodejs.org/ 

If any questions arise, please raise a question on our Discord channel 

- https://discord.gg/WZpysvAeTf 

BIO  

Develop since I was 10 years old  

KEY TAKE-AWAYS FROM THE TALK  

Learn the basics of building and deploying cloud-native, distributed, event-driven app

https://go.dok.community/slack 

https://dok.community/  

ABSTRACT OF THE TALK  

In this talk, Maxwell is going to share his thoughts about Service Mesh and database operations, called Database Mesh, and figure out the design of Pisanix.  

Running databases on Kubernetes is hard,  Database Mesh is building a new standard of databases management on cloud. It tries to make database operation more easier, and provides a framework for different kind of databases.  

Such as using a CRD called VirtualDatabase to describe a virtual database, developers will only need to know about this, meanwhile DBAs will have to bind TrafficStrategy and DatabaseEndpoint to VirtualDatabase, let Pisanix running like a MySQL proxy for applications. More features like SecurityClaim and TrafficQoS will be added to simplify the database operation work.  

BIO  

* Focus on SaaS and Database Mesh 

* Previously worked at 株式会社ネットスターズ、ChuanyangTech、AntFinancial、Yeepay 

* Linuxer & Gopher & Rustacean，open source advocator，cloud native envanglist 

* “CaaSOne” Community Cofounder 

* Contributor of multiple cloud native open source projects 

* Participated in writing Analysis and practice of Tensorflow、Istio Handbook  

KEY TAKE-AWAYS FROM THE TALK  

What's the difference of Service Mesh and Database Mesh.  

Why do people need a mesh for databases.  

What does Pisanix can help DBAs daily work.  

The current status of Pisanix.

https://go.dok.community/slack 

https://dok.community/  

ABSTRACT OF THE TALK  

As a community we are committed to making learning how to run stateful workloads on Kubernetes as accessible and inclusive as possible.  Using our DoK Landscape which we launched at Kubecon, we will be doing a guided "tour" of the technologies featured on there. We will go one by one, providing tutorials with concrete instructions for you tp get hands on experience. At the end of each sprint we will share our learnings in public.  

BIO  

CNCF Ambassador. Data on Kubernetes (@Dokcommunity) Community Leader, amateur rapper, tech from the human side.

https://go.dok.community/slack

ABSTRACT OF THE TALK

What about your streaming and analytic workloads? If you are all-in on Kubernetes you can't forget about these important parts of your infrastructure. I'll talk about the current state of the art. Why organizations may hesitate to go beyond deploying databases in Kubernetes and most important, some key things you need to be successful.

BIO

Patrick McFadin is the co-author of the upcoming O’Reilly book “Managing Cloud-Native Data on Kubernetes” He currently works at DataStax in Developer Relations and as a contributor to the Apache Cassandra project. Patrick has worked as Chief Evangelist for Apache Cassandra and as a consultant for DataStax, where he had a great time building some of the largest deployments in production. Previous to DataStax, he held positions as Chief Architect, Engineering Lead and Database DBA/Developer.

KEY TAKE-AWAYS FROM THE TALK

People should walk away with a better understanding of what it takes to deploy streaming and analytic workloads in Kubernetes.