Data on Kubernetes Community

https://go.dok.community/slack

https://dok.community/ 

From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)

Graph databases are the fastest growing data store in the world. According to Gartner, the application of graph processing and graph DBMSs will grow at 100 percent annually through 2022 to continuously accelerate data preparation and enable more complex and adaptive data science. However, it is often difficult for data and analytics professionals to distinguish between different implementation models, and to fit them to their use case. This panel will speak directly to Kubetnetes users and provide them with the context they need to run stateful workloads.

Topics for discussion would be: 

- graph databases on Kubenetes challenges, lessons learnt, and patterns for deployment

https://go.dok.community/slack

https://dok.community/ 

From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)

With the advent of microservices in Kubernetes, individual developer teams now manage their own data, middleware, and databases. Automated tests and CI/CD pipelines have to be revisited to include these new requirements.

This session will discuss and demonstrate how to use Kustomize and Tekton to provide Kube-Native automated workflows taking into account new parameters such as database operators, StorageClass and PVC. In this talk, Nic is also going to provide some insights on how to optimize Tekton to work with multiple workspaces and overcome some of the affinity limitations.

The demonstration will focus on building a comics cards web application using a flask-based frontend and leveraging postgreSQL as the database. It will cover the automation of multiple lifecycle stages:

- local laptop testing automation on K3S with Kustomize

Nicolas is an experienced hands-on technologist, evangelist, and product owner who has been working in the fields of Cloud-Native technologies, Open Source Software, Virtualization, and Datacenter networking for the past 17 years. Passionate about enabling users and building cool tech solving real-life problems, you'll often see him speaking at global tech conferences and online events, spreading the word, and walking the walk with customers and users.

https://go.dok.community/slack

https://dok.community/ 

From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)

This talk considers the motivations behind yet another storage topology and examines a modern, flexible architecture for delivering high-performance storage under Kubernetes.

Nick Connolly is a pioneer of storage virtualisation and the Chief Scientist at DataCore, where his background in real-time computing and multiprocessing led to the creation of a world-class high-performance storage stack on Windows. He holds patents ranging from highly scalable algorithms through to data protection techniques. Recently he has been working with OpenEBS to bring the power and performance of NVMe to Kubernetes.

https://go.dok.community/slack

https://dok.community/ 

From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)

Many ML pipelines depend on shared filesystems for input, output and intermediate data storage. Standards such as CSI have made it possible for applications in Kubernetes to access a variety of data storage systems. Yet, data scientists still have to deal with low-level details of data access in order to execute their pipelines in Kubernetes. Datashim is a framework that manages the lifecycle of a Dataset object, a CustomResourceDefinition that represents a source of data. Datashim takes care of the details of data access while Kubernetes pods can declaratively access the data by referencing a Dataset in their specifications. This talk will describe Datashim and the Dataset object, discuss its use in ML pipelines, and demonstrate how its pluggable architecture is designed for the development of caching, scheduling and governance plugins. Datashim is an incubating project of the Linux Foundation Data and AI Foundation

Srikumar Venugopal is a Research Scientist in IBM Research Europe in Dublin, Ireland. His research interests lie in the area of cloud computing and large-scale distributed systems, specifically in the topics of middleware, resource management, and scalability. He is the co-founder and current lead for the Datashim project.

https://go.dok.community/slack 

https://dok.community/ 

From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)

Jakub works at Red Hat as Senior Principal Software Engineer. He has long-term experience with messaging and currently focuses mainly on Apache Kafka and its integration with Kubernetes. He is one of the maintainers of the Strimzi project which provides tooling for running Apache Kafka on Kubernetes. Before joining Red Hat he worked as messaging and solution architect in the financial industry.

https://go.dok.community/slack

Jake is a lead developer and software architect at DataStax with over 20 years of experience in the areas of distributed systems, finance, and manufacturing. He is a member of the Apache Foundation and is on the project committee of the Apache Cassandra, Arrow, and Thrift projects. Jake has a reputation for developing creative solutions to solve difficult problems and fostering a culture of trust and innovation. He believes the best software is built by small diverse teams who are encouraged to think freely. Jake received his B.S. in Computer Science from Lehigh University along with a minor in Cognitive Science.

https://go.dok.community/slack

https://go.dok.community/slack

https://dok.community/ 

From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)

Come here about our experience scaling Cassandra on EKS to over 1000 nodes and 20 million transactions per second. This session will cover the lessons learned, successes, failures, and tools used to get there.  Usability is Matt’s mission. He has worked with Federal, Fortune 500, and small businesses to help collect, mine and interact with data. When solving a problem, Mr. Overstreet synthesizes experience from a liberal arts and technical background.  Matt has previously presented community webinars for DataStax and spoken at the search focused Haystack conference.

https://go.dok.community/slack

ABSTRACT OF THE TALK

In this talk I share my personal experience where when I was working for a client the company had a hard time innovating and did not have a lot of faith in their software/infrastructure engineers.

From a 'distance' I saw a lot of potential where they already had knowledgable engineers but lacked the experience to get started as a company but also as individual engineers.

I created a community at that client, filled with their engineers. It was open to everyone who wanted to learn and we saw everyone's input as equal. Because of that safe space the company now runs almost all their IT in the cloud in a way that made both the engineers and the managers happy. All simply by just being a nice guy!

BIO

Hi my name is Fabian Met and I have been working in the cloud native space for the past couple of years. I love to learn new tech and do so almost every day!

When I learn something new I then always use that knowledge to teach people how tech that might seem difficult at first can actually be quite manageable! Making sure that people do not experience these huge hills they sometimes face trying to learn something new is what I love to do!

My role as Techfluencer means that I will help companies and people alike to see the benefit and required investment to start innovating and using new technology. I don't do this as an authoritative figure, instead I love to talk to people and hear their stories and challenges. Listening to them and talking to them usually leads to insight for both parties and possible solutions that previously might not have seen feasible!

KEY TAKE-AWAYS FROM THE TALK

Even though you might not be in a position of power, I will show you that no matter who you are, be that a manager, engineer or even the janitor you can help make positive change in your company!

https://go.dok.community/slack

ABSTRACT OF THE TALK

What if databases only ran in a single place? That would be useless. But it's what we get with most database-as-a-service offerings in the public cloud. Kubernetes has the potential to change the game and let users run complex database anywhere they want. This talk tells the story of how we built a portable ClickHouse cloud service on Kubernetes. It runs on AWS and GCP, of course. Even better, we're working on managing databases in *any* Kubernetes cluster, even those behind firewalls and NAT gateways. We can pull it off because Kubernetes is truly portable. We'll share what we've learned about using Kubernetes portability create a new breed of managed database. We'll speculate about what this portends for the future of database management.

Robert Hodges has worked with over 20 different DBMS types starting with M204 back in 1983. His current favorite is ClickHouse, a popular open source data warehouse. He caught the Kubernetes bug at VMware in 2018 and has been working on it ever since. In his day job he runs Altinity, an enterprise software and services provider for ClickHouse data warehouses.

KEY TAKE-AWAYS FROM THE TALK

Kubernetes is a great data platform even for complex, stateful systems like data warehouses. This talk shows how Kubernetes' portability is not just a convenience. It is a powerful tool that allows users to locate data anywhere it's convenient but still have a SaaS user experience for administrators.