We've all been affected by COVID-19 in one way or another, resulting in big changes in workplace functionality, productivity, and even our relationships within the Apache Kafka® and Confluent communities as meetings and events have needed to turn virtual. 

Ale Murray (Global Community Manager, Confluent) shares interesting trends, changes in community metrics, and what we’ve done to adapt as a response. Ale also explains what makes a comprehensive community program and the value of community meetups in light of the pandemic. 

Despite how much we miss in-person interactions, by digitizing events and focusing on the community, we saw great growth in attendance and engagement across our Slack community, online hackathons, MVP program, and online meetups over the last couple of months, proving that nothing can stop this amazing community from thriving.

EPISODE LINKS

• Get involved with the Confluent Community
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer
• Use 60PDCAST to get an additional $60 of free Confluent Cloud usage*

Author Sam Newman catches up with Tim Berglund (Senior Director of Developer Advocacy, Confluent) in the virtual studio on what microservices are, how they work, the drawbacks of microservices, what splitting the monolith looks like, and patterns to look for. 

The pair talk through Sam's book “Monolith to Microservices” chapter by chapter, looking at key components of microservices in more detail. Sam also walks through database decomposition, integrating with new technology, and performing joins in event streaming architecture. Lastly, Sam shares what he’s excited for in the future, which includes “Monolith to Microservices Volume II.”

EPISODE LINKS

• Monolith to Microservices
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer
• Use 60PDCAST to get an additional $60 of free Confluent Cloud usage* 

Tim Baldridge (Senior Software Engineer, Cisco) joins us on Streaming Audio to talk about event streaming, stream processing use cases, and µKanren. 

First, Tim shares about his work at Cisco related to intaking viruses, the backend, and finding new ways to process data. Later, Tim talks about interesting bank and airline use cases, as well as his time at Walmart, taking a closer look at specific retail use cases and the product that Walmart used to process data streams. 

If you’re curious about what µKanren is, how it relates to relational programming, the complex math that goes into the workflow of µKanren, and how Apache Kafka® holds up to all other event streaming platforms, Tim also dives into that too. 

EPISODE LINKS

• µKanren: A Minimal Functional Core for Relational Programming
• It's Actors All The Way Down  
• Den of Clojure Build Your Own Logic Engine
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer

Confluent Platform 5.5 introduces long-awaited JSON Schema and Protobuf support in Confluent Schema Registry and across other platform components. 

Support for Protobuf and JSON Schema in Schema Registry provides the same assurances of data compatibility and consistency we already had with Avro, while opening up Kafka to more businesses, applications, and use cases that are built upon those data serialization formats. 

Tushar Thole (Engineering Leader, Confluent) and David Araujo (Product Manager, Confluent) share about these new improvements to Confluent Schema Registry, the differences between Apache Avro™, Protobuf, and JSON Schemas, how to treat optional fields, some of the arguments between Avro and Protobuf, and why it took some time for Schema Registry to support JSON Schemas and Protobuf.

Later, they talk about custom plugins, adding another layer of safety in Confluent Platform 5.5, and their vision for data governance.

EPISODE LINKS

• Introducing Confluent Platform 5.5
• Confluent Platform Now Supports Protobuf, JSON Schema, and Custom Formats
• Download Confluent Platform
• Getting Started with Protobuf in Confluent Cloud
• Read articles by Robert Yokota 
• Schema Validation with Confluent Platform 5.4 
• Playing Chess with Confluent Schema Registry
• JSON Schema specs
• Send feedback to [email protected]
• Fully managed Apache Kafka as a service! Try free.
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer

Apache Kafka® is a powerful toolset for microservice architectures. In this podcast, we’ll cover how Boden, an online retail company that specializes in high-end fashion linked to the royal family, used streaming microservices to modernize their business. 

Matt Simpson (Solutions Architect, Boden) shares a real life use case showing how Kafka has helped Boden digitize their business, transitioning from catalogs to online sales, tracking stock, and identifying buying patterns. Matt also shares about what he's learned through using Kafka as well as the challenges of being a product master. And lastly, what is Matt excited for for the future of Boden? Find out in this episode!

EPISODE LINKS

• Digital Transformation in Style: How Boden Innovates Retail Using Apache Kafka
• Learn about Boden
• ETL and Event Streaming Explained ft. Stewart Bryson
• Connecting Snowflake and Apache Kafka ft. Isaac Kunen
• Instagram for Kensington Palace
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer

Isaac Kunen (Senior Product Manager, Snowflake) and Tim Berglund (Senior Director of Developer Advocacy, Confluent) practice social distancing by meeting up in the virtual studio to discuss all things Apache Kafka® and Kafka Connect at Snowflake. 

Isaac shares what Snowflake is, what it accomplishes, and his experience with developing connectors. The pair discuss the Snowflake Kafka Connector and some of the unique challenges and adaptations it has had to undergo, as well as the interesting history behind the connector. 

In addition, Isaac talks about how they’re taking on event streaming at Snowflake by implementing the Kafka connector and what he hopes to see in the future with Kafka releases. 

EPISODE LINKS

• Download the Snowflake Kafka Connector
• Paving a Data Highway with Kafka Connect ft. Liz Bennett
• Making Apache Kafka Connectors for the Cloud ft. Magesh Nandakumar
• Machine Learning with Kafka Streams, Kafka Connect, and ksqlDB ft. Kai Waehner
• Connecting to Apache Kafka with Neo4j
• Contributing to Open Source with the Kafka Connect MongoDB Sink ft. Hans-Peter Grahsl
• Connecting Apache Cassandra to Apache Kafka with Jeff Carpenter from DataStax
• Why Kafka Connect? ft. Robin Moffatt
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer

Happy 100th episode of Streaming Audio! Thank you to everyone who has listened, subscribed, left a review, and mostly, for sharing our passion for event streaming. We can't wait for the next 100! To celebrate, Ben Stopford (Senior Director of the Office of the CTO, Confluent) hosts an AMA (ask me anything) with Tim, covering 62 questions in total—from his career, his time at Confluent, Marvel vs. DC, and what he looks for in a new hire, to how to nail your next conference talk. 

We hope you enjoy this special 100th episode of Streaming Audio: a podcast about Apache Kafka®, Confluent, and the cloud.

EPISODE LINKS

• The Song of the Strange Ascetic
• Avoiding Lock-In
• Blogs by Ben Stopford 
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer

Kelsey Hightower was already an advocate, just like all other developers, long before joining Google officially as a developer advocate and Kubernetes expert. Gaining trust in your product, process, and the way you develop code requires the ability to explain those things well. 

Kelsey reflects on the journey that brought him to where he is today and how Kubernetes has evolved over the years too, including what makes Kubernetes so successful. But Tim is not the only one with questions. Kelsey asks a few of his own: does Apache Kafka® want to be a database? Does Kafka want to be a system of record? Is there overlap between Kubernetes and Kafka? Can you run Kafka on Kubernetes?

EPISODE LINKS

• Kubernetes the Hard Way
• Join the Confluent Community Slack
• Learn about Kafka at Confluent Developer

If you’ve ever wondered what Apache Kafka® is, what it’s used for, or wanted to learn about Kafka architecture and all its components, buckle up! In today’s episode, Michael Noll (Principal Technologist, Confluent) and Tim Berglund (Senior Director of Developer Advocacy, Confluent) discuss a series of fundamental questions: What is Kafka? What is an event? How do we organize and store events? And what is Kafka Streams? 

Over the course of this episode, Michael covers an in-depth look into Kafka technology and core concepts: the process of reading from a topic, differences between tables and streams, mutability, and what ksqlDB is and what its event streaming database features accomplish. If you've ever wanted to get a better grasp on how Kafka works, this episode is for you!

EPISODE LINKS

• Streams and Tables in Apache Kafka: A Primer
• Streams and Tables in Apache Kafka: Topics, Partitions, and Storage Fundamentals
• Streams and Tables in Apache Kafka: Processing Fundamentals with Kafka Streams and ksqlDB
• Streams and Tables in Apache Kafka: Elasticity, Fault Tolerance, and other Advanced Concepts
• Browse the Confluent Hub
• Join the Confluent Community Slack
• Learn more with Kafka tutorials, resources, and guides at Confluent Developer

There are two primary industries within the Internet of Things (IoT): industrial IoT (IIoT) and consumer IoT (CIoT), both of which can benefit from the Apache Kafka® ecosystem, including Kafka Streams and Kafka Connect. Kai Waehner, who works in the advanced tech group at Confluent with customers, defining their needs, use cases, and architecture, shares example use cases where he’s seen IoT integration in action. He specifically focuses on Walmart and its real-time customer integration using the Walmart app. Kafka Streams helps fine-tune the Walmart app, optimizing the user experience, offering a seamless omni-channel experience, and contributing to business success. 

Other topics discussed in today’s episode include integration from various legacy and modern IoT data sources, latency sensitivity, machine learning for quality control and predictive maintenance, and when event streaming can be more useful than traditional databases or data lakes.

EPISODE LINKS

• Apache Kafka 2.5 – Overview of Latest Features, Updates, and KIPs
• Machine Learning with Kafka Streams, Kafka Connect, and ksqlDB ft. Kai Waehner
• Blog posts by Kai Waehner
• Processing IoT Data from End to End with MQTT and Apache Kafka®
• End-to-End Integration: IoT Edge to Confluent Cloud
• Apache Kafka is the New Black at the Edge in Industrial IoT, Logistics, and Retailing
• Apache Kafka, KSQL, and Apache PLC4X for IIoT Data Integration and Processing
• Streaming Machine Learning at Scale from 100,000 IoT Devices with HiveMQ, Apache Kafka, and TensorFlow
• Event-Model Serving: Stream Processing vs. RPC with Kafka and TensorFlow
• Join the Confluent Community Slack
• Learn about Kafka at Confluent Developer