Confluent Developer ft. Tim Berglund, Adi Polak & Viktor Gamov

Serverless Stream Processing with Apache Kafka ft. Bill Bejeck


Listen Later

What is serverless?

Having worked as a software engineer for over 15 years and as a regular contributor to Kafka Streams, Bill Bejeck (Integration Architect, Confluent) is an Apache Kafka® committer and author of “Kafka Streams in Action.” In today’s episode, he explains what serverless and the architectural concepts behind it are. 

To clarify, serverless doesn’t mean you can run an application without a server—there are still servers in the architecture, but they are abstracted away from your application development. In other words, you can focus on building and running applications and services without any concerns over infrastructure management. 

Using a cloud provider such as Amazon Web Services (AWS) enables you to allocate machine resources on demand while handling provisioning, maintenance, and scaling of the server infrastructure. 

There are a few important terms to know when implementing serverless functions with event stream processors: 

  • Functions as a service (FaaS)
  • Stateless stream processing
  • Stateful stream processing

Serverless commonly falls into the FaaS cloud computing service category—for example, AWS Lambda is the classic definition of a FaaS offering. You have a greater degree of control to run a discrete chunk of code in response to certain events, and it lets you write code to solve a specific issue or use case. 

Stateless processing is simpler in comparison to stateful processing, which is more complex as it involves keeping the state of an event stream and needs a key-value store. ksqlDB allows you to perform both stateless and stateful processing, but its strength lies in stateful processing to answer complex questions while AWS Lambda is better suited for stateless processing tasks. 

By integrating ksqlDB with AWS Lambda together, they deliver serverless event streaming and analytics at scale.

EPISODE LINKS

  • What is Serverless?
  • Serverless Stream Processing with Apache Kafka, AWS Lambda, and ksqlDB
  • Stateful Serverless Architectures with ksqlDB and AWS Lambda 
  • Serverless GitHub repository
  • Kafka Streams in Action
  • Watch the video version of this podcast
  • Join the Confluent Community
  • Learn more with Kafka tutorials, resources, and guides at Confluent Developer
  • Live demo: Intro to Event-Driven Microservices with Confluent

SEASON 2
Hosted by Tim Berglund, Adi Polak and Viktor Gamov
Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed
Music by Coastal Kites
Artwork by Phil Vo

  • 🎧 Subscribe to Confluent Developer wherever you listen to podcasts.
  • ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.
  • 👍 If you enjoyed this, please leave us a rating.
  • 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.
...more
View all episodesView all episodes
Download on the App Store

Confluent Developer ft. Tim Berglund, Adi Polak & Viktor GamovBy Confluent

  • 4.8
  • 4.8
  • 4.8
  • 4.8
  • 4.8

4.8

43 ratings


More shows like Confluent Developer ft. Tim Berglund, Adi Polak & Viktor Gamov

View all
Software Engineering Radio by se-radio@computer.org

Software Engineering Radio

273 Listeners

Economist Podcasts by The Economist

Economist Podcasts

4,194 Listeners

Motley Fool Money by The Motley Fool

Motley Fool Money

3,219 Listeners

a16z Podcast by Andreessen Horowitz

a16z Podcast

1,080 Listeners

Thoughtworks Technology Podcast by Thoughtworks

Thoughtworks Technology Podcast

42 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

626 Listeners

The Official SaaStr Podcast: SaaS | Founders | Investors by SaaStr

The Official SaaStr Podcast: SaaS | Founders | Investors

173 Listeners

AWS Podcast by Amazon Web Services

AWS Podcast

205 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

143 Listeners

The Diary Of A CEO with Steven Bartlett by DOAC

The Diary Of A CEO with Steven Bartlett

8,232 Listeners

The Journal. by The Wall Street Journal & Spotify Studios

The Journal.

5,980 Listeners

Waveform: The MKBHD Podcast by Vox Media Podcast Network

Waveform: The MKBHD Podcast

5,967 Listeners

Morning Brew Daily by Morning Brew

Morning Brew Daily

2,987 Listeners

Grit by Kleiner Perkins

Grit

189 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

547 Listeners