AWS Bites

Eoin and Luciano are back with some more AWS messaging services material! This time we talk about Kinesis Data Streams and everything there’s to know about them! In this episode of AWS Bites, we cover the difference between data streaming and queue or message buses. How Kinesis can let you process large batches of messages in near real time. Why you might use Kinesis as a queue or pub/sub bus in some cases. A few different ways to use Kinesis with the focus on avoiding too much complexity. We also provide some insights based on our experience with Kinesis Data Streams including real use cases, monitoring tips and some gotchas to be aware of.

In this episode, we mentioned the following resources:

- Our introductory episode about what services you should use for events: https://www.youtube.com/watch?v=CG7uhkKftoY

- Amazon Kinesis Producer Library (KPL): https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html

- Amazon Kinesis Consumer Library (KCL): https://docs.aws.amazon.com/streams/latest/dev/shared-throughput-kcl-consumers.html

- Kinesis Library wrapper for Node.js: https://github.com/awslabs/amazon-kinesis-client-nodejs

- Kinesis Library wrapper for Python: https://github.com/awslabs/amazon-kinesis-client-python

- Kinesis Data Stream with captures from DynamoDB (for CDC): https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/kds.html

- Kinesis Data Stream with captures from Aurora: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.Overview.html

- Kinesis Data Analytics: https://aws.amazon.com/kinesis/data-analytics/

- Kinesis Firehose: https://aws.amazon.com/kinesis/data-firehose/

- Must-read on Kinesis: Anahit Pogosova’s two-part blog series: https://dev.solita.fi/2020/05/28/kinesis-streams-part-1.html & https://dev.solita.fi/2020/12/21/kinesis-streams-part-2.html

- Cloudonaut Kinesis vs MSK: https://cloudonaut.io/versus/messaging/kinesis-data-streams-vs-msk/

- Deep Dive on Lambda Consumers for Kinesis / Heeki Park - re:invent 2020 https://www.youtube.com/watch?v=tCYwc7-wwsU

You can listen to AWS Bites wherever you get your podcasts. Find all the links on https://awsbites.com

Do you have any AWS questions you would like us to address?

Luciano and Eoin deep dive into SNS discussing what it does, how it differs from EventBridge and SQS and how you can use it to send messages to customers but also for microservices communication. In this new episode dedicated to AWS events and messaging services, we learn everything there is to know about SNS including advantages, limitations and cost. This episode complements the episode about EventBridge, giving another perspective on when to use SNS and when to pick EventBridge instead.

In this episode, we mentioned the following resources:

- Our previous episode about EventBridge: https://www.youtube.com/watch?v=UjIE5qp-v8w

- Our previous episode about all things SQS: https://www.youtube.com/watch?v=svoA-ds8-8c

- Our introductory episode about what services you should use for events: https://www.youtube.com/watch?v=CG7uhkKftoY

- A comparison between EventBridge and SNS by Cloudonaut: https://cloudonaut.io/eventbridge-vs-sns/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

Eoin and Luciano continue their series about event services. In this episode, they chat about EventBridge and explore why this AWS service has such a great potential for event-based serverless applications. This episode presents some interesting examples of when and how to use EventBridge. It also covers all the different classes of events that you can manage with EventBridge: AWS events, third-party events and custom events. We discuss limits and pricing and, finally, we show how things can go wrong and how much you can end up paying for it. We conclude the episode with some tips and resources to avoid shooting yourself in the foot and get good observability when using EventBridge.In this episode, we mentioned the following resources:

- Our previous episode about all things SQS: https://www.youtube.com/watch?v=svoA-ds8-8c

- Our introductory episode about what services you should use for events: https://www.youtube.com/watch?v=CG7uhkKftoY

- List of AWS services that can trigger EventBridge events: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-service-event.html

- An example of how to make HTTP calls directly from EventBridge (Sheen Brisals): https://medium.com/lego-engineering/amazon-eventbridge-api-destinations-demystified-part-i-23fa70d9a04d

- How to test when using EventBridge (by Paul Swail): https://serverlessfirst.com/eventbridge-testing-guide/

- Eventbridge CLI tool: https://github.com/spezam/eventbridge-cli

- Lumigo CLI: https://github.com/lumigo-io/lumigo-CLI#lumigo-cli-tail-eventbridge-bus

- EventBridge Atlas: https://eventbridge-atlas.netlify.app/

- EventBridge Canon: https://eventbridge-canon.netlify.app/

- Accelerate Serverless Adoption with EventBridge (talk by Sheen Brisals): https://www.youtube.com/watch?v=sTZpoSGOSOI

- Series of Articles by Sheen Brisals on EventBridge: https://sbrisals.medium.com/table-of-contents-set-pieces-16c1ca1ecb33

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Luciano and Eoin take a deep dive into SQS as part of a series on AWS event services and event-driven architecture. We talk about the kind of problems SQS can solve, all of the SQS features and how to configure and use SQS to achieve reliability and scalability without all the complexity. We also take some time to detail how SQS works with Lambda in terms of scaling, batching and filtering.

In this episode, we gave a special mention to three highly-recommended re:Invent 2021 talks on the topic of Enterprise Integration Patterns with AWS services:

In addition, we mentioned the following resources.

- SQS: https://aws.amazon.com/sqs/

- Using Lambda with SQS: https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html

- Lambda SQS Scaling: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-sqs-scaling/

- SLIC Watch (Serverless plugin for easy dashboards and alarms): https://github.com/fourTheorem/slic-watch 

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS:

Do you have any AWS questions you would like us to address?

In this episode of AWS Bites podcast, Luciano and Eoin talk about AWS services related to events and message passing like SQS, SNS, Event Bridge, Kinesis and Kafka (MSK).

We discuss in which context is convenient to use messages and events and we deliver a quick walkthrough of all the services discussing major features and some practical examples on how to use them.

In this episode we mentioned the following resources:

- SNS: https://aws.amazon.com/sns/

- SQS: https://aws.amazon.com/sqs/

- Event Bridge: https://aws.amazon.com/eventbridge/

- Kinesis: https://aws.amazon.com/kinesis/

- Kafka (MSK): https://aws.amazon.com/msk/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

In this episode of AWS Bites podcast, Eoin and Luciano talk about whether it is worth to get an AWS certification and why. We discuss why a certification can be important from the perspective of individuals and companies, what are the certifications available and how are they grouped. Finally we try to provide some suggestions for a study plan and give various useful resources and tips.

In this episode we mentioned the following resources:

- Official AWS certifications landing page: https://aws.amazon.com/certification/

- Passing all the AWS certifications, article by Adam Elmore: https://adamelmore.medium.com/descent-into-cloud-madness-12-aws-certifications-in-6-weeks-965de12c626d

- Luciano’s post about AWS Solution Architect Associate exam notes and tips: https://loige.co/aws-solution-architect-associate-exam-notes-tips

- Udemy and Tutorialspoint: https://udemy.com, https://www.tutorialspoint.com

- Adrian Cantrill’s AWS certification training material: https://cantrill.io

- A Cloud Guru: https://acloudguru.com

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

In this episode of AWS Bites podcast, Luciano and Eoin talk about what the serverless Developer Experience (DX) is like and whether we should consider it still immature or not.

In this context we discuss some of the struggles with local development, what are some tools that can make our lives easier and how we can improve the overall serverless experience.

In this episode we mentioned the following resources:

- LocalStack: https://localstack.cloud/

- Serverless offline plugin: https://github.com/dherault/serverless-offline

- SAM Local: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-start-api.html

- SAM Accelerate: https://aws.amazon.com/blogs/compute/accelerating-serverless-development-with-aws-sam-accelerate/

- CDK HotSwap / CDK Watch: https://aws.amazon.com/blogs/developer/increasing-development-speed-with-cdk-watch/

- SST: https://docs.serverless-stack.com/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

In this episode of AWS Bites, Eoin and Luciano discuss some interesting strategies to take a workload to the cloud. Most of this episode is built around “6 Strategies for Migrating Applications to the Cloud” by Stephen Orban: (https://aws.amazon.com/blogs/enterprise-strategy/6-strategies-for-migrating-applications-to-the-cloud/) in which you can find the concepts of the “6 Rs for cloud migrations”: Retire or Retain, Re-purchase, Re-host, Re-platform and Re-architecture.

Throughout the episode we share our take on this approach and we try to provide our perspective and suggestions on how to apply this strategy and get some significant ROI for this project. Moving to the cloud is never a trivial thing and there are many elements to consider, especially in a medium/big organisation. This episode will help you to lay out all the elements that matter the most and come up with a solid plan for your next cloud migration.

In this episode we mentioned the following resources:

- 6 Strategies for Migrating Applications to the Cloud: https://aws.amazon.com/blogs/enterprise-strategy/6-strategies-for-migrating-applications-to-the-cloud/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

- https://twitter.com/eoins

In this episode of AWS Bites podcast, Luciano and Eoin talk about some ideas to stay update with the evergreen world of AWS. This is a much less structured episode than usual and we informally discuss some of our favourite resources including blogs, articles, YouTube channels, other podcasts and Twitter profiles.

In this episode we mentioned the following resources:

- AWS what’s new (official): https://aws.amazon.com/new/

- AWS Podcast ​​https://aws.amazon.com/podcasts/aws-podcast/

- Werner Vogel’s blog: https://www.allthingsdistributed.com/

- Jeremy Daly’s Off by None Newsletter: https://offbynone.io/

- Cloudonaut’s blog: https://cloudonaut.io/

- Cloudonaut’s YouTube channel: https://www.youtube.com/cloudonaut

- Yan Cui’s blog: https://theburningmonk.com/

- Yan Cui’s Real World Serverless: https://realworldserverless.com/

- AWS Builder Library: https://aws.amazon.com/builders-library/

- Borislav Hadzhiev’s blog: https://bobbyhadz.com/

- Marcia Villalba’s Foobar on YouTube: https://www.youtube.com/c/FooBar_codes

- Adam Elmore’s AWS.fm podcast https://aws.fm/

- Emily Shea on Twitter: https://twitter.com/em__shea

- Emily Shea’s re:Invent 2021 “Getting started building your first serverless web application”: https://www.youtube.com/watch?v=DdyhdnWVukc

- Danilo Poccia on Twitter: https://twitter.com/danilop

- Heitor Lessa on Twitter: https://twitter.com/heitor_lessa

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

In this episode, Eoin and Luciano explore the various pros and cons of AWS Cloud Development Kit (CDK).

We start by describing what CDK is and what it is used for. Then we spend a bit of time covering the details of how CDK actually works defining L1, L2 and L3 (patterns) constructs and the integration with CloudFormation.

In the central part of the episode we deep dive into a bunch of pros and cons of CDK, mostly trying to describe the tradeoffs and the pitfalls.

Finally, we close the episode by giving a piece of advice on what we believe is the best way to get started with CDK to minimize the surprises and be able to reap all the benefits of this amazing tools.

In this episode we mentioned the following resources:

- CDK: https://aws.amazon.com/cdk/

- CloudFormation: https://aws.amazon.com/cloudformation/

- Terraform: https://www.terraform.io/

- CDK PIpelines: https://docs.aws.amazon.com/cdk/api/latest/docs/pipelines-readme.html

- CDK Patterns: https://cdkpatterns.com/

- Create Resources conditionally with CDK: https://loige.co/create-resources-conditionally-with-cdk

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter: