AWS Bites

If you are thinking of using an external CICD tool to deploy to AWS you are probably wondering how to securely connect your pipelines to your AWS account.

You could create a user for your CICD tool of choice and copy some hard coded credentials into it, but, let’s face it: this doesn’t feel like the right - or at least the most secure - approach!

In the previous episode we discussed how AWS and GitHub solved this problem by using OIDC identity providers and this seems to be a good solution to the problem.

In this episode of AWS Bites we will try to demystify the secrets of OIDC identity providers and explain how they work and what’s the trust model between AWS and an OIDC provider like GitHub actions. We will also explain all the steps required to integrate AWS with GitHub, how JWT works in this particular scenario and other use cases where you could use OIDC providers.

In this episode, we mentioned the following resources:

- GitHub docs explaining how to integrate with AWS as an OIDC provider: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect

- Article “What’s in a JWT” https://loige.co/whats-in-a-jwt

- jwtinfo, CLI tool to inspect JWT: https://github.com/lmammino/jwtinfo

- AWS action to assume a role from a GitHub Pipeline: https://github.com/aws-actions/configure-aws-credentials#assuming-a-role

- Great post by Elias Brange detailing how to setup GitHub OIDC integration for AWS: https://www.eliasbrange.dev/posts/secure-aws-deploys-from-github-actions-with-oidc/

- Previous episode on why you should consider GitHub Actions rather than AWS CodePipeline: https://awsbites.com/44-do-you-use-codepipeline-or-github-actions/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts. See https://awsbites.com for all the links.

Do you have any AWS questions you would like us to address?

Automated, Continuous Build and Continuous Delivery are must-haves when building modern applications on AWS. To achieve this, you have numerous options, including third party providers like GitHub Actions and Circle CI, and the AWS services, CodePipeline and CodeBuild. In this episode we focus on GitHub Actions and we compare it with the native AWS features offered by services like CodePipeline and Code Build. In particular we discuss what CodePipeline offers and how to set it up, what the tradeoffs are and when to choose one over the other. We also discuss when you should look outside AWS to a third-party provider and highlight when GitHub Actions can be a great fit for your AWS CI/CD needs!

In this episode, we mentioned the following resources:

- Example pipeline for a serverless mono repo using CDK is available in SLIC Starter: https://github.com/fourTheorem/slic-starter/tree/main/packages/cicd

- 50+ official actions provided by GitHub themselves: https://github.com/actions

- How to configure OIDC integrations with AWS and other services like GitHub: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services

- GitHub Actions billing details: https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions

- Workshop illustrating how to create CodeBuild and CodePipeline resources using CDK: https://cdkworkshop.com/20-typescript/70-advanced-topics/200-pipelines/3000-new-pipeline.html

- Paul Swail’s article “Why I switched from AWS CodePipeline to GitHub Actions”: https://serverlessfirst.com/switch-codepipeline-to-github-actions/

- A tutorial article by AWS showing how to authenticate and use GitHub actions to build & deploy a web app to an EC2 instance https://aws.amazon.com/blogs/devops/integrating-with-github-actions-ci-cd-pipeline-to-deploy-a-web-app-to-amazon-ec2/

- Other examples of when it is OK to ditch AWS services for third party (previous podcast episode): https://awsbites.com/43-when-is-it-ok-to-cheat-on-aws/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts. See https://awsbites.com

Leave a comment here or connect with us on Twitter:

- https://twitter.com/eoins

We do love AWS, but sometimes we have to admit that it’s not always a silver bullet. There are definitely use cases where it might be worth considering alternatives to AWS.

In this episode we will discuss some of these use cases and try to highlight what are the advantages that other platforms or services can have over AWS in very specific circumstances. First of all we clarify why we like AWS and why (and when) it’s worth sticking with it. Then, we discuss what are some of the reasons why it might be worth considering alternatives to AWS. At this point we go into the specifics and talk about authentication services (Auth0), search services (ElasticSearch, Algolia), CDN Services (GitHub Pages, Netlify, Vercel, CloudFlare, Fastly, Akamai), Databases (MongoDB Atlas, Digital Ocean managed databases, IBM Compose, CloudFlare D1, Upstash, Confluent Kafka), Headless CMS services (ContentFul, Storyful, AirTable, Google Spreadsheet), Virtual Machine services (Digital Ocean, Linode).

- Episode 3. "​​How do you deploy a static website on AWS?”: https://awsbites.com/3-how-do-you-deploy-a-static-website-on-aws/

- Auth0: https://auth0.com/

- Amazon OpenSearch: https://aws.amazon.com/opensearch-service/the-elk-stack/what-is-opensearch/

- Elastic Cloud: https://www.elastic.co/cloud/

- Algolia: https://www.algolia.com/

- Vercel: https://vercel.com/

- Netlify: https://www.netlify.com/

- MongoDB Atlas: https://www.mongodb.com/atlas/database

- Digital Ocean managed database: https://try.digitalocean.com/managed-databases/

- Compose (now IBM Cloud Databases): https://www.compose.com/

- Upstash: https://upstash.com/

- Confluent: https://www.confluent.io/

- AirTable: https://airtable.com/

- Linode: https://www.linode.com/

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

We recently talked about migrating a monolithic application to AWS, using EC2, load balancers, S3 and RDS. In this episode we want to talk about a slightly different setup, where we are going for containers instead of EC2 and we want to deploy them in Fargate. In this We are going to cover all the components you will need in your architecture, the reasons to choose Fargate over any alternatives and discuss some CDK tricks to get started in a quick way (and the pitfalls that might come with them).

In this episode, we mentioned the following resources:

- CDK ECS Patterns: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs_patterns-readme.html

- How to fine tune the health checks to speed up the deployment process: https://www.qovery.com/blog/how-to-speed-up-amazon-ecs-container-deployments

- Previous Episode “37. How do you migrate a monolith to AWS without the drama?”: https://awsbites.com/37-how-do-you-migrate-a-monolith-to-aws-without-the-drama/

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Lambda functions are small units of code that achieve a specific purpose. It’s always a good idea to keep your code short, clean and simple. And yet, sometimes you find yourself writing lots of boilerplate code in every function to do common things, like parsing events, validation, loading parameters and lots more.

The Middy.js framework was designed to help you keep Node.js Lambda function code simple, letting you focus on the business logic and clearing away duplication and boilerplate. By the end of this episode, you will know: How Middy.js works with JavaScript and TypeScript, how to perform validation, event parsing and parameter loading, and how you can even write and use your own Middy middleware. Finally you’ll get to know who is using Middy and how you could contribute to the Middy ecosystem.

In this episode, we mentioned the following resources:

- Middy Website and documentation: https://middy.js.org/

- How to get started with middy (official docs): https://middy.js.org/docs

- Middy official middlewares (official docs): https://middy.js.org/docs/middlewares/intro

- How to write your own middlewares (official docs): https://middy.js.org/docs/category/writing-middlewares

- Middy integrations (official docs): https://middy.js.org/docs/category/integrations

- Interview with Taco Bell in an episode of Real World Serverless where they mention how they use middy: https://www.listennotes.com/podcasts/real-world/56-serverless-at-tacobell--k5gAQBMHSb/

- Open source projects using Middy: https://github.com/middyjs/middy/network/dependents?package_id=UGFja2FnZS00Njc1NDUzOTU%3D

- Lambda Power Tools for TypeScript Middy Integration: https://awslabs.github.io/aws-lambda-powertools-typescript/0.10.0/core/logger/#capturing-lambda-context-info

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Identity Access Management, also known as IAM, can be an intimidating service when getting started with AWS. But IAM is also one of those core services that you can’t really avoid. In this episode we try to distill down everything that you need to know to understand IAM and start to use it proficiently. We cover what IAM is, why it is so important, how authentication and authorization work, what policy documents are and how to write them, how a user or an application get credentials to interact with AWS and finally many examples, tips and tricks.

In this episode, we mentioned the following resources:

- IAM access analyzer:  https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html

- Our previous episode on how to manage AWS credentials: https://www.youtube.com/watch?v=X4uA6EtNyos

- Understanding how IAM Policies work (AWS Docs): https://docs.aws.amazon.com/IAM/latest/UserGuide/intro-structure.html

- Policy simulator: https://policysim.aws.amazon.com/

- Policy evaluation logic (AWS Docs): https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html

- How the Signature v4 (sigv4) algorithm works (AWS Docs): ​​https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html

- Actions, resources, and condition keys for AWS services - https://docs.aws.amazon.com/service-authorization/latest/reference/reference_policies_actions-resources-contextkeys.html

- Video IAM Concepts by beabetterdev: https://www.youtube.com/watch?v=_ZCTvmaPgao

- Re:Invent session “Getting started with AWS identity” by Becky Weiss: https://www.youtube.com/watch?v=Zvz-qYYhvMk

- Re:Invent session “Become an IAM Policy Master in 60 Minutes or Less” by Brigid Johnson:  https://www.youtube.com/watch?v=YQsK4MtsELU

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:

When it comes to building and deploying microservice applications on AWS, there are 2 emerging best practices: use a separate AWS account per application (and environment) and decouple communication between separate systems using events (instead of point-to-point communication). Can we use these two best practices together? Yes, but we will need to find a way to pass messages between AWS accounts! In this episode we discuss how to do that using EventBridge as a cross-account event backbone! We discuss why these 2 suggestions are well established best practices, what are the pros and cons that they bring to the table, what an event backbone is and why EventBridge is a great service to implement one. Finally, we will discuss a case study and an example implementation of this pattern in the context of an e-commerce application built with a microservices architecture.

In this episode, we mentioned the following resources:

- Article “How to use EventBridge as a Cross-Account Event Backbone” https://dev.to/eoinsha/how-to-use-eventbridge-as-a-cross-account-event-backbone-5fik

- Repository with example code: https://github.com/fourTheorem/cross-account-eventbridge/

- Article “What can you do with EventBridge?” (fourTheorem blog): https://www.fourtheorem.com/blog/what-can-you-do-with-eventbridge

- For great ideas on structuring event payloads, take a read of Sheen Brisals' post on the Lego Engineering blog: https://medium.com/lego-engineering/the-power-of-amazon-eventbridge-is-in-its-detail-92c07ddcaa40

- Article “What do you need to know about SNS?” (fourTheorem blog) which includes a comparison of SNS and EventBridge: https://www.fourtheorem.com/blog/what-do-you-need-to-know-about-sns

- AWS Bites Episode 23: “What’s the big deal with EventBridge?” : https://youtu.be/UjIE5qp-v8w

- AWS Community Day talk by Luc van Donkersgoed “Event-Driven Architecture at PostNL Scale”

https://www.youtube.com/watch?v=nyoMF1AEI7g

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

#aws #microservice #eventbridge

When it comes to choosing compute services on AWS, there are a lot of options, including EC2, ECS, Lambda, EKS… New ones keep emerging all the time! Selecting the right one for each application is no longer an easy choice. In this episode we discuss why you need compute services and what kinds of problems should be offloaded to something else entirely. We suggest how you can develop a methodology to make the selection process easier and less biased within your company. We discuss at a high level what are some of the different compute options available in AWS and finally we provide a few different options example use cases and describe how we picked the compute service for each.

In this episode, we mentioned the following resources:

- InfoQ article “A Recipe to Migrate and Scale Monoliths in the Cloud”: https://www.infoq.com/articles/cloud-migrate-scale/

- Our previous episode about migrating monoliths to the cloud: https://www.youtube.com/watch?v=GYa2RkYDfBQ

- Article on choosing the right compute service: https://www.fourtheorem.com/blog/aws-compute

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

#aws #compute #lambda

Migrating monoliths to the cloud can be a scary, expensive and time consuming and time consuming activity. Many companies try to avoid it even if it could be beneficial for them, just because they think it would require too much work and be too risky. But there are interesting compromises and incremental approaches that can be used to simplify and de-risk this kind of migration. The idea is that you don’t necessarily have to dramatically re-engineer your application to move it to the cloud (and start to take advantage of it).

In this episode, based on an InfoQ article that we recently published, we discuss a fictional use case where a company with a monolithic application managed to move to the cloud with a minimum amount of change. The move to the cloud has brought more scalability and resilience for the company to move forward and expand. But it also brings new challenges and opportunities. We will discuss all of this in more detail and by the end of this episode you should have a checklist for migrating monoliths to the cloud with minimal effort.

In this episode, we mentioned the following resources:

- InfoQ article “A Recipe to Migrate and Scale Monoliths in the Cloud”: https://www.infoq.com/articles/cloud-migrate-scale/

- Our previous episode about other cloud migration strategies: https://www.youtube.com/watch?v=IDh4eQzbvyg

- Our previous episode about the difference between CloudFormation and Terraform for infrastructure as code: https://www.youtube.com/watch?v=mLkOH2I0rX8

- Our previous episode about the pros and cons of CDK for infrastructure as code: https://www.youtube.com/watch?v=jjyNTNQdW2s

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS:

Do you have any AWS questions you would like us to address?

#aws #monolith #migration

There are lots of options for programming languages on AWS these days but one of the most popular ones remains JavaScript. In this episode of AWS Bites we discuss what it’s like to develop with JavaScript, Node.js and TypeScript on AWS and what’s new in this field.

We explore why you would choose JavaScript and what are the trade-offs that come with this choice. We present some of the main features of the all-new AWS SDK v3 for JavaScript. We discuss runtime support and tooling for AWS Lambda and finally some interesting developments in the JavaScript ecosystem for the cloud and AWS.

- Our previous episode on What language to use for lambda: https://www.youtube.com/watch?v=S0tpReRa6m4

- AI as a Service by Eoin Shanaghy and Peter Elger (book): https://www.manning.com/books/ai-as-a-service

- Node.js Design Patterns by Mario Casciaro and Luciano Mammino (book): https://www.nodejsdesignpatterns.com/

- AWS SDK for JavaScript v3 high level concepts (including command based model): https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html#high-level-concepts

- AWS SDK for JavaScript v3 paginators using Async Iterators: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html#paginators

- Mocking support for the AWS SDK for JavaScript v3: https://aws.amazon.com/blogs/developer/mocking-modular-aws-sdk-for-javascript-v3-in-unit-tests/

- Various interesting benchmarks on different Lambda runtimes: https://github.com/theam/aws-lambda-benchmark - https://filia-aleks.medium.com/benchmarking-all-aws-lambda-runtimes-in-2021-cold-start-part-1-e4146fe89385 - https://www.simform.com/blog/aws-lambda-performance/

- Support for ESM modules in AWS Lambda (Node.js 14): https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/

- The Middy Framework (middleware pattern for AWS Lambda): https://middy.js.org/

- Lambda Power Tools library for TypeScript: https://awslabs.github.io/aws-lambda-powertools-typescript/

- Yan Cui’s article on performance improvements with bundling: https://lumigo.io/blog/3-major-ways-to-improve-aws-lambda-performance/

- ZX project (scripting with JavaScript) by Google: https://github.com/google/zx

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter: