Last Week In AWS Podcast

Links:

• GitHub organizations: https://alsmola.medium.com/securing-github-organizations-9c33c850638
• CloudTrail would spew other accounts’ credentials your way: https://onecloudplease.com/blog/security-september-cataclysms-in-the-cloud-formations
• Spot on: https://research.nccgroup.com/2022/01/13/10-real-world-stories-of-how-weve-compromised-ci-cd-pipelines/
• Some excellent points: https://www.darkreading.com/cloud/enterprises-are-sailing-into-a-perfect-storm-of-cloud-risk
• “Amazon EC2 customers can now use ED25519 keys for authentication with EC2 Instance Connect”: https://aws.amazon.com/about-aws/whats-new/2022/01/ed25519-keys-authentication-ec2-instance-connect/
• “Integrating AWS Security Hub, IBM Netcool, and ServiceNow, to Secure Large Client Deployments”: https://aws.amazon.com/blogs/apn/integrating-aws-security-hub-ibm-netcool-and-servicenow-to-secure-large-client-deployments/
• “Best practices for cross-Region aggregation of security findings”: https://aws.amazon.com/blogs/security/best-practices-for-cross-region-aggregation-of-security-findings/
• Assume AWS IAM Roles using SAML.to in GitHub Actions: https://github.com/saml-to/assume-aws-role-action

Transcript

Corey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.

Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They’ve also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That’s S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.

Corey: So, most interesting this week is probably my request for AWS to support a different breed of SSH key. No, it’s not a joke. Listen on and we’ll get there.

So, from the security community last week, everyone talks about how to secure AWS environments. This post takes a different direction and talks about how to secure GitHub organizations, which makes sense if you think about it as an area to focus on. If you compromise an org’s GitHub repositories, it’s basically game over for that company.

I also came across this post from 2020, talking about how if asked politely, CloudTrail would spew other accounts’ credentials your way. How many more exploits like this have we seen and just never been told about?

NCC Group has some great stories up about compromising CI/CD pipelines, and they are all spot on. Because nobody really thinks about the Jenkins box that has everyone working with it, outsized permissions, and of course, no oversight.

Enterprise cloud risk is a very real thing, so a post from Josh Stella, who’s the CEO of Fwage—though he pronounces it as ‘Fugue’—and it makes some excellent points, and also cites me, so of course, I’m going to mention it here. We incentivize the behaviors we want to see more of. There’s a security lesson in there somewhere.

Corey: This episode is sponsored in part by our friends atNew Relic. If you’re like most environments, you probably have an incredibly complicated architecture, which means that monitoring it is going to take a dozen different tools. And then we get into the advanced stuff. We all have been there and know that pain, or will learn it shortly, and New Relic wants to change that. They’ve designed everything you need in one platform with pricing that’s simple and straightforward, and that means no more counting hosts. You also can get one user and a hundred gigabytes a month, totally free. To learn more, visitnewrelic.com. Observability made simple.

Now, from AWS, what have they said? “Amazon EC2 customers can now use ED25519 keys for authentication with EC2 Instance Connect”. I really wish they’d add support for ECDSA keys as well, and no, this is not me making a joke. Those are the only key types Apple lets you store in the Secure Enclave on Macs that support it, and as a result, you can use that while never exporting the private key. I try very hard to avoid having private key material resident on disk, and that would make it one step easier.

“Integrating AWS Security Hub, IBM Netcool, and ServiceNow, to Secure Large Client Deployments”. I keep talking about how if it’s not simple, it’s very hard to secure. AWS, IBM, and ServiceNow, all integrating is about as far from “Simple” as is possible to get.

“Best practices for cross-Region aggregation of security findings”. And this was a post that I was about to snark that it should be as simple as “Click the button,” but then I read my post, and to my surprise and yes, delight, it already is. Good work.

And in the land of tool, I found a post talking about how to assume AWS IAM Roles using SAML.to in GitHub Actions, and I really wish that that was first-party, but I’ll take what I can get. Because again, I despise the idea of permanent IAM credentials just hanging out in GitHub or on disk or, realistically, anywhere. I like these ephemeral approaches. You can be a lot more dynamic with it and breaching those credentials doesn’t generally result in disaster for everyone. And that’s what happened last week in AWS security.

Corey: Thank you for listening to the AWS Morning Brief: Security Edition with the latest in AWS security that actually matters. Please follow AWS Morning Brief on Apple Podcast, Spotify, Overcast—or whereve...

Want to give your ears a break and read this as an article? You’re looking for this link.

https://www.lastweekinaws.com/blog/clickops

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

Links:

• S3 Bucket Negligence Award: http://saharareporters.com/2022/01/10/exclusive-hacker-breaks-nimc-server-steals-over-three-million-national-identity-numbers
• Anyone in a VPC, any VPC, anywhere: https://Twitter.com/santosh_ankr/status/1481387630973493251
• A disgruntled developer corrupts their own NPM libs ‘colors’ and ‘faker’, breaking thousands of apps: https://www.bleepingcomputer.com/news/security/dev-corrupts-npm-libs-colors-and-faker-breaking-thousands-of-apps/
• “Top ten security best practices for securing backups in AWS”: https://aws.amazon.com/blogs/security/top-10-security-best-practices-for-securing-backups-in-aws/
• Glue: https://aws.amazon.com/security/security-bulletins/AWS-2022-002/
• CloudFormation: https://aws.amazon.com/security/security-bulletins/AWS-2022-001/
• S3-credentials: https://simonwillison.net/2022/Jan/18/weeknotes/

Transcript

Corey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.

Corey: This episode is sponsored in part by my friends at Thinkst Canary. Most companies find out way too late that they’ve been breached. Thinkst Canary changes this and I love how they do it. Deploy canaries and canary tokens in minutes, and then forget about them. What’s great is then attackers tip their hand by touching them, giving you one alert, when it matters. I use it myself and I only remember this when I get the weekly update with a, “We’re still here, so you’re aware,” from them. It’s glorious. There is zero admin overhead to this, there are effectively no false positives unless I do something foolish. Canaries are deployed and loved on all seven continents. You can check out what people are saying atcanary.love. And, their Kube config canary token is new and completely free as well. You can do an awful lot without paying them a dime, which is one of the things I love about them. It is useful stuff and not a, “Oh, I wish I had money.” It is spectacular. Take a look. That'scanary.love because it’s genuinely rare to find a security product that people talk about in terms of love. It really is a neat thing to see.Canary.love. Thank you to Thinkst Canary for their support of my ridiculous, ridiculous nonsense.

Corey: So, yesterday’s episode put the boots to AWS, not so much for the issues that Orca Security uncovered, but rather for its poor communication around the topic. Now that that’s done, let’s look at the more mundane news from last week’s cloud world. Every day is a new page around here, full of opportunity and possibility in equal measure.

This week’s S3 Bucket Negligence Award goes to the Nigerian government for exposing millions of their citizens to a third party who most assuredly did not follow coordinated disclosure guidelines. Whoops.

There’s an interesting tweet, and exploring it is still unfolding at time of this writing, but it looks that making an API Gateway ‘Private’ doesn’t mean, “To your VPCs,” but rather, “To anyone in a VPC, any VPC, anywhere.” This is evocative of the way that, “Any Authenticated AWS User,” for S3 buckets caused massive permissions issues industry-wide.

And a periodic and growing concern is one of software supply chain—which is a fancy way of saying, “We’re all built on giant dependency chains”—what happens when, say, a disgruntled developer corrupts their own NPM libs ‘colors’ and ‘faker’, breaking thousands of apps across the industry, including some of the AWS SDKs? How do we manage that risk? How do we keep developers gruntled?

Corey: Are you building cloud applications with a distributed team? Check out Teleport, an open-source identity-aware access proxy for cloud resources. Teleport provides secure access for anything running somewhere behind NAT: SSH servers, Kubernetes clusters, internal web apps, and databases. Teleport gives engineers superpowers.

Get access to everything via single sign-on with multi-factor, list and see all of SSH servers, Kubernetes clusters, or databases available to you in one place, and get instant access to them using tools you already have. Teleport ensures best security practices like role-based access, preventing data exfiltration, providing visibility, and ensuring compliance. And best of all, Teleport is open-source and a pleasure to use. Download Teleport at goteleport.com. That’s goteleport.com.

AWS had a couple of interesting things. The first is “Top ten security best practices for securing backups in AWS”. People really don’t consider the security implications of their backups anywhere near seriously enough. It’s not ‘live’ but it’s still got—by definition—a full set of your data just waiting to be harvested by nefarious types. Be careful with that.

And of course, AWS had two security bulletins, one about its Glue issues, one about its CloudFormation issues. The former allowed cross-account access to other tenants. In theory. In practice, AWS did the responsible thing and kept every access event logged, going back for the full five years of the service’s life. That’s remarkably impressive.

And lastly, I found an interesting tool called S3-credentials last week, and what it does is it helps generate tightly-scoped IAM policies that were previously limited to a single S3 bucket, but now are limited to a single prefix within that bucket. You can also make those credential sets incredibly short-lived. More things like this, please. I just tend to over-scope things way too much. And that’s what happened Last Week in AWS: Security. Please feel free to reach out and tell me exactly what my problem is.

Corey: Thank you for listening to the AWS Morning Brief: Security Edition with the latest in AWS security that actually matters. Please follow AWS Morning Brief on Apple Podcast, Spotify, Overcast—or wherever the hell it is you find the dulcet tones of my voice—a...

Want to give your ears a break and read this as an article? You’re looking for this link.

https://www.lastweekinaws.com/blog/orca-security-aws-and-the-killer-whale-of-a-problem

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

Links:

• Comes with a cryptominer: https://krebsonsecurity.com/2022/01/norton-360-now-comes-with-a-cryptominer/
• You could be federally charged with wire fraud for paying off a security researcher: https://www.justice.gov/usao-ndca/pr/former-uber-chief-security-officer-face-wire-fraud-charges-0
• A source code leak of its Azure App Service: https://www.theregister.com/2021/12/24/azure_app_service_not_legit_source_code_leak/
• “Comprehensive Cyber Security Framework for Primary (Urban) Cooperative Banks (UCBs)”: https://aws.amazon.com/blogs/security/comprehensive-cyber-security-framework-for-primary-urban-cooperative-banks/
• “Disabling Security Hub controls in a multi account environment”: https://aws.amazon.com/blogs/security/disabling-security-hub-controls-in-a-multi-account-environment/
• Ipv6-ghost-ship: https://github.com/aidansteele/ipv6-ghost-ship

Transcript

Corey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.

This episode is sponsored in part by our friends at Rising Cloud, which I hadn’t heard of before, but they’re doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they’re using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they’re able to wind up taking what you’re running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I’m somewhat skeptical, but their customers seem to really like them, so that’s one of those areas where I really have a hard time being too snarky about it because when you solve a customer’s problem and they get out there in public and say, “We’re solving a problem,” it’s very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it’s worth exploring. So, if you’re looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That’s risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.

Welcome to Last Week in AWS: Security. Let’s dive in. Norton 360—which sounds like a prelude to an incredibly dorky attempt at the moonwalk—now comes with a cryptominer. You know, the thing that use tools like this to avoid having on your computer? This is apparently to offset how zippy modern computers have gotten, in a direct affront to Norton’s ability to make even maxed-out laptops run like total garbage. Speaking of total garbage, you almost certainly want to use literally any other vendor for this stuff now.

“What’s the worst that can happen?” Is sometimes a comforting thought when dealing with professional challenges. If you’re the former Uber CISO, the answer to that question is apparently, “you could be federally charged with wire fraud for paying off a security researcher.”

And lastly, Azure continues to have security woes, this time in the form of a source code leak of its Azure App Service. It’s a bad six months and counting to be over in Microsoft-land when it comes to cloud.

Let’s take a look what AWS has done. “Comprehensive Cyber Security Framework for Primary (Urban) Cooperative Banks (UCBs)”. This is a perfect case study in what’s wrong with the way we talk about security. First, clicking the link to the report in the blog post threw an error; I had to navigate to the AWS Artifact console and download the PDF manually. Then, the PDF is all of two pages long, as it apparently has an embedded Excel document within it that Preview on my Mac can’t detect. The proper next step is to download Adobe Acrobat for Mac in order to read this, but I’ve given up by this point. This may be the most remarkable case of AWS truly understanding its customer mentality that we’ve seen so far this year.

Are you building cloud applications with a distributed team? Check out Teleport, an open-source identity-aware access proxy for cloud resources. Teleport provides secure access for anything running somewhere behind NAT: SSH servers, Kubernetes clusters, internal web apps, and databases. Teleport gives engineers superpowers. Get access to everything via single sign-on with multi-factor, list and see all of SSH servers, Kubernetes clusters, or databases available to you in one place, and get instant access to them using tools you already have. Teleport ensures best security practices like role-based access, preventing data exfiltration, providing visibility, and ensuring compliance. And best of all, Teleport is open-source and a pleasure to use. Download Teleport at goteleport.com. That’s goteleport.com.

“Disabling Security Hub controls in a multi account environment”. I hate that this is a solution instead of a native feature, but it’s important. There are some Security Hub controls that are just nonsense. “Oh no, you didn’t encrypt your EBS volumes.” “Oh dear, you haven’t rotated your IAM credentials in 90 days.” “Holy CRAP, the S3 bucket serving static assets to the world is world-readable.” You get the picture.

And a tool I found fun, “Port Knocking” is an old security technique in which you attempt to connect to a host on a predetermined sequence of ports. Get it right and you’re now able to connect to the host in question on the port that you want. ipv6-ghost-ship has done something similar yet ever more ridiculous: It takes advantage of the fact that IPv6 means that each EC2 instance gets 281 trillion IP addresses to only accept SSH connections when the last three octets of the IP address on the instance match the time-based authentication code. This is a ridiculous hack, and I love it oh so very much. I’m Chief Cloud Economist at The Duckbill Group, and this has been Last We...

Want to give your ears a break and read this as an article? You’re looking for this link.

https://www.lastweekinaws.com/blog/azures-terrible-security-posture-comes-home-to-roost/

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill