Last Week In AWS Podcast

Links

• Follow Last Week In AWS on Twitter

Transcript
Corey: Gravitational is now Teleport because when way more people have heard of your product than your company, maybe that’s a sign it’s a time to change your branding. Teleport enables engineers to quickly access any computing resource, anywhere on the planet. You know, like VPNs were supposed to do before we all started working from home, and the VPNs melted like glaciers. Teleport provides a unified access plane for developers and security professionals seeking to simplify secure access to servers, applications, and data across all of your environments without the bottleneck and management overhead of traditional VPNs. This feels to me like it’s a lot like the early days of HashiCorp’s Terraform. My gut tells me this is the sort of thing that’s going to transform how people access their cloud services and environments. To learn more, visit goteleport.com.

Pete: Hello, and welcome to AWS Morning Brief. I am Pete Cheslock, and I'm also here, again, with Jesse DeRose. Hey, Jesse, how's it going?

Jesse: Not too bad. Thanks for having me.

Pete: It is part two of AWS Storage Day. If you haven't had the chance to listen to last week's episode, Jesse and I dove into some of the new features really focusing on what we would think is the biggest feature of AWS Storage Day, which was the S3 Intelligent Tiering. Go back and listen to it if you didn't hear about it. But essentially, Amazon keeps extending out features [00:01:34 unintelligible] this Intelligent Tiering platform. And we talked a little bit about it last week. 

But there were a lot of announcements as part of Storage Day, some pretty impressive, and some that were maybe a little underwhelming. We'll let you be the judge of that because some of these things could be incredibly important for you as—maybe—someone who operates on Amazon. So, now what we're going to do is we're going to dive into some of the other features, not only additional interesting S3 features, but there were a lot of new features announced around EBS, and EFS, and FSx, and all of the different ways that you can interact with AWS storage. I don't want to call it the biggest feature of this section because I think—let's be honest—they're all equally meh features, right, Jesse?

Jesse: Yeah.

Pete: I think that's going to be the common thread. Again, you might look at some of these features and go, “Finally, my life is so much better because they've announced this feature.” But I got to say, outside of Intelligent Tiering, Storage Day felt a little weak. But let's dive in anyway. S3 Replication; if you are replicating your data from one S3 bucket to another bucket, another region, which maybe you need to do for compliance reasons, disaster recovery reasons, some of the new features they added are around replication metrics and notifications. 

Now, previously, these metrics and notifications were only available if you used the Time Control Replication, and that is a additional charge to get a predictable SLA for your data to be backed up. They made these metrics now available for anyone, so that's actually awesome to hear that they’ve really just extended that out and are kind of giving you something for free. Additionally, they now replicate delete markers, which I swear I looked at a bunch of documents to understand better what delete markers mean, and the best I got to it, I don't actually really understand the problem from before, other than as you delete a version of something in the source, the delete marker moves over. But then maybe the previous versions are in the destination. That was my gist of it, Jessie, what was your gist of that one?

Jesse: Yeah, I struggled a little bit with some of these previously because S3 replication always felt like this magical hand-wavy feature where you turned it on and then just waited, and eventually your objects would show up in your destination bucket or destination folder. But there wasn't really any clear path to what was going on behind the scenes. So, I'm really excited to see that now these metrics and notifications are available to everyone, not just to folks who were using the Replication Time Control feature, and allows everybody to more easily understand how their data is replicating between S3 buckets behind the scenes. So, I feel good about this one. I feel like this is definitely a step in the right direction. I'm really excited to see that this is now broadly available for everybody that's using S3. I think it will make using S3 Replication easier for a lot of folks who need it for business purposes or any other use case.

Pete: Yeah, absolutely. Another really awesome feature—I was actually excited for this because, of course, it must affect me in my day-to-day—S3 object ownership is now available for all the Amazon regions and amazingly supported by CloudFormation, which I feel like is always an afterthought. But what this allows you to do is you can use this feature too, when you upload files, it'll make sure that the ownership is assumed by the bucket you've uploaded it into. And so this gets around a lot of hairy issues that come into S3 permissioning, IAM permissioning. I mean, S3 permissioning, in general, predates IAM. I don't know how many people actually know that. And I think because of it, there are some really gnarly edge cases people run into, and this is a big problem solver.

Jesse: I am really, really excited about this feature release, I cannot say how many times we've run into this edge case with some of our internal tooling because we have effectively copied or synced data from a client's S3 bucket into our S3 bucket, and we don't gain ownership. And that becomes such a permissioning headache to be able to do anything with that data once we have it in our S3 bucket. So, I'm really excited to see that object ownership is now not only a first-class citizen but now is also built into and supported by AWS CloudFormation.

Pete: Yeah, absolutely. Another new feature: it has to do with Outpost actually, and you can get S3 on Outposts now which, that's truly amazing if you think about it. Now, I don't know of anyone who actually is using Outposts, and I would love to chat with someone who can, if they're even allowed to, or if they're stuck under an NDA. But what an Outpost allows you to do is essentially purchase a rack of AWS; it's a rack of servers and storage with Amazon APIs. If you really just think about that for a second, that's pretty impressive. 

And if you are going to do hybrid cloud, and you have maybe some data locality requirements like you really need data in a specific location and that's not a region that Amazon supports, or you have data centers, or there's always some requirements, you can now get S3 on there. And they said that they can support 48 or 96 terabytes of S3 capacity per Outpost. What that actually means—like, is that a rack? Is that a whole rack? Is that just a single S3 configuration? Hard to really know. There's no API to go and provision an Outpost yet.

Jesse: Yeah, I'm really curious about this one to see how folks end up using it because I'm super excited that this is a feature that's now available. I love the idea of Outposts, even though it may not be a business use case for us internally. But I'm really curious to see how thi...

Want to give your ears a break and read this as an article? You’re looking for this link.

Sponsors

• Gravitational
• Linode

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

Links

• Follow Last Week In AWS on Twitter

Transcript

Corey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.

Pete: Hello, and welcome to AWS Morning Brief. I am Pete Cheslock. Corey, while being back from his paternity leave, is still not here. We are having too much fun. And by we, I mean I'm joined again with Jesse DeRose. Hey, Jesse. 

Jesse: Thanks as always for having me, Pete. 

Pete: It's so much fun to again chat with people outside of my little family unit, that we've just decided not to give this back to Corey. And luckily, Corey has many other podcasts that he does, he was pretty happy to give it away.

Jesse: I feel like you should never talk about your children that way, but he's got a plethora at this point. So, he's willing to kind of share the wealth.

Pete: Exactly. And if you notice, we have a new theme song that came out, I think it was last week was the first week that we brought in the new theme song, which is I think much in line with a previous episode where we talked about ’80s breakdancing movies that the new theme song kind of has that vibe to it.

Jesse: I hope you're wearing the Members Only jean jacket that I sent you, along with the shades to match the uniform.

Pete: Yeah. I mean, I was born in ’80, so the ’80s for me, I was very young. I'm kind of waiting for the ’90s movies to come around again because I want to rock out my JNCO jeans and my wallet chain. 

Jesse: [laugh], yes.

Pete: And all that good stuff.

Jesse: I am ready.

Pete: Exactly. Well, what are we talking about today? Well, earlier this week, AWS Storage Day 2020 happened on Tuesday. If you were a part of that, it was a free online event. As Amazon called it, a full day online event. Except it was only about four hours long, so kind of mailing it in on that one, huh?

Jesse: Can we start discussing that with our boss and say that a full day of work is technically just four hours? Can we just start working with that going forward?

Pete: Yeah, we'll just say it right now. So, hey, Corey, we're done for the day. Put in the old college four.

Jesse: [laugh]. That's what you say, “I put in the old college try. I just did my full day of four hours, according to AWS. So, this has been great. I'll talk to you tomorrow.”

Pete: Exactly. Well, Storage Day this year—it's the second year in a row if I'm remembering it correctly. 2019 was the last year they did that—and I feel like this kind of ties into the fact that there's just so many announcements that happened around re:Invent, that leading up into re:Invent, you have a lot of announcements to maybe soften the blow for a lot of folks. And Storage Day, really is just this whole day—well, four hours worth of a whole day—talking about everything related to storage. And we're talking about things like S3, EBS, EFS, FSx, for the five huge enterprises that probably use FSx. 

Although if you actually do use FSx, I'd be curious to hear about how you like it and what you think of it because we don't really hear a lot of people using it. But these are all the services, plus many more, that Amazon talked about as part of its Storage Day.

Jesse: Yeah, it was a really interesting discussion. I greatly appreciate that AWS broke out this discussion prior to AWS re:Invent, but they dropped a lot of knowledge on us all at once, and in, like, rapid-fire succession, I was really, kind of… not necessarily surprised, but there's a lot of information that they shared all at once. And I have to admit that after sitting through this presentation, I now have a greater appreciation for Apple's slow presentation style. As much as I hate it; as much as I hate sitting for an hour and a half for one announcement while they toot their own horn, I have to say that the buildup and getting me involved in the story and bringing me along with them. It works, it absolutely works. And it was kind of hard for me to pick up on all the things that went on during AWS Storage Day this year because there was a lot of things going on.

Pete: And honestly, the fact they give so much information is really amazing in, I guess, both their ability to tout, in many cases, minor feature changes that most SaaS businesses would just turn on and maybe blog about. But this is—obviously the engine of AWS is so good at discussing their wins. But you're right, it's just a huge amount. On Monday, Jeff Barr of course, wrote the blog post with a lot of these details, linking to countless other blog posts. And I think it really speaks to just how, probably every, or nearly every Amazon service ties into storage in some way. It's a huge, huge part of this ecosystem. 

Jesse: Absolutely. 

Pete: So, as you can imagine, there were so many new features that we're not even going to be able to cover them all throughout the course, but we did want to call out some of the big ones, or at least what we thought were the biggest ones, the most interesting new features, new product announcements that came out, and also just touch on some of the other things that we thought were pretty interesting as well. And yeah, there was a lot of fun stuff. I think the biggest one that was announced was the S3 Intelligent-Tiering, which is a class storage tier within S3, adds additional levels of archive access. So, if you imagine Intelligent-Tiering, you know, you have the automatic tiering of data from frequently accessed to infrequently accessed as things age out, they essentially automate that for you. So, as things are not accessed, you just start automatically paying less for them. And anything automatic in a cost savings world is going to help you save money. 

If you don't have to think about it and it just does it for you, it's fantastic. Well, Intelligent-Tiering added in these additional tiers—which they are Glacier—level tiers. They are additional places that your data can eventually move to as they start aging out based on a whole series of criteria. But there's caveats. There's more caveats now. 

Before, one of the interesting things that we actually learned as part of this—because it was buried in a pricing page footnote—is that when you store something into Intelligent-Tiering, there is a minimum storage time period that you will get charge...

Want to give your ears a break and read this as an article? You’re looking for this link.

Sponsors

• nOps
• Linode

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

Links

• Follow Last Week In AWS on Twitter

Transcript
Corey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.

Pete: Hello, and welcome to the AWS Morning Brief. I am Pete Cheslock. Corey is not here. He's never coming back. No, I'm just kidding, he's just not joining us for the Friday Morning Brief for a little while. Maybe we'll invite him back as a guest, but until then, I'm again joined by Jesse DeRose. Welcome back yet again, Jesse.

Jesse: Thank you so much for having me, I am so happy that Corey has not figured out that we just reset all of his passwords to ‘1234’ and locked him out of everything.

Pete: We did add an exclamation point to the end, and we made it very secure, but I do think it’s the—

Jesse: Very secure.

Pete: —it’s the ultimate troll to essentially take over Corey’s podcast for a period of time—while of course, he's taking care of his children—and essentially just inviting him back as a guest on it. So, I think that'll be fun. Maybe we'll have to do that: invite him back as a guest on his own podcast.

Jesse: I love it.

Pete: Well, we're here today to talk about, maybe, potentially contentious topic certifications. Are they good, or are they a bag of crap?

Jesse: This is a spicy one. I'm excited for this conversation.

Pete: So, certifications, this is a business that's more profitable for AWS than SimpleDB is.

Jesse: Nailed it.

Pete: Their whole certification ecosystem has really just blown up. I mean, I've been a part of the Amazon ecosystem since nearly the beginning, working for a startup back in 2009 timeframe; we were very early, and there was no certification, there was no re:Invent. I mean, all that stuff came after. And just looking now at the amount of certifications that exist, you've got, kind of, your default Cloud Practitioner level, you've got Solutions Architect Associate Level, Developer Level, you've got Professional Level, you can be a DevOps Engineer Professional. 

But then, more importantly, they even have these specific specialties in addition, so you can have an advanced networking specialty, or an ML or data analytics. It's really interesting how this has just exploded across the ecosystem, and having been to many re:Invents, they put a good amount of effort into certifying a lot of engineers at those events. But Amazon certifications are actually not the only thing we're talking about today. It's a big part of what we're talking about, but there's a lot of certifications out there. And for a lot of people, that's how they got into the industry. So, there's potentially a lot of good, but that's not always the case.

Jesse: Yeah. I honestly have a lot of mixed feelings on certifications. And honestly, there's strongly mixed feelings on certifications. So, what I really want to talk about today is, are they good? Or are they crap? Are they things that are ultimately beneficial for you to sit for and to take, or are they a waste of your time? And honestly, I think it all really boils down to which certification you're looking at and what do you want to do with it? What's the ultimate end goal for getting this certification? Because that can ultimately really influence whether or not this certification is going to be worth your time and money.

Pete: Exactly. I mean, what is the point of these to begin with? I mean, other than being just a great cash cow for some businesses?

Jesse: Yeah, I like to think about it like—I compare it to a college degree. I know it's not but I think about it in the same sense of like—

Pete: See, that's a very spicy comparison for some people who have paid lots of money for a college degree—much like myself—to compare it to a certification, but I like where you're going with this, so give it to me.

Jesse: I'm sorry for all the listeners who just dropped off and returned back to the latest episode of the Adventure Zone or Serial. I appreciate for those of you who are still with us to continue on. For me, a certification can provide a lot of similar opportunities for a college degree in terms of, it's a way to validate your knowledge. It's a way for you to prove, “Hey, I understand these ideas, these concepts,” that maybe you wouldn't be able to validate otherwise. And it validates your knowledge externally, and it gives you the opportunity to show a potential employer, “Hey, I have proven that I am familiar with these topics related to your business, and that is why you should hire me, or that is why you should consider giving me this promotion or giving me this opportunity.” It really gives a candidate an opportunity to derisk yourself. And I have proof. I have third-party-validated proof that I am familiar with these things.

Pete: Look, Jeff Bezos personally signed—actually I don't know if that's the case. It's probably Andy Jassy—personally signed my certification. So, it's like Andy Jassy is giving me this job recommendation, and Andy Jassy’s stamp of approval.

Jesse: “Do you want us to get Andy Jesse on the phone? Because we can get him on the phone right now, and he can confirm that he personally approved me for this role.”

Pete: Exactly. I mean—and I, of course, say that he stamped mine. So, interestingly enough, I do not have any Amazon certifications, but you do Jessie.

Jesse: I do. I have the Solutions Architect Associate certification.

Pete: So, I have at various points in the last couple of companies I've worked at have looked at getting an Amazon certification, and honestly, I have had the same kind of thought processes you just mentioned, which was, what will this give me? And will it give me for my time, and let's not say the money aspect because, for all these scenarios that I'm dealing with, the company was going to pay for it.

Jesse: Sure.

Pete: So, that was less of a risk, but it is my time. I don't want to take it and fail it, and have to take it again; that's just wasteful. So, I’d want to spend some time preparing and reviewing. But if I were to get—at this stage of my career, having been working with Amazon for a long time, if I were to get a Cloud Practitioner or Solutions Architect Associate, does this open up any doors for me? And to be honest, at my stage of my caree...

Want to give your ears a break and read this as an article? You’re looking for this link.

Sponsors

• nOps
• Linode

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

Links

• Last Week In AWS Twitter: https://twitter.com/lastweekinaws
• https://wellarchitectedlabs.com/

Transcript

Corey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.

Pete: Hello, and welcome to the AWS Morning Brief. I am Pete Cheslock. I'm still here. I'm going to be here for a while I guess, but not alone. I'm here with Jesse. Jesse, thank you again for coming on board and keeping me company.

Jesse: Always a pleasure.

Pete: It's honestly just nice to talk to someone else that's outside of my little family unit or my pandemic crew.

Jesse: I would say it's nice to get paid to just talk about my feelings. But I mean, I'm not technically getting paid for this.

Pete: Yeah, I feel like I'm just trying to balance the conversations with coworkers, podcasting this, my kids at this point, have more Zooms than I do.

Jesse: [laugh]. I think that probably says something about our social lives and about ourselves. And I feel like I need to go rethink everything.

Pete: Well, my son who is six years old, he does a better job of managing his mute button than most full-grown adults I know.

Jesse: I feel like that's the fun thing. I really want to see how the next generation is going to grow up with technology, better understanding the mute button, and all of this video content than we do.

Pete: It is hilarious to hear my daughter yelling at her friends, “You're on mute.” [laugh]. Oh, well, what is not on mute today is both of us. We are talking about the most loved Amazon service, Amazon QuickSight.

Jesse: I think it's technically going to be on blast today rather than on mutes.

Pete: Yeah, I think we're going to struggle to keep this one on time. So, if we go long, I apologize in advance. But we're talking about QuickSight, which for those that maybe have never heard of QuickSight before, it's Amazon's business intelligence tool. The question you're probably asking yourself, to be perfectly honest, is why? Why did you even try QuickSight? 

Like what point, what thing were you solving that made you think of QuickSight? So, we're going to tell that story. But first, let's just pivot into BI tools, business intelligence tools. That's the category that QuickSight is technically in. So, we'll talk a little about that, and also how we actually use BI tools within Duckbill because that'll give you, hopefully, the context into answering that question of, “Why did you even try QuickSight, Pete? Why?”

Jesse: I mean, I feel like there's probably still going to be people asking us why after this podcast, and I'm sorry for those listeners. We don't have an answer for you. Maybe we're just masochists. We don't know.

Pete: It's just because it's there, I think is what the final answer is. [laugh].

Jesse: Absolutely. So, business intelligence tools solve a whole variety of problems and we could probably do an entire episode on them in general. They help you gain insights from your data, which is fantastic. I absolutely love that this is even a category of service out there. But today specifically, to keep it on track, we want to specifically talk about gaining insights from your spend data, your AWS spend data. And to do that, we really need to start by talking about the AWS Cost and Usage Report.

Pete: Yeah, the Cost and Usage Report—you might hear it referred to as the CUR. I heard it referred to as the CUR often and it took me quite a while to actually figure out what anyone was talking about. So, if you hear someone say the CUR, they probably mean the Cost and Usage Report. But this is the v2, we'll call it, version of the Amazon billing data. 

It's incredibly high fidelity, I think is the term. It's very granular; there's a lot of data in there. And it's not enabled by default; you need to actually go turn it on. But what's awesome about this tool is it can provide you some really deep insight into where your money is going, and the only cost for it is the cost to store the data. And the Cost and Usage Report itself, when you turn this report on and have it dumped into your S3 bucket location of choice, you can actually have it store into a couple different file formats. 

One of them is Excel CSV format. And the other one is a Parquet format, which is a columnar data store and is a lot more efficient for this type of data. And it's the Parquet version of this that we use, we tell our clients—clients of Duckbill—to turn this on and turn it on with Parquet because then you can use tools like Athena to query your data and just leave it in S3 and run those ad hoc queries. So, Athena, though, which we're not talking about Athena, is challenging to use, in some cases—

Jesse: Yeah.

Pete: —you have to know SQL, which if you don't know SQL you're kind of in a bad spot. So, we use a BI tool, a very popular one called Tableau to query our data on Athena. So, Athena is kind of the engine, you could also obviously put your CUR data into an actual database. But largely, the queries we're doing, these are all human-generated. We're fine if they take seconds; they don't need to happen in milliseconds.

Jesse: Yeah, I mean, there's lots of solutions out there. There's third party commercial apps like Tableau and Looker—RIP—there's open-source options like Metabase. But of course then, in true AWS fashion, there's also a hastily integrated acquisition called QuickSight.

Pete: So, I have this memory in my head—and hopefully someone will correct me if they're listening to it, and I'm wrong here—but I feel like QuickSight was actually an acquisition. Like Amazon, which really doesn't usually acquire a lot of teams or businesses into Amazon Web Services, with like a couple of pretty rare exceptions, I'm almost positive, that QuickSight was actually some other product that Amazon acquired into it. But the history of QuickSight from at least the Amazon umbrella started around 2015 is when they announced it at re:Invent, and I was there for that announcement. I remember that announcement clearly, and I still actually kind of laugh at it when it came out. Now, first off, that was 2015 is when it was announced, and not for nothing, it does not look like it has gotten much better in the five years that it's been operating since launch.