Last Week In AWS Podcast

About Corey Quinn

Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.

Links

• Trend Micro
• ChaosSearch
• @QuinnyPig

Transcript

Corey: This episode is brought to you by Trend Micro Cloud One™. A security services platform for organizations building in the Cloud. I know you're thinking that that's a mouthful because it is, but what's easier to say? “I'm glad we have Trend Micro Cloud One™, a security services platform for organizations building in the Cloud,” or, “Hey, bad news. It's going to be a few more weeks. I kind of forgot about that security thing.” I thought so. Trend Micro Cloud One™ is an automated, flexible all-in-one solution that protects your workflows and containers with cloud-native security. Identify and resolve security issues earlier in the pipeline, and access your cloud environments sooner, with full visibility, so you can get back to what you do best, which is generally building great applications. Discover Trend Micro Cloud One™ a security services platform for organizations building in the Cloud. Whew. At trendmicro.com/screaming.

Corey: Welcome to the AWS Morning Brief. I'm Cloud Economist Corey Quinn, And this is the Whiteboard Confessional segment that has increasingly been taken over by my colleague, Pete Cheslock, as we tear through various cloud-based companies, public filings as they race to go public and inflict their profits, or in more cases, losses on the public markets. Pete, thanks for joining me.

Pete: I am super happy to be back again, and making my mother happy that I'm actually using that MBA that I spent all that time to get.

Corey: So, we could wind up talking just about how Palantir is awful in a variety of ways. My personal favorite was the letter that their CEO attached saying that effectively engineers were stupid and didn't see the big picture, which is a weird thing to say about a whole group of people you're actively trying to hire, but all right. Let's talk about their S-1 filing. This has been anticipated for a while. What do you think?

Pete: Well, Palantir has been around for a very long time. I think it's been around a lot longer than a lot of people realize. You know, early 2000s. It was technology built to tie data together and to be honest, I only know—I’ve ever heard of one company actually using Palantir—the technology—a commercial company. They were actually using it as a SIM—SIM, whatever you want to call it—Security Information Management System—

Corey: Event management or something like that. Yeah.

Pete: Exactly. And ironically enough, that company actually—that was using Palantir—replaced it with an Elasticsearch ELK stack, which I thought was fascinating. I know nothing about their software, but I was very fascinated to read the S-1 because there's been this mythology around it and you can hear so much about insiders at Palantir, employees selling their shares in this wide secondary market. So, I was very curious to see what we were going to find, and there are definitely some interesting bits within.

Corey: There certainly are. And it's strange because for a while Palantir was doing interesting things in the market. They were offering $20,000 referral bonuses to people who referred engineers in for certain roles, and you didn't have to be a Palantir employee to do it, which was fascinating. They've recently moved headquarters from Palo Alto over to Denver, Colorado, which… okay. They are claiming it's for this whole lofty mission. Let's not kid ourselves: it's a tax play. [laughs]. 

And there's also a whole bunch of interesting stuff buried in here. But yeah, in many ways, this is a legacy company in some respects. It's been around almost 20 years. And strangely, I don't know about you, but I don't know anyone who works for Palantir. I did a little digging in preparation for this episode, and it turns out, I actually kind of do, but they're very quiet about it. It's one of those things where people don't want to be called out for working at a company that is this particular flavor of controversy, and I can't say I blame them.

Pete: Yeah, I haven't looked through my LinkedIn to see if any of my connections have ever worked there. Granted, it's such a West Coast company that me out in the East Coast, be pretty rare to run into anyone out here who's kind of taken their time and done the Palantir. I have heard, again, the rumors that they've always paid very well, and—

Corey: They would kind of have to.

Pete: You know, in the Bay Area, you kind of have to. And competing for talent against other places who pay really well, like Netflix, and Uber, and all these other big companies that are out there. So, it's a big competition for the top talent.

Corey: Oh, yeah. And most of what they do is data analytics. They take in a whole bunch of data, and they crunch a whole bunch of numbers and come out with other stuff. Historically, they have been focused on selling their services to governments, but now they're expanding in the enterprise story as well. And that is, of course, going to be a bit of a challenge for them as they expand into it, but we can talk about what they do, how they do it, and all the other challenges. Let's talk about Cloud. What do we know about their cloud environment based upon their public filing?

Pete: Well, they talk about their commitments. So, this is something you often see in S-1s of their various cloud commitments, and I think this one was super interesting in that they listed commitments for about $1.5 billion in cloud commitments over six years, and this was an agreement they entered into at the end of last year. Just a massive, massive amount of cloud spend commitment, right?

Corey: Yeah, it’s a quarter billion dollars a year in spend. Which is, again, we see a number of customers in that range pretty frequently, it's not always typical to see the better part of a decade done to satisfy those commitments, though. Usually they're, “Well, this stuff is always changing. Let's talk about doing this for the next three years.” Six is a bit on the outside range of what we tend to see. 

What's fun to me was the breakdown of that commitment, which was just—I've been using this as a talking point for a week now—which is they have to undisclosed cloud companies in this part. They mention elsewhere that they use Azure and that they use AWS. Great. Fine. For one cloud provider, they have a six-year commitment of $1.49 ...

Links Mentioned

• Want to give your ears a break and read this as an article? You’re looking for this link: https://www.lastweekinaws.com/blog/dipping-my-toes-into-the-digitalocean 

Sponsors

• A Cloud Guru: https://acloudguru.com
• New Relic: https://newrelic.com

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

About Corey Quinn

Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.

Links

• Trend Micro Cloud One™
• ChaosSearch

Transcript

Corey: This episode is brought to you by Trend Micro Cloud One™. A security services platform for organizations building in the Cloud. I know you're thinking that that's a mouthful because it is, but what's easier to say? “I'm glad we have Trend Micro Cloud One™, a security services platform for organizations building in the Cloud,” or, “Hey, bad news. It's going to be a few more weeks. I kind of forgot about that security thing.” I thought so. Trend Micro Cloud One™ is an automated, flexible all-in-one solution that protects your workflows and containers with cloud-native security. Identify and resolve security issues earlier in the pipeline, and access your cloud environments sooner, with full visibility, so you can get back to what you do best, which is generally building great applications. Discover Trend Micro Cloud One™ a security services platform for organizations building in the Cloud. Whew. At trendmicro.com/screaming.

Corey: Welcome to the AWS Morning Brief: Whiteboard Confessional series, where I am joined once again by my colleague, Pete Cheslock. Pete, thanks for taking the time to tolerate my slings, arrows, and other various forms of cynicism.

Pete: You know, I didn't take that much offense to the fact that I have an MBA, so I decided to come back and see if we can make use of that investment.

Corey: So, fun story. The last one of these that we did was talking about—who's one was that? They're starting to run together at this point.

Pete: That was Sumo Logic’s.

Corey: That's right. And it was, “Oh, let’s talk about what they're doing.” And then throughout the day, I think five tech companies all filed to go public, which is just bizarre. So, we're going to take a couple more episodes to slice and dice a couple more that were of interest to us.

Pete: Yeah, absolutely. We're going to chat about one that I was honestly been waiting for because of the hype and the myths around this company. But it's a big data company called Snowflake.

Corey: They're very special and unique.

Pete: They're very special. I think—I often will listen to CNBC in the background, it's kind of interesting to get little words, and sometimes tech pops up into a CNBC broadcast. When Snowflake filed. I think one of the announcers had said something to the effect of, “I don't know why you'd want to be called Snowflake.” [laughs]. So, I had a good chuckle at that one.

Corey: Because they've been around longer then that's been a disparaging term used by jerks.

Pete: [laughs]. Exactly, exactly. So, they filed their S-1 in that flurry with a whole slew of other companies—which we will definitely get to at least one more of those—and honestly, this company, I've never worked there. I did at one point go through a sales process which I can share some of my thoughts and opinions there, but the reason why I was so excited to see this one is because of the sheer amount of VC money that this company has raised, well over—I don't know if ‘well over’ but definitely over a billion dollars of VC funding raised. It's crazy.

Corey: My comment at one of the big tech conferences last year—back when that was a thing we went to—was I was walking around their booth, and I noticed that they had this mock-up of a race car suspended in the air. And then I realized, “Oh, my God, that isn't a mock-up.” Which told me at that point that if you're paying retail pricing for Snowflake, you're probably doing something very wrong.

Pete: Yeah, absolutely. I think to dive into one of my favorite Snowflake stories, at a previous company, we were checking Snowflake out—we got connected with them via some connections our head of product had and some success that we heard that Snowflake had with helping, you know, a data warehouse. That's what it is: it's a data warehouse technology. If you're in the Amazon ecosystem, you might be using Redshift. Snowflake can do some of those things, it can do some other things.

Corey: Why would I use something like Snowflake instead of Redshift? I mean, for starters, naive approach as well, okay, this is in a different Amazon account, so at minimum, I'm going to be paying data transfer in and out on both sides. But again, we're talking data warehousing, the data transfer is usually something of a rounding error compared to all the extra cost goes into that.

Pete: And this is where I think a lot of their growth in the early days came from a lot of the deficiencies in Redshift. In technologies, in the investment that Amazon was doing there, Snowflake could do a lot of things just simply better. I think additionally, too, they were probably taking a lot of business from Oracle shops and things of that nature. But I do know a friend of mine at his company, they had a well over a million dollars a month in Redshift spend, and they actually moved over to Snowflake as a cost-savings initiative. It was significantly cheaper. 

But what’s, I think, so fascinating, when I heard that I was like, “Well, hold on a second. You know, Snowflake runs inside of Amazon.” So, I'm always curious of how that relationship exists with Amazon where you've got some account manager who's going to lose on some big spend of an Amazon customer by their Redshift spend going down dramatically, but then whoever the account manager for Snowflake must just be super excited by that because obviously their spend is going to go up.

Corey: Yeah, on some level, if you're running a data warehouse on top of AWS, from the high-level AWS perspective, well, is it spend that’s going to happen on your account, or is it spend that’s going to happen on Snowflakes account? It's not likely that you're going to be building everything on top of AWS, and then Snowflake is going to be running its stuff on another provider. The data transfer charges there become exceedingly non-trivial.

Pete: Yeah, absolutely. One of the things that is interesting about how Snowflake works, at least from my recollection a few years ago, is that you can stream your data into S3 which is very cost-effective. Snowflake can actually ingest your data from S3, and what they basically do is they put it into their S3. And you pay the same S3 pricing. I remember the sales guy. 

He w...

Links Mentioned

• Want to give your ears a break and read this as an article? You’re looking for this link: https://www.lastweekinaws.com/blog/8-aws-terms-project-managers-need-to-know/ 

Sponsors

• A Cloud Guru: https://acloudguru.com
• New Relic: https://newrelic.com

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

About Corey Quinn

Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.

Links

• Trend Micro Cloud One™
• ChaosSearch

Transcript

Corey: This episode is brought to you by Trend Micro Cloud One™. A security services platform for organizations building in the Cloud. I know you're thinking that that's a mouthful because it is, but what's easier to say? “I'm glad we have Trend Micro Cloud One™, a security services platform for organizations building in the Cloud,” or, “Hey, bad news. It's going to be a few more weeks. I kind of forgot about that security thing.” I thought so. Trend Micro Cloud One™ is an automated, flexible all-in-one solution that protects your workflows and containers with cloud-native security. Identify and resolve security issues earlier in the pipeline, and access your cloud environments sooner, with full visibility, so you can get back to what you do best, which is generally building great applications. Discover Trend Micro Cloud One™ a security services platform for organizations building in the Cloud. Whew. At trendmicro.com/screaming.

Corey: Welcome to the AWS Morning Brief, what is normally the Whiteboard Confessional slot, but lately, I had such a good time speaking last week with my colleague Pete Cheslock that we're back again today. Say hello, Pete.

Pete: Hello.

Corey: So, as of the day we are recording this, earlier in the week, the Sumo Logic S-1 has been released, which means that Sumo Logic—motto, “We do logs, too.”—also is going public, which seems to be a bit of a flurry lately of companies deciding to, well, to be uncharitable, inflict themselves on the public markets.

Pete: Yeah, it turns out when you take venture capital money, eventually those venture capitalists, they would like to see a return. So, kind of make sense in a little ways, but at the same time, it's just, I guess, another location to raise money.

Corey: One of the problems that I've run into across the monitoring space, as these companies go public is—let's ignore the fact that it seems like none of them seem to be making money in a profitable basis. I mean, I haven't looked at the details yet, but Sumo is losing money, correct?

Pete: Oh, yeah. Yeah, absolutely. Although let's be really honest, that's not really a dig at Sumo. I mean, they all lose money. [laughs].

Corey: And to be fair, they also raised only—quote-unquote, “only”—$340 million while they were private. But there's a strange inflection here around how monitoring companies seem to work in this space. I don't know who sponsors any given episode of this show until after I've already recorded it, so I'm really hoping it's not them, but if it is, our goal is to be authentic. And it seems to me that there's very little differentiation in all of these companies that offer log analysis, for the most part. I mean, ChaosSearch, where you used to work, had something actually innovative in this space where the data lives in S3 and you can query it without having to pay the same extortionate rates that everything else did. But by and large, most of the rest of the players in this space, it seems the differentiator is starting to be marketing. Am I missing something stupendous?

Pete: No, I think you're spot on there, and you can normally see it when you look at a company's S-1. So, that S-1 includes a lot of information within there, but some of the key points are—at least that I kind of look at—are some of their financial statements; I'm just curious what their revenue is, what it costs to bring in that revenue, profit and everything else. But these companies, they break out their operating expenditures across things like research and development, sales and marketing, and for a lot of these marketing companies, you'll find their spend in sales and marketing to be just huge. In many ways, their spend is nearly their revenue. And let's not forget you still have engineers and your Amazon bill that you have to pay for as well. So, they seem to be very marketing-centric because it's a knife fight out there in the monitoring space, monitoring and logging. It seems like every day, there’s a new logging and monitoring company popping up with just a different way of doing things.

Corey: I get that it's a hard space and these problems are incredibly challenging. The challenge that I run into though is, in many cases, I just want a centralized place where I can effectively look at the logs in real-time as events happen, and start looking for specific patterns with various filters, and that's about it. And it seems like that is a somewhat naive use case—which I get—but then every company out there is chasing Splunk in one form or another. Because Splunk was the first company that really did this right, and they charged the appropriately high ransom in order to make that happen, and then everyone else seemed to go through a generation of, “We’re like Splunk, only not horribly expensive.” And then it became increasingly complex and down this entire path to a point where now, I'm looking at any of these tools and it turns out I need to take a class before I'm able to use them effectively, to learn their own variants of SQL, or how to wind up pointing it at some esoteric data source I'd forgotten.

Pete: Yeah, I think—and I've actually had a bunch of conversations with—as you would expect from spending some time at a logging data analytics company—but there's almost like multiple waves of logging that has happened. And Splunk was kind of the first in many ways. They created a revolutionary way of storing data. That was what they built. That was the core technology way earlier than a lot of other people were dealing with this problem. 

They also focused a lot in the SIM/SIEM—that's security, information, event management. So, they sold in a lot of ways to these security companies. And then you had companies that started to pop up that were in the more of the monitoring space, like the Datadog and the New Relics of the world. Datadog and New Relic were getting the requests, “Well, we want logging, too. Like, we're paying for this.” And so then they started consolidating on logging. 

And then you had kind of the next generation was like, well, it costs too much money to use these hosted vendors, and the reason it costs so much is because they're using these open source technologies to store this log data, so there's no real innovation there, and this next wave of logging companies that exist out there are all like the, “Well, what if you didn't index your data? What if you just tagged it really, really well?” And that's this third wave we're into now, wher...

Links Mentioned

• Want to give your ears a break and read this as an article? You’re looking for this link https://www.lastweekinaws.com/blog/everything-you-need-to-know-about-route-53-resolver-query-logging/

Sponsors

• A Cloud Guru: https://acloudguru.com
• New Relic: https://newrelic.com/

Sponsors

• New Relic: https://newrelic.com/

Never miss an episode

• Join the Last Week in AWS newsletter
• Subscribe wherever you get your podcasts

Help the show

• Leave a review
• Share your feedback
• Subscribe wherever you get your podcasts

What's Corey up to?

• Follow Corey on Twitter (@quinnypig)
• See our recent work at the Duckbill Group
• Apply to work with Corey and the Duckbill Group to help lower your AWS bill

About Corey Quinn

Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.

Links

• Dear Google Cloud, Your Deprecation Policy is Killing You 
• A Cloud Guru
• The Duckbill Group
• ChaosSearch

Transcript

Corey: Normally, I like to snark about the various sponsors that sponsor these episodes, but I'm faced with a bit of a challenge because this episode is sponsored in part by A Cloud Guru. They're the company that's sort of famous for teaching the world to cloud, and it's very, very hard to come up with anything meaningfully insulting about them. So, I'm not really going to try. They've recently improved their platform significantly, and it brings both the benefits of A Cloud Guru that we all know and love as well as the recently acquired Linux Academy together. That means that there's now an effective, hands-on, and comprehensive skills development platform for AWS, Azure, Google Cloud, and beyond. Yes, ‘and beyond’ is doing a lot of heavy lifting right there in that sentence. They have a bunch of new courses and labs that are available. For my purposes, they have a terrific learn by doing experience that you absolutely want to take a look at and they also have business offerings as well under ACG for Business. Check them out. Visit acloudguru.com to learn more. Tell them Corey sent you and wait for them to instinctively flinch. That's acloudguru.com.

Corey: Welcome to the AWS Morning Brief. In lieu of the Whiteboard Confessional’s traditional approach today, I want to talk about a different kind of whiteboard issue. Specifically the whiteboard interview you wind up taking at Google, which is just a giant red herring because the real question is “How well did you erase the whiteboard afterwards?” so it aligns with their turning stuff off that people love policy. I'm joined this week by my colleague, Pete Cheslock from The Duckbill Group. Welcome, Pete.

Pete: Hello.

Corey: So, we're talking today about Steve Yegge’s article that went around the internet three times over the weekend, and it was titled “Dear Google Cloud, Your Deprecation Policy is Killing You.” Normally, you would think that that would be some form of clickbait headline, but it's not. It was a massive 23-minute long read, as per Medium. And we will, of course, throw a link to this in the [show notes]. But, Pete, what was your take on this thing?

Pete: Well, I missed it on the first go around, but when you sent me over the link, and the first thing I saw was Medium saying, a 23 minute read, and you had told me how this post had blown up. I think that really speaks for how incredibly well written this post is about this particular issue, that people in this world are willing to invest 23 minutes to read it. I was locked into it the whole time. It held my attention the whole time because of just how deep it went into Google and just how they operate.

Corey: Steve Yegge is famous for doing the platforms rant back in 2012 or so. He's a former Amazon employee who I think spent something like seven years at Amazon, about an equal time at Google, left to go run architecture at Grab a couple of years back, and then, due to these unprecedented times, is now independent/doing his own thing right now. So, that is an absolutely fascinating trail because when he writes about this stuff, he knows what he's talking about. This isn't one of those, “Eh, I’m just going to go ahead and pen something that's poorly articulated, and see what happens.” What's more amazing is I haven't seen much in the way of pushback on this. The points that he hits in this article are pretty damning, and even people from Google are chiming in with, “Yeah, that tracks.”

Pete: Yeah, and for all those listening that maybe haven't read this yet, maybe going to go read it after listening to this. What the real, I guess, crux of this post is about is how Google aggressively deprecates things and the kind of culture within Google that really drives that world to happen, and how just opposite it is to a company like Amazon. I think my biggest takeaway from this was this light bulb, “Oh my goodness, it all makes sense now,” idea of how aggressively Google deprecates things has to do with code cleanliness. They don't like five different APIs to do the same thing, so they'll deprecate four of them and keep things clean and whatever. And what I think is really interesting, too, that I read in here is how internally, this works great for Google Because they have all these tools that can automatically update code, and update APIs, and let people know if a deprecation is happening. But he compares this to, like, Java and Emacs, which historically, take decades—if ever—fully removing APIs. It was a really fascinating read.

Corey: It really was and one thing that stuck with me was, it makes perfect sense in hindsight. If you are Google and can dictate how all of your employees write software that makes it into production and have automated tooling to go back and handle deprecations for you, then great, that does work. The problem is, is that the rest of the world is not like your internal engineers. The problem I see behind Google Cloud, by and large, is that it assumes that everyone tends to write software the way that Google engineers would. That's not a valid assumption, I assure you. I write software that is nothing like anyone who calls themself a software engineer would ever write, but your cloud offering has to support my nonsense.

Pete: Right. Compare this to Amazon. So, that's one of the biggest other takeaways that really hit me. And this came up when I think I was looking at a cloud bill and seeing SimpleDB still on it. SimpleDB, which they don't really market it, they don't tell you to use it, it's not part of design things. Although, Corey, you can correct me if I'm wrong there, if it's included, if it's really talked about much anymore, I don't think it is—

Corey: No, they've tried to bury it, but they are still hiring for that team from time to time.

Pete: That's—yeah, I remember you had mentioned that. And so think about that. Think about the m1.medium, right. I think m1.medium was the first instance type back in 2006—

Could you imagine if Amazon deprecated some...