Sign up to save your podcasts
Or
Share Episode #46: Serverless Use Cases with Gareth McCumskey (Part 2)
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Episode #46: Serverless Use Cases with Gareth McCumskey (Part 2)
About Gareth McCumskey:
Gareth McCumskey is a web developer with over 15 years of experience working in different environments and with many different technologies including internal tools development, consumer focused web applications, high volume RESTful API's and integration platforms to communicate with many 10's of differing API's from SOAP web services to email-as-an-api pseudo-web services. Gareth is currently a Solutions Architect at Serverless Inc, where he helps serverless customers planning on building solutions using the Serverless framework as well as Developer advocacy for new developers discovering serverless application development.
- Twitter: @garethmcc
- LinkedIn: linkedin.com/in/garethmcc
- Portfolio: gareth.mccumskey.com
- Blog Posts: serverless.com/author/garethmccumskey/
Watch this episode on YouTube: https://youtu.be/5NXi-6SmZsU
Transcript:
Jeremy: One of the things that I know I've seen quite a bit of is people using just the power of Lambda compute, to do things right? And what's really cool about Lambda is Lambda has a single concurrency model, meaning that every time a Lambda function spins up, it will only handle a request from one user. If that request ends, it reuses warm containers and things like that. But, if you have a thousand concurrent users, it spins up a thousand concurrent containers. But you can use that not just to process requests from let's say frontend WebSocket or something like that. You can use that to actually run just parallel processing or parallel compute.
Gareth: Yeah. This is one of what do they call it, the Lambda supercomputer.
Jeremy: Right.
Gareth: You can get an enormous amount of parallel... Try to say that three times quickly. Parallelization with Lambda. I mean, like I said, by default you get a 1000 Lambda functions that you can spin up simultaneously. And if you ask nicely... Well, you don't even have to ask nicely, just ask them and AWS will increase that to 10,000 simultaneous. And it's really impressive how much compute you can do, to the point where, at one point I was working with a company looking to try to do some load testing of an application.
They had an instance where, on Black Friday, the tech kept falling over. They wanted to try to get some load testing in beforehand to make sure that it can handle at least a certain amount of volume. Because you can never entirely predict what your traffic patterns will look like. But at least let's try something. And they spend a lot of time looking at commercial solutions out there because there are a few of them out there that try to help with that.
And they normally try to do about 500 to maybe a 1000 simultaneous users or simulated users, which is impressive but not quite good enough when you're an organization that's going to be having 10,000 to 20,000 simultaneous users on your site at a time. That gets a bit rough. So the move was then to try and build some load testing application ourselves. And this was initially tricky to do because we were trying to do this using the traditional VMs, virtual machines, and containers in some way, try to get EC2 instances up and running to try and run multiple simultaneous users at a time, in a single VM, using essentially a combination of these end to end testing tools where you can simulate a user flow from loading the homepage to going to a product page, adding to cart, going to checkout. Doing all of this on a staging environment so that you could simulate the whole user for all the way to purchase, the sort of main line to purchase as it were, make sure that you could get a few thousand users all the way to there without issue.
And what ended up happening was these virtual machines just couldn't cope with the load of all these simultaneous users running on a single machine even with inordinate amounts of CPU and RAM on them. So the idea came to us to try and do this with Lambda instead. So what ends up happening is, because you have a thousand simultaneous Lambda functions, AWS also architects this in a way that the noisy neighbor effect of all of these Lambda functions is almost nothing. You can't say nothing.
There has been some research I've read that shows there is a bit of a noisy neighbor effect between Lambda functions. But one interesting thing that we found was this is reduced when you increase the size of your Lambda functions to the maximum memory size, which is pretty cool. Because then uses an entire machine essentially or virtual machine as it were. So now you're limiting the effect of that noisy neighbor effect happening. Which means you can then also run 10 to 20 simultaneous users on that single member function with that enormous amount of size.
And if you have a thousand of those, well now you've got a thousand Lambda functions with 10 to 20 users per Lambda function, running an end to end test, pointed at a single staging environment. That's a pretty powerful bit of load testing you can perform there. And Lambda being as flexible as it is, we needed to import a binary to execute the end to end testing framework that we were using.
So you can use Lambda asynchronously to help you spin up the required binaries, import all of these items in and then synchronize the start of the tests through SNS, for example, which can just fan out the go command to all of these Lambda functions waiting to execute. And that was it. We have 15 to 20,000 users, load testing and application. And that's going to tell you whether you're ready for Black Friday or not.
Jeremy: Right. Yeah, no, I think it's an awesome use case. And I mean the parallel load testing, I mean, just the amount that you can get. I mean even you try to run something on your local machine and you're trying to do just to simulate some things. You only have so many threads that you can open up, so many users you can simulate. And to do this reliably, I mean, some of these testing sites, you can go and get some of these... Use some different sites to do it.
But they get pretty expensive if you want to do regular tests. If you run a thousand concurrent Lambda functions, even at the maximum memory, and it takes maybe five minutes to run your full load test, you're talking about a couple of dollars every time that runs. Right? I mean, so the expense there is amazingly low. I think that's a super useful use case. There are some more specialty things though that you can do with Lambda.
And Lambda is very, very good at, or I should say Lambda is built to receive triggers, right? Serverless is an event driven system. So there are all kinds of triggers for Lambda functions. And I'm sure we probably could talk about a thousand different ways that you could trigger a Lambda function and do something. Everything from connecting to an SQS Queue to like you said, the DynamoDB streams and things like that.
But one of the interesting triggers for Lambda functions, is actually getting an email that is received from SES, which is the Simple Email Service that AWS has. and I find this actually to be really interesting. You did something interesting with that.
Gareth: Yeah. We worked with an organization who essentially they handled requests for medical insurance. So other companies would send this organization an email with information about users who needed medical insurance. And these email...
View all episodes
By Jeremy Daly & Rebecca Marshburn
5
2929 ratings
About Gareth McCumskey:
Gareth McCumskey is a web developer with over 15 years of experience working in different environments and with many different technologies including internal tools development, consumer focused web applications, high volume RESTful API's and integration platforms to communicate with many 10's of differing API's from SOAP web services to email-as-an-api pseudo-web services. Gareth is currently a Solutions Architect at Serverless Inc, where he helps serverless customers planning on building solutions using the Serverless framework as well as Developer advocacy for new developers discovering serverless application development.
- Twitter: @garethmcc
- LinkedIn: linkedin.com/in/garethmcc
- Portfolio: gareth.mccumskey.com
- Blog Posts: serverless.com/author/garethmccumskey/
Watch this episode on YouTube: https://youtu.be/5NXi-6SmZsU
Transcript:
Jeremy: One of the things that I know I've seen quite a bit of is people using just the power of Lambda compute, to do things right? And what's really cool about Lambda is Lambda has a single concurrency model, meaning that every time a Lambda function spins up, it will only handle a request from one user. If that request ends, it reuses warm containers and things like that. But, if you have a thousand concurrent users, it spins up a thousand concurrent containers. But you can use that not just to process requests from let's say frontend WebSocket or something like that. You can use that to actually run just parallel processing or parallel compute.
Gareth: Yeah. This is one of what do they call it, the Lambda supercomputer.
Jeremy: Right.
Gareth: You can get an enormous amount of parallel... Try to say that three times quickly. Parallelization with Lambda. I mean, like I said, by default you get a 1000 Lambda functions that you can spin up simultaneously. And if you ask nicely... Well, you don't even have to ask nicely, just ask them and AWS will increase that to 10,000 simultaneous. And it's really impressive how much compute you can do, to the point where, at one point I was working with a company looking to try to do some load testing of an application.
They had an instance where, on Black Friday, the tech kept falling over. They wanted to try to get some load testing in beforehand to make sure that it can handle at least a certain amount of volume. Because you can never entirely predict what your traffic patterns will look like. But at least let's try something. And they spend a lot of time looking at commercial solutions out there because there are a few of them out there that try to help with that.
And they normally try to do about 500 to maybe a 1000 simultaneous users or simulated users, which is impressive but not quite good enough when you're an organization that's going to be having 10,000 to 20,000 simultaneous users on your site at a time. That gets a bit rough. So the move was then to try and build some load testing application ourselves. And this was initially tricky to do because we were trying to do this using the traditional VMs, virtual machines, and containers in some way, try to get EC2 instances up and running to try and run multiple simultaneous users at a time, in a single VM, using essentially a combination of these end to end testing tools where you can simulate a user flow from loading the homepage to going to a product page, adding to cart, going to checkout. Doing all of this on a staging environment so that you could simulate the whole user for all the way to purchase, the sort of main line to purchase as it were, make sure that you could get a few thousand users all the way to there without issue.
And what ended up happening was these virtual machines just couldn't cope with the load of all these simultaneous users running on a single machine even with inordinate amounts of CPU and RAM on them. So the idea came to us to try and do this with Lambda instead. So what ends up happening is, because you have a thousand simultaneous Lambda functions, AWS also architects this in a way that the noisy neighbor effect of all of these Lambda functions is almost nothing. You can't say nothing.
There has been some research I've read that shows there is a bit of a noisy neighbor effect between Lambda functions. But one interesting thing that we found was this is reduced when you increase the size of your Lambda functions to the maximum memory size, which is pretty cool. Because then uses an entire machine essentially or virtual machine as it were. So now you're limiting the effect of that noisy neighbor effect happening. Which means you can then also run 10 to 20 simultaneous users on that single member function with that enormous amount of size.
And if you have a thousand of those, well now you've got a thousand Lambda functions with 10 to 20 users per Lambda function, running an end to end test, pointed at a single staging environment. That's a pretty powerful bit of load testing you can perform there. And Lambda being as flexible as it is, we needed to import a binary to execute the end to end testing framework that we were using.
So you can use Lambda asynchronously to help you spin up the required binaries, import all of these items in and then synchronize the start of the tests through SNS, for example, which can just fan out the go command to all of these Lambda functions waiting to execute. And that was it. We have 15 to 20,000 users, load testing and application. And that's going to tell you whether you're ready for Black Friday or not.
Jeremy: Right. Yeah, no, I think it's an awesome use case. And I mean the parallel load testing, I mean, just the amount that you can get. I mean even you try to run something on your local machine and you're trying to do just to simulate some things. You only have so many threads that you can open up, so many users you can simulate. And to do this reliably, I mean, some of these testing sites, you can go and get some of these... Use some different sites to do it.
But they get pretty expensive if you want to do regular tests. If you run a thousand concurrent Lambda functions, even at the maximum memory, and it takes maybe five minutes to run your full load test, you're talking about a couple of dollars every time that runs. Right? I mean, so the expense there is amazingly low. I think that's a super useful use case. There are some more specialty things though that you can do with Lambda.
And Lambda is very, very good at, or I should say Lambda is built to receive triggers, right? Serverless is an event driven system. So there are all kinds of triggers for Lambda functions. And I'm sure we probably could talk about a thousand different ways that you could trigger a Lambda function and do something. Everything from connecting to an SQS Queue to like you said, the DynamoDB streams and things like that.
But one of the interesting triggers for Lambda functions, is actually getting an email that is received from SES, which is the Simple Email Service that AWS has. and I find this actually to be really interesting. You did something interesting with that.
Gareth: Yeah. We worked with an organization who essentially they handled requests for medical insurance. So other companies would send this organization an email with information about users who needed medical insurance. And these email...