Serverless Chats

About Nica Fee

Nica Fee is a Serverless Developer Advocate for New Relic. She's worked with and written about serverless for the last two and a half years. She recently spoke at Deserted Island DevOps, which you might know as the tech conference that happened in Animal Crossing. She writes regularly for The New Stack.

• Twitter: twitter.com/ServerlessMom
• Twitch: twitch.tv/noctnica
• TikTok: tiktok.com/@serverless_mom

Watch this video on YouTube: https://youtu.be/yM4q0NSFz0M

About New Relic
New Relic One is an observability platform built to help engineers create more perfect software. From monoliths to serverless, you can instrument everything, then analyze, troubleshoot, and optimize your entire software stack. All from one place.

• Website: newrelic.com/
• LinkedIn: linkedin.com/company/new-relic-inc-
• Twitter: twitter.com/NewRelic
• Telemetry Data:  newrelic.com/platform/telemetry-data-platform
• Blog: blog.newrelic.com/

Transcript

Jeremy: Hi, everyone. I'm Jeremy Daly and this is Serverless Chats. Today, I'm speaking with Nica Fee. Hey, Nica. Thanks for joining me.

Nica: Hey. Thanks so much for having me, Jeremy. Longtime fan, so it's great to be on.

Jeremy: Well, thank you. So you are a developer advocate at New Relic, and I'd love it if you could tell the listeners a bit about your background and what led you to New Relic and sort of what's new with New Relic.

Nica: Sure, yeah. That's great. I was actually at New Relic back in the day. I was at New Relic as a support engineer until about 2015 I believe, and left to go and become a full-time developer and full-time coder. And my path took me back sort of... As I was sort of coding full-time and just clearing queues and writing features and fixing bugs, I really started to miss some of the community building that I'd done previously. Especially actually when I was at New Relic back in the day, I was one of the people who was starting meetups and doing that kind of community building. And so I started trying to pursue that as a job which is how I got into dev advocacy. Dev advocacy, you get to tinker and you get to play and build stuff, and you also get to try to get other people excited about it and try to show it to people.

So I was doing that for Stackery, which is a serverless deployment tool, for two years, and had some success there and built some skills and really enjoyed it, and that's where I got kind of very into AWS and cloud engineering. So yeah. Now I'm back at New Relic, and it's such an interesting time to be at New Relic, and be looking at how we can go and talk to developers. Something that is interesting about being here is that everybody is talking about the time I was last here. Everybody's talking about, "Hey. There was a time when New Relic was something that lone engineers would install on one server and something would go down." They'd be like, "Well, I can see right here what the problem is." And then some exec was saying, "Hey. Let's go use this tool. It sounds great."

And right now, the question is, can we get back there? Can we get back to the place where it's a tool that developers love and that they're the ones saying, "Hey. We got to use this," rather than... As are so many developer tools being something where most people know it from the CTO coming in and being like, "We're using this tool. I met this guy on the golf course. He's told me great things about it. It's got a great spec sheet. We're using it. Everybody's going to use it now," right? So the question is, can we get back there to being in that space? So that's sort of what I'm doing New Relic because I'm trying to go talk to actual engineers about what it does and how it can help them.

Jeremy: Awesome. Well, I mean, one thing about New Relic is that they just released the New Relic One platform. I want to say the new, New Relic One platform, but it seems kind of hard to say new twice. But first of all-

Nica: We actually do that every year. We should do New Relic but then a starburst at the side. It's new this time.

Jeremy: It's even newer. Well, first of all, I want to thank New Relic because they are sponsoring this episode which is amazing, which again shows their incredible amount of support towards the community as well. So I do think that this is a great opportunity.

Nica: Can I give a quick shout out on that one, actually?

Jeremy: Absolutely.

Nica: As a dev advocate, I am actually really actively looking for stuff that is exciting in the community that we can help support. And so, obviously, you were very high on my list. I said, "Hey. We got to do this." But I don't see everyone. I don't know everything. So if you're listening to this and you have either an open source project on observability or you're doing community events or running a podcast that maybe is a little bit less famous than this guy, get in touch with me. Hit me up on Twitter. Show me your stuff. I would love to hear about it. My situation right now is I don't know enough people to support, not that I can't do that. So yeah. I want to hear from you all, if possible.

Jeremy: Oh, that's awesome. That's a great offer, and anyone listening, please take up that offer because I think it could be quite amazing. So anyway. So-

Nica: ...keep expensing GitHub sponsorships. But for the moment, that's just one that's just one like, "Well, just do it." And I'll just fight with AR after I get it done. And I'm sorry. Go ahead.

Jeremy: No, no. Not at all. I appreciate that. All right, so let's get back to New Relic for a second and this New Relic One platform. I do want to go through this because it is actually pretty cool. I mean, the entire thing has been completely rewritten. It's all new, right? Not all rewritten. I shouldn't say it that way, but-

Nica: Yeah. I had hopped into... So I came on five months ago now, and I got into New Relic. And I had kind of... I was excited. There was obviously tons of new stuff since I'd last been there five years, but I was a little confused. There was some stuff that looked the same as what I'd seen five years ago. There was other stuff that was like, "Oh, this is kind of a nice little interface." And there were things in this new interface which was sort of part of the site at the time. You could do stuff like every chart, you could go see what is the actual query that's building this chart. You could go and edit it. You could go and facet it, and you can make it more sophisticated, save it out. Oh, it was really neat. But that was only kind of part of the site. And it was like, "Hey. This doesn't feel 100% cohesive." And I'm like, "Maybe it's just me. Maybe I'm not trained or something."

But what's been happening and what's been released in the last few weeks is the whole site is the same very clean, cohesive experience now so that you can do stuff like if you're monitoring AWS Lam...

About Heitor Lessa

Heitor Lessa is a Principal Specialist Solutions Architect at Amazon Web Services. He has spent the last 10 years in a number of roles, focusing on networking, infrastructure, and development. Since joining AWS in 2013, he’s been helping organizations of all sizes and segments across EMEA to design cloud native applications as well as software development best practices.

• Twitter: twitter.com/heitor_lessa
• Email: [email protected]
• Serverless Application Lens Whitepaper: d1.awsstatic.com/whitepapers/architecture/AWS-Serverless-Applications-Lens.pdf

Watch this episode on YouTube: https://youtu.be/bFjT3TrpbZg

Transcript

Jeremy: Hi, everyone. I'm Jeremy Daly and this is Serverless Chats. Today I'm speaking with Heitor Lessa. Hey Heitor, thanks for joining me.

Heitor: Thanks for inviting me. It is a pleasure to be here.

Jeremy: I'm super excited to have you here. You are a principal specialist solutions architect at Amazon Web Services. Why don't you tell the listeners what you do at Amazon Web Services and sort of what a principal specialist solutions architect does.

Heitor: I know it's a long title. I guess we can just say I'm a solutions architect at AWS. My day to day is basically working with customers and enable developer teams to find the best solutions on how to either build something on AWS or migrate, let's say a microservices monolith to a microservices or optimize something that they have.

But more recently, I'm also working with customers to help them build developer communities' inside. Similar to what we have at Amazon which bring in pros and stuff like that.

Jeremy: Very cool. Now I know you're doing a million different things. I don't know how you're not running AWS yet. I think you're next in line, I think. But you're doing a million things there. And one of the things though that you've been working on in the past and I know you're still involved with it, is the Well-Architected Serverless Lens. And I want to get into this because this is one of those things where if you're trying to find best practices and you're reading all these blog posts and you're looking at anti-patterns and good patterns and all this kind of stuff, I think it gets really, really confusing.

And your team and a bunch of other people at AWS and a bunch of the community heroes and all kinds of people got together and put together this Serverless Lens. If people are familiar with the Well-Architected Framework, which is talked about quite a bit, there's also this thing called the Well-Architected Serverless Lens. What's the difference between those two things?

Heitor: Sure. So the Well-Architected started way back in 2016, even before that to be fairly honest, where customers were looking to use AWS but we have roughly 50 services back then. Compared to today we have a lot more. And basically those customers were asking, "How do I use X service versus the other service? How do I go to production with this critical application? How do I model from my on-premises applications to something more cloud native? How do I migrate?" And things like this. Or even specific questions like, "How do I set up a multi-account? How do I better protect my accounts from a security perspective or billing?"

Well-Architected brings all those best practices that are agnostic from a workload perspective that typically applies to many of them, whether using serverless or containers, so usually what would work really well. But the challenge of Well-Architected as the platform evolved, we started to have more high level services like serverless or some service like AI/ML, which you have to treat them slightly different. The best practices still apply on how you set up AWS accounts, how do you do backups, how do you think about relational databases versus NoSQL databases?

But when you get to things like Multi-AZ and EBS volumes for serverless, they don't quite make sense. The Lens was a project to say, what are the customers using that Well-Architected actually helped them but they still lack a lot of good practices that are very specific to the technology they chose? So serverless was one of them. IoT was also another one. And more recently, last month we also announced Analytics Lens. If you're interested in big data, AI, those pieces, Analytics covered that pretty well. That's the difference.

The Lens is a... It doesn't replace, Well-Architected, it's more as an add on to the all these best practices we've been sharing for the past few years.

Jeremy: Yeah, because as an add on, it makes sense. The original serverless or sorry, the original Well-Architected Framework has, I think, 47 questions or so that asked you about specific areas and there's the five pillars and we'll get into some of that because I do think it's interesting to think of it that way. But the Serverless Lens just has more questions. What's the reason for all those extra questions?

Heitor: Sure. Well-Architected, when we started the Lens, if I'm not mistaken, again, there was the 47 questions but now we had just a recent update where some of those questions might change now. But the Serverless Lens, I think, if I'm not mistaken, we started with 31 questions because we were trying to get every single detail of servers and every best practice. But that was primarily a academic paper. So Lens started as a let's set up a document where you can go and find out when do I use serverless, is serverless as a good thing for me? How do I choose between all these services? How do I know the operational best practices for serverless?

As we started digging into those best practices, we felt we needed a lot more questions to dive into, okay, what type of metrics do you need? What type of alarm do you need? When do you use containers versus Lambda functions? When do you use orchestration versus synchronous calls? We only started in 2017. We had all these questions that customers were asking us. We put together into a document and we started writing. That took us roughly six to 10 months to put together into 50 pages when we announced Serverless Lens.

Jeremy: Right. And then that was a white paper, like you said. That was just a document. But now that's been moved into the Well-Architected tool, which is pretty cool. If anyone's used that or hasn't, I suggest you go and try it out. But that just takes you through and asks you all those questions and you can kind of keep track of your progress. How did you go from the white paper to the tool?

Heitor: Yeah. In 2017, when we announced, Werner went on the stage and talked about this idea of getting those best practices for serverless. And in 2018, we got an immense amount of downloads for Lens. If I'm not mistaken, it was over 20,000 downloads in less than six months, specifically for serverless best practices.

But then those questions started to ask more. How about Alexa? How about X, Y, Z? So what we found was trying to keep writing those pieces into the document was pretty difficult to keep up with the serverless space as well and how much it changes. What we found was instead of keep adding more questions and more pages of documents, we came together and thought, what if we evolved the lens project into the console? The customer would go to the console and say, "I want to review my architec...

About Paul Johnston:

Paul Johnston is an interim CTO, CTO and strategist who has particular interests in serverless, cloud, startups and climate change. Formerly, Paul served as a Senior Developer Advocate at AWS for Serverless and CTO of multiple startups, including one of the world’s first serverless startups. Paul is also a co-founder of ServerlessDays.

• Twitter: twitter.com/PaulDJohnston
• Medium: medium.com/@PaulDJohnston
• Project Drawdown: drawdown.org/
• Roundabout Labs: roundaboutlabs.com/
• Leading Edge Forum: leadingedgeforum.com/
• White Paper: The State of Data Center Energy Use in 2018
• IPCC Special Report: Global Warming of 1.5 ºC
• Blog post: To fix Climate Change, stop being a techie and start being a human

Watch this episode on YouTube: https://youtu.be/DTpP7RGXV6g

Transcript

Jeremy: All right. We talked about the big three a little bit and compared them in terms of their green and stuff like that, but in that paper that you wrote, you have this cloud league table in there where you compare them. I'd love to know more, what about Alibaba and Oracle and IBM and some of these other things, where do they all stack up against one another?

Paul: They aren't as big. Let's just be clear on that one. They aren't as big, and their green credentials are less clear. For example, Alibaba is very big in China for obvious reasons, it's a Chinese business and yes, they have a footprint outside of China, but they're a primarily Chinese business. When we looked and researched and were trying to find out about all of their green credentials, we found very little information whatsoever. It was almost non-existent. We found a little bit about efficiency in data centers and putting things in cold regions of China. You're like, "Well, that doesn't actually change anything if you're growing at a massive rate." It changes the conversation a little bit.

Jeremy: Can you do that though? Can you pack your servers in snow? Does that help with the cooling bill?

Paul: It depends on the server, I suppose. But I'd say you end up with this, not every conversation is equal. This is shown across the political spectrum as well. The conversation in China is very different to the conversation outside, in terms of industrial nations. Industrialized nations such as the U.S., UK, Australia, and Europe as well. You have very different social context. But anyway, coming back to Alibaba, we just found very little information. IBM and Oracle, actually, both had lots of information. IBM has had a commitment to renewables and sustainability for a very long time. I think since the '70s if I remember right. They have good credentials, but they don't offset their renewables in data centers or regions.

Paul: I think I remember, from the top of my head, because I can't remember everything, they are renewable in the UK. I think Oracle are renewable in the UK anyway. But some of these regions, they are renewable, but not all of them. But it's not clear unless you dig into the paper and unless you dig into their information. Nobody, as far as I can tell, has a little green dot against a region that says, "This one's renewable." It would be so much easier if they did. These other organizations, none of them, apart from Google and Microsoft have really made a play for being the green advocates in the space. Amazon does, but that's a whole other conversation which I'm not going to go into at this point. But the three lower down, I think, are struggling to be able to play in the same conversation. I think they would like to be seen as green, but I don't think they are really pushing the agenda because they don't see it as a point of differentiation.

Jeremy: All right. Then what does the tech industry have to do as a whole? I know you had some recommendations in your paper about this, but just what are maybe the top two or three things that the tech industry as a whole could do to address the climate crisis?

Paul: The climate crisis as a whole.

Jeremy: Or I guess their impact on it anyways. Let's start with that, we could build from there.

Paul: I don't know anymore. I think I've gone backwards and forwards on-

Jeremy: Don't give up Paul, don't give up.

Paul: It's not that. It's just there are so many things. I think my biggest thing is the tech industry needs to find its activist voice. I think that would be my point. I think sitting there and going, "Oh, everything's going to get fixed by technology," is entirely the wrong approach. I think my personal view, as much as I like Tesla's technology, I don't think Tesla is going to save the world. I'm not an Elon Musk fan, I find him very difficult in a number of different ways. That's as much as I'm going to say, but I find-

Jeremy: I don't think he listens to this podcast, so don't worry about it.

Paul: That's fine. But I find a lot of people within tech look at techno utopianism, and let's call it that because I think that's a pretty simple way of it. That technology will save us. The more that I look and the more that I look at what is happening in the world and the speed the technology is evolving and what we need to do in the speed that we need to do it in terms of climate change, I don't think we're going to get anywhere near fast enough technology evolution. We have to do something else. We can't just expect technology to catch up, fix it, and just for us to carry on.

I think technology needs to learn to find its activist voice. I think we need to be activists against those organizations who are not doing enough, who say they're green and are not, and in this, Amazon, I will call this out. Much as I love their serverless technologies and the people who work there are brilliant, the wider organization I think is not doing a good enough job in terms of its green credentials. That hasn't been good enough from my point of view. I want them to do better. Because actually, I think a green Amazon would be a great thing for the world and I think they can do it. I have seen Amazon and what it does when it's amazing, and I think if they turn themselves around and actually did the green thing properly, then I think that the hope for the future would be significantly higher.

That is why I want Amazon to change is because I think they have the power to be a force for good, and I think they're not doing that at the moment enough. That's one. I think find the activist, find the place that you within tech want to change and go and change it. Because I don't think we have anywhere near as much time as people think. I think your career in tech in 10 years time will not look like the career in tech that you think it is now. We are in a completely changing environment. Depending on what happens in the U.S. in the next few years, the world is changing around us. We are in an inflection point that I don't think many people are aware of. This is just...

About Paul Johnston

Paul Johnston is an interim CTO, CTO and strategist who has particular interests in serverless, cloud, startups and climate change. Formerly, Paul served as a Senior Developer Advocate at AWS for Serverless and CTO of multiple startups, including one of the world’s first serverless startups. Paul is also a co-founder of ServerlessDays.

• Twitter: twitter.com/PaulDJohnston
• Medium: medium.com/@PaulDJohnston
• Project Drawdown: drawdown.org/
• Roundabout Labs: roundaboutlabs.com/
• Leading Edge Forum: leadingedgeforum.com/
• White Paper: The State of Data Center Energy Use in 2018
• IPCC Special Report: Global Warming of 1.5 ºC
• Blog post: To fix Climate Change, stop being a techie and start being a human

 Watch this episode on YouTube: https://youtu.be/SI2-WU_0zgs

Transcript

Jeremy: Hi everyone. I'm Jeremy Daly and this is Serverless Chats. Today I'm speaking with Paul Johnston. Hey Paul, thanks for joining me.

Paul: Thank you very much for having me.

Jeremy: So you are a consultant through Roundabout Labs and a research associate at the Leading Edge Forum. So why don't you tell the listeners a bit about your background and what you have been up to lately?

Paul: So, yeah, background's a bit confusing. It's always a little bit strange. I don't have this whole 14 years at any one big company or anything like that. I spent many years in tech, 20 years, working with various different startups from my own business, that kind of thing. Then I worked in a startup in 2015 that effectively started using AWS Lambda.

So this is where the serverless comes in. And I was one of the first companies to start using Lambda in any kind of scale in a startup as a kind of first principle. And then I went from there to using it in that startup in 20 countries. Went a bit mad, a tiny, tiny budget from AWS. And I was like, "Well, this kind of worked so I'm going to keep doing it," and started telling people. AWS took notice, gave me a job, that was quite fun. Was a senior developer advocate for serverless at AWS for a while.

And then didn't stay there all that long, but it was really enjoyable while I was there. And moved away from there to go and do some consulting, which I've done since 2018. 2018? 2018. And then from there...

Jeremy: What year is it again?

Paul: Honestly, this year has gone on for a very long time.

Jeremy: Right, right.

Paul: And since then I have done some consulting in various different projects, tech projects. But one of the things I've done is worked on working out how tech and climate work and how they intersect. And one of the projects I've been working on is a research project for the Leading Edge Forum, which if any of you know Simon Wardley, that's the organization that he works for.

And I've been working on a project to look at how climate change is going to affect business over the next 10 years from a tech angle very much, so from a data and a tech angle. And just trying to see what lessons we can learn and what things are going to be coming up in the future. So kind of many and varied, shall we say?

Jeremy: All right. Well, listen, I have been wanting to get you on the show for a very long time, because I think this whole climate change thing is hugely important. I have two young daughters. I think about their future. I think about the junk we pour onto the earth, the pollution, the amount of carbon dioxide we're creating.

And one of the things that I think really attracted me to serverless in the beginning was not just, you know, obviously not having to manage servers, which is great. But this idea that maybe by sharing tenancy on a big server and only using the compute that we needed to, I was thinking in the back of my head, I'm like, "Well, maybe that reduces the amount of energy we use."

And so I know you have dug into this tremendously, and I mean, you're an expert on this stuff. And so I'd love to go through all these things with you, just get your insights, get some thoughts on this stuff. We can talk about serverless and some details of serverless as well. I'm sure that's what the listeners want to hear, but I love this idea of going green with serverless. Because I think it's hugely important. I think it's a step in the right direction.

But maybe we could start and just, or start by saying, how does serverless technology compare to traditional technology or traditional servers when it comes to green computing?

Paul: So it's a very, very good question. It's almost impossible to answer in some ways, but it's really, really easy to answer in others. So one of the things you want to look at, first place you want to start is, well, effectively you want a definition of what serverless computing is.

Paul: So let's just kind of take function as a service is kind of the base enabling technology, shall we say, for most serverless computing. Because I think most people will kind of see serverless and they'll go, "Right. What does that mean?" And so you want to drop it down to something that is kind of tangible.

So you want to talk about function as a service really as being the base enabler, because serverless for me is about business value and getting as much as possible out of your technology in terms of applications and all of those elements.

And so I think when you talk about function as a service, what you get is you get a pay for what you use. So you know that you are using as little electricity for your application as you possibly can. And so what you're trying to do is go, "Well, I want to be as green as possible." So being as green as possible means actually reducing as much of your usage as possible.

That's essentially what we mean by being green is actually reducing and actually using as little, as close to zero, in terms of compute as we possibly can. So what does that mean? How do you do that? Well, in terms of building an application, don't build the application to start. Just don't build the application at all if you possibly can. If you can build it on the basis of lots and lots of caching or not running any servers at all, then great, do that.

If you can do it on the basis of only running compute when you absolutely have to, then great. If your application can scale down to zero and it literally can, nothing is running if nobody's using it, that again is... That's the kind of thinking that goes into being green and being serverless, which is why serverless is something that for me works really well alongside an environmental conversation. Because it's not just about what does the techno- how does the technology work?

It's actually, well, this approach allows me to say, well, it gives me business value. It gives me environmental value. And actually when you come down to it, it just works out as better common... It's more common sense when you actually try. And w...

About Erica Windisch

Erica Windisch was the co-founder and CTO of IOpipe where she helped organizations maximize the benefits of serverless applications by minimizing debug time and providing correlation between business intelligence and application metrics. She is now a Principal Engineer at New Relic.

As an advocate and pioneer of cloud computing since 2001, Erica is always pushing forward as technology and the industry adapt. She was an early contributor to OpenStack and maintainer of the Docker project where she worked on hardening Linux containers and establishing corporate and community security policies.

Erica is a champion of AWS Lambda and serverless technologies, and she speaks frequently at conferences about AWS Lambda and other AWS solutions. She's passionate about systems architecture, security, and the future she sees for machine-automated, low-code development.

• Twitter: twitter.com/ewindisch
• New Relic: newrelic.com
• Personal email:  [email protected]
• Professional email: [email protected]

Watch this episode on YouTube: https://youtu.be/T1t_P_zqOiE

Transcript

Jeremy: Hi, everyone. I'm Jeremy Daly and this Serverless Chats. Today I'm joined by Erica Windisch. Hey, Erica, thanks for joining me.

Erica: Hello. Hi. Thanks for having me. Or thank you for having me.

Jeremy: So you are a principal engineer and architect at New Relic. And you're also an AWS Serverless Hero. So I'd love it if you could tell the listeners a bit about your background and what you've been doing at New Relic.

Erica: Oh, gosh. Okay, well, my background is pretty deep. So, I'm at New Relic now. Before New Relic, I was the founder and CTO of IOpipe, which was an observability product for serverless applications. Now, I am working as an architect and principal engineer for New Relic. And if we're going to rewind history a little bit. I previously was a security engineer working at Docker, where I founded their security team and their security processes. I was involved in OpenStack from very early, since its founding. And then before that I had actually had my first company and we had built a cloud. We actually had our own cloud services. We were building from 2003 actually, building out horizontally scalable cloud services. And I said, "Well." We bought really early into that pets versus cattle idea.

Jeremy: Nice, nice. Well, so obviously you're doing a lot with observability. And you're doing that in New Relic, that's sort of what New Relic does. IOpipe was all about that. I know a lot of the team has gone over from IOpipe to New Relic, to continue to work and expand their services. And I'd love to talk to you about that today. We've done a number of shows where we've talked about observability. But that was probably almost a year ago at this point. And I'd love to get a sense of sort of, where things have gone, where things are going. You know maybe what the future is going to look like. I got a bunch of other things I want to talk to you about. But maybe you could just start, just in case listeners don't know, what do we mean by observability?

Erica: Oh, gosh. The way I see it is, being able to really see what's happening in your applications, in your infrastructure and doing that... I would say early monitoring. Things like Nagios was, I would not consider observability that was monitoring. It was very much, very reactive. There was zero product... It was not productive at all using something like Nagios. Logging products give you some ability to start getting into, being able to be proactive. And I think that observability kind of ties in some of the concepts from logging, and ties it in with your metrics and ties in being highly correlated. And also deeper into your application, having traces in your application, having contacts for your applications. For instance, just having a trace and knowing that, say an API gateway triggered a Lambda is one piece of information that you can have, but knowing say, the resource path, the HTTP method, things like that. That's a deeper set of insight that I think is necessary. And definitely fits within an observability picture that is very much say different and distinct from something like Nagios. Or even just plain text logs.

Jeremy: Right. Yeah. And we've talked about on the show the three pillars, right? You've got monitoring, tracing and logging. And so monitoring, like you said, is that sort of general like just something goes wrong, maybe you get an alert, something like that. The logging bit is obviously logging data. But let's get more into tracing a little bit. What do we mean by tracing?

Erica: Sure. The way I look at tracing as being able to see the relationship between various components, and not just the components. And I think this also where maybe tracing generally... in our industry historically, has been this service talks to this or that service. And that service talks to another service, etc. I think of it as this function communicates to this other function. And that is true, even outside of serverless, where functions are the primitive. Serverless was a really great place for us to start because it's already segmented into functions. But if you're looking at a microservice, there's no reason that you can't think about your code, about say this functions or this component or this resource path is communicating to this other function and also, contextually. So, for instance, maybe this service only calls DynamoDB when it's inserting data. Or when the API gateway, there's a put request, right? That triggers a put into DynamoDB.

You don't get a put into DynamoDB when you do a delete on a API gateway. So that's the kind of context that I think is really interesting for things like tracing. That is a little bit more I think, beyond what traditional tracing solutions have been doing.

Jeremy: Right. Because I mean, it's a lot different in these distributed systems. I mean, even if you're just talking to one microservice, it's usually you talk to one microservice and maybe you want to see that continuity there, or service X called service Y. But now with serverless specifically, we have you function X calling service Y, which generates an event that then gets picked up by EventBridge, and then another service picks it up and so forth. So we can get into why it's important in serverless applications, but is there anything else, where observability is different in the sense of monitoring? I mean, you mentioned the idea of being a little bit proactive. What do you mean by being more proactive?

Erica: Well, by being proactive I guess I'm referring to the fact that things again, rewinding history a little bit and going to something very distinctly not observably like Nagios. Again saying, "This very reactive." Something went down and now we then asked it, "If it's up?" And I said, "It's..." And we couldn't reach it, so then we determined that it's down. I think that kind of step two of that kind of journey towards observability, would say, "Okay. Well, we have logs, we have a logging product. And the logs told me that when, I don't know. This service tried running to the Syslog Server, they got an error." Well, when I get that error, I know that at least this sys...

About Sven Al Hamad

Sven Al Hamad is co-founder and CEO of Webiny Serverless CMS. Sven has worked with the largest media and ecommerce customers in Europe as their trusted advisor on the topics of web performance and architecture, and has a proven track record of successful delivery of several multi-million dollar projects for large enterprises. Sven is also an experienced entrepreneur who has acted as a CTO in 4 different startups.

• Twitter: twitter.com/svenalhamad
• Email: [email protected]
• Webiny: webiny.com
• Webiny Twitter: twitter.com/WebinyPlatform
• Github: github.com/webiny

Watch this episode on YouTube: https://youtu.be/9TSmOcLBr0k

Transcript

Jeremy: Hi everyone. I'm Jeremy Daly and this is Serverless Chats. Today I'm speaking with Sven Al Hamad. Hey Sven. Thanks for joining me.

Sven: Hey, Jeremy. Thanks for having me.

Jeremy: So you are the CEO and co-founder of Webiny. So why don't you tell the listeners a little bit about your background and what Webiny is?

Sven: Yeah. So well, in terms of my background, I'm a developer. I started maybe 20 years ago, even more coding websites and many other stuff along the way, but also worked in the enterprise world for several years and decided to start Webiny about a year and a half ago, maybe two years back. But now I'm more focused on the business side. And what Webiny is, it's essentially an open source framework for building full stack applications that deploy to serverless infrastructure like AWS Lambda and similar. So it's all about creating serverless solutions.

Jeremy: Awesome. All right. So that's what I want to talk to you about obviously the Webiny platform, what you can do with it. And I'd like to start by kind of going through more of the details, right? Because I think we get confused with maybe what a CMS is versus what a application platform is versus what a cloud provider is. I mean, so I think it can get very confusing if you don't dive deep into the docks and even I was looking at the Webiny site and I was like, "All right, it's a CMS, but it also has a framework for building things." And so I'd love to go through that. And then we can talk about a couple of other things just to get your insight on serverless, but let's start at the beginning. So why did you build Webiny? What was the thing that triggered that?

Sven: So I started researching the serverless market in general. That was late 2017 and the more I dived deeper into the potential of serverless and serverless infrastructure, I kind of understood that serverless has such a big potential that actually it can become the standard, how we are building all applications in the future. Essentially, if you want to build an application five years from now, you're going to build it on serverless infrastructure. Serverless is going to become that standard.

And at the same time, I looked at the market, in terms of the solutions that are available today to help you do that. Well, there was nothing that I could use out of the box to help me build an application. There are tools to help you monitor and deploy serverless applications, but there are no tools to help you build actually a full stack serverless application. So I saw a big opportunity there, but also at the same time, I had a web design development agency many years back where we were all testing different CMSs and different solutions on building things. So from that learning and that experience, I decided, "Okay, let's take that. Let's take the serverless market, which is new and has great potential. Let's build a solution for that market that also is open source at the same time, so it benefits the whole ecosystem and the community as a whole."

Jeremy: Yeah. And who among us, hasn't owned a web development company in the past? I know I did for about 12 years and honestly it's funny because we built a CMS. I mean, CMSs were, this was before WordPress, right? So I mean, WordPress comes along and it changes a bunch of things. But one thing that was always a pain for me, and I know we can get into this as part of what the tool offers. But it was things like building forms. If I had to build one more HTML form in my web development company, I mean, I was ready to just, I don't know, go stick my head in a closet or something like that because it was just, it's so tedious, it's so repetitive. And as one of the things I think we've done really well or we've done a good job of is we've abstracted away a lot of these things and tried to make it so that where we make the undifferentiated heavy lifting, much easier.

But a big part of that, and this is something that always scared me, every time you build a new project, it's less about the interface, it's less about, maybe the backend, it's a lot about the data, right? We want to make sure that our data is secure, that our data is backed up. So I'd love to just talk about the data model that you have built into Webiny, because again, it supports a bunch of different things. But could you explain that?

Sven: Yeah, so well just handling data in a serverless environment comes with its own challenges. We found that really early on. So we decided to go with a MongoDB, particularly MongoDB Atlas to store the data, but how you actually talk to a MongoDB database from a Lambda function it's not the best today. If you don't use some specialized solutions, but you also find a similar problem with MySQL. That's why you created the MySQL library to help you do that. But essentially, what we did with Webiny, we have this notion of multicloud in our minds and we didn't want to get locked into specific databases. And we built a data library, a library that handles how we talk to databases and how we model the data models actually inside Webiny.

And that library is also open source, it's called Komodo and through Komodo, we built pretty much all the data models that you see in Webiny today in all our applications. And the beauty is that with Komodo, on the other side, you have these adapters for MongoDB. We also have an adapter for MySQL that's not published there, but we're planning on also building an adapter for DynamoDB and things like that. So we had to kind of also put some constraints on the data model. We didn't want to make it fully no SQL because we also want to support SQL in the future. So we put some constraints there but Komodo is kind of lifting off all the complexities there for us as a user. And on the other side, what we found is another challenge with handling pretty much TCP connections to the database. So we built, if you look at our architecture, we built something called the database proxy, essentially a Lambda function to which all our Lambda functions, talk to.

And only that Lambda function has the actual connections to the data, but it's like a funnel. So you can kind of limit how many connections you send to the actual database, reducing the number of zombie connections and so on. Some database have really, really smart abilities. So you can programmatically tell it, "Kill this connection or open another connection." But MongoDB doesn't have that. So it's pretty much kind of a, just having that funnel really low and then queuing up all the requests there. So but yeah, the data model has being a challenging topic for us there was a lot of iteration, a lot ...

About Rafal Wilinski
Rafal Wilinski is the founder of Dynobase, a professional GUI Client for DynamoDB with mission to onboard thousands of developers to Cloud-native and Serverless world. In addition to founding Dynobase, Rafal distributes a weekly newsletter “This Week in DynamoDB” and is a Serverless Engineer at Stedi. Prior to his current roles, Rafal got his start in developing mobile games, and is a cloud native engineer, AWS certified architect, and Serverless Framework contributor and maintainer.

• Twitter: twitter.com/rafalwilinski
• Dynobase: dynobase.dev/
• This Week in DynamoDB: dynobase.dev/newsletter/

Watch this episode on YouTube: https://youtu.be/41YJAflfnP4

Transcript

Jeremy: Hi everyone! I'm Jeremy Daly, and this is Serverless Chats. Today I'm chatting with Rafal Wilinski. Hey Rafal, thanks for joining me.

Rafal: Hi, thanks for having me.

Jeremy: So you are the creator of Dynobase and an independent AWS consultant. Can you tell the listeners a little bit about yourself and what Dynobase is all about?

Rafal: Yeah, sure. As you mentioned, I'm founder of Dynobase, professional graphical interface for DynamoDB, and also right now an independent AWS consultant, mostly focusing on serverless solutions. I'm deeply passionate about AWS and mostly serverless, since, I think, 2016, because I attended the first Serverless Conf back in London, and that's why I became so excited about this whole field. I'm running my own blog which is called Servicefull, because we had so many discussions about what serverless is and how bad a name that is for a paradigm of technology.

So I decided to actually steal the term coined by Patrick Debois, which is Servicefull, because full of services. I'm writing about serverless, about cloud, and I recently merged that with my own page, but you can still go there. It's going to redirect you. Before going all in into AWS, I was actually making mobile games. I've made a few of them, one of them became even quite popular, which was called Voxel Rush. But my parents were saying that making mobile games isn't a real job, so that's why I transitioned into making web and cloud, and now I'm here. Less than a year ago, I started Dynobase, which was trying to solve my problems with UX and UI with DynamoDB, but I guess we'll talk about it a little bit later.

Jeremy: Right. Yeah. And actually, that's why I want to talk to you about today is Dynobase. So you and I have been communicating for quite some time. You know I'm a huge fan of DynamoDB. I love just the scale of it. I love what you can do with it. Rick Houlihan opened up, I think, everyone's mind or minds with what you can do in regards to relational structures in there and how you can access data in different ways in the single table design stuff, which is quite fascinating. But I really want to get into Dynobase, what it does, what's the purpose of it, but maybe we just start at the beginning. Why did you create Dynobase?

Rafal: Yeah, sure. Actually, there is sort of I think quite interesting story behind it because it all started more than a year ago when I was working at X-Team. I was working on kind of quite a big project for the educational space. I was working as AWS DevOps engineer, I was setting up infrastructures, it was all set up on containers. But we received a new requirement to create a fully real time community platform. Our architects, Raynard, which is a great friend and also an engineer, approached me and said that, "Hey, I know that you're super interested in serverless. I know that you've contributed some pieces to serverless framework, and you write about it. So maybe we can try evaluating all those new tools that AWS released, like AppSync, Amplify, DynamoDB, and try to create something on top of that."

And that sounded super good because I could finally use serverless technologies at my day job. So we immediately rushed into evaluating those tools. We watched a lot of random session including Rick Houlihan about single table design and it was mind blowing and charming at the same time. But when we were evaluating those tools, we realized that if you would like to use AppSync, we probably have to use Amplify and if we go with Amplify, we can't go with single table design, because if you're using Amplify, you're creating graphical schema, which is then translated to separate DynamoDB tables, which is not working with single table design super good. So we decided to use serverless framework DynamoDB single table design to create everything. And we rushed into the implementation without having thought all about testing processes, about debugging because we decided to learn as we go because we had that opportunity.

And when we started implementation, obviously, there was a lot of bugs and there was a lot of mistakes. We had our single table design schema, designed very good, but it was a new field for us. So we obviously committed a lot of beginner mistakes. While we were doing that, we started checking a lot of data, a lot of records inside DynamoDB just in console, because we needed to check if that specific record was inserted correctly if the data that it's inside database is actually good, or we wanted to modify some records manually. And while I was doing that, I realized that I'm spending definitely too much time switching between browsers, between regions, between AWS accounts, because, for instance, you can't have open two separate AWS regions, two separate AWS consoles for two regions in one browser. It was actually a pain. The same for regions. So you had to actually kind of hack your browser. You had to have opened many browsers and it was just super messy. There was no bookmarks. There was no history. Scanning speed was quite bad.

So I decided, I'm an engineer, I don't like wasting my time fighting with software. I like automation, I like solving stuff. So I decided to hack something really quickly using React and Electron, like a tool which is going to allow me just query in a little bit easier fashion, and it's going to be easier. Yeah. And I've asked a few of my friends who are also working with DynamoDB, which are engineers, if they are sharing the same pains as I am, and it appeared that a lot of people actually have the same problem of DynamoDB, that the DynamoDB itself is super good. It's super powerful database. It's enabled things that you could never imagine before. But the way that you access it, the way you modify records directly when you debug it, it's not super good, especially if you're working with the local version of DynamoDB. You can't easily access what's inside.

And while I had that confirmation that it's not only my own problem because I initially wanted to open source the solution. I felt that, hey, I was working in open source space for the past five or even more years, I've committed so many lines of code, I've committed so many things to serverless framework and to other tools, and I haven't received any money for it. It was greedy approach, but, you know. So, I decided actually to turn it into a product and try creating a business of it because it seemed that many people would pay for solving their problem. And when I realized that it can actually work. I had this moment of big excitement and I gained a lot of power just to work tirelessly, even after my day job, just to finish this UI. I think I did the first prototype in a month.

It took me something like 100 hours. I was about to release it. I announced on Twitter to...

About Matthieu Napoli:

Matthieu Napoli is a software consultant and founder of null. Matthew has been developing web applications for more than 10 years with PHP and JavaScrapt as a full-stack developer, lead dev, and CTO, while maintaining several open source projects. He’s also the author of PHP-DI, Silly, Couscous, and Bref.

• Twitter: twitter.com/matthieunapoli
• Personal website: mnapoli.fr
• null: null.tc
• Bref: bref.sh
• Serverless PHP Newsletter: serverless-php.news

Watch this episode on YouTube: https://youtu.be/H8tkZcjQxOA

Transcript

Jeremy: Hi everyone. I'm Jeremy Daly, and this is Serverless Chats. Today, I'm chatting with Matthieu Napoli. Hey Matthieu. Thanks for joining me.

Matthieu: Hi, thanks for having me.

Jeremy: You are a serverless consultant and the founder of Null. Why don't you tell the listeners a little bit about your background and what Null does?

Matthieu: Yes. I created Null two years ago. My goal was to be able to both work in open source as well as work for clients. I use that company to do trainings around Bref, around serverless and also to provide consulting services.

Jeremy: What about your background?

Matthieu: I started as a developer about 10 years ago, and I've been working mostly as a developer. I've been configuring servers, setting up servers for a while. That's why I'm also really interested in serverless. I've been looking at that very closely lately.

Jeremy: Great. All right. I want to talk to you today about serverless and PHP and Bref. Serverless is obviously the topic of this podcast. We've seen quite a bit of movement in the serverless space over the course of the last five years or so. PHP sometimes gets some slack on the internet, but can you give me a brief background as to why you chose PHP?

Matthieu: Yes. That's a very good question and that's a good way to start, because indeed a lot of people have opinions about PHP, and sometimes for good reason. I started with PHP just because it was simple. That's what I love about this language. At the time, like when PHP arrived, it was about 25 years ago. The web was about creating CGI applications, using C or whatever, and PHP arrived and simplified everything. It made the web accessible to a lot of people. I find that really amazing.

That's why I started with PHP as well. I wanted to build a simple website. Yeah. I started with PHP because of that, but I'm seeing the same thing today with serverless. It's making infrastructure, it's making hosting applications accessible again to developers and I find that amazing. Yeah. I started with PHP. I kind of got stuck with this language throughout my jobs and lately PHP has become a very interesting language. To be honest, it's really interesting.

If you've used PHP in the past, I really encourage you to give it another look. It's really worth it. While I do talk a lot and use a lot of PHP, I enjoy using JavaScript as well. TypeScript lately. Really, really interesting language. Life is full of things to learn about, I guess.

Jeremy: Right. Absolutely. I agree with you on PHP. I started with PERL and CGI way, way back when, and then I think I started using PHP 3.0 or something like that with MySQL databases. You're right. It completely changed things. From that we got WordPress for better or for worse, but I think that like 80% of the web runs on PHP.

Matthieu: Yeah. That's a huge market, which is interesting when we're going to talk about AWS Lambda later. Yeah. PHP is huge and I don't think this is something that we can ignore.

Jeremy: Right. Right. Okay. Speaking of this PHP, you realized that there was a gap in the serverless ecosystem for PHP, and so you wrote something called Bref. Can you tell the listeners what that's all about?

Matthieu: Exactly. Yes. I am a developer. I like writing code, creating applications. I don't like setting up servers and all of that stuff. This is why I created Bref. I wanted a simple way to put my PHP code online. At the time I was looking into serverless, looking into AWS Lambda and I discovered, of course, that AWS Lambda does not support PHP. I created Bref to bridge the gap, run PHP on Lambda and provide a lot of tools, documentation, examples. Yeah. Anything that you may lack to create those serverless applications. I would say that Bref is more than just a runtime. It's a whole stack.

Jeremy: Right. There's actually two parts of Bref, right? Why don't you explain those two different parts?

Matthieu: Yeah. I realized over time that there are two major use cases when you look at Bref and what you can do with PHP on Lambda. In the first case, you know about AWS Lambda. You know how it works. You know why you use it. The only thing that's missing is that you want to run PHP for some reason. Maybe you want to use PHP and you want to run it on Lambda. The first part of Bref is a runtime that works just like any other language on Lambda.

Yeah. You can write functions in PHP, handle queue messages, SQS queue messages, EventBridge messages react to S3 events, API gateway events as well. You know, the usual. The second use case is different. Instead of adapting PHP to run on Lambda, there are people that know PHP and do not really know about Lambda and what they can do with it. I take it the other way around and I adapt Lambda to PHP. The approach is that users don't have to change anything in their code.

They can take their Laravel application, Symfony application or whatever, and hopefully put it in Lambda and it just works. That's a second runtime. This runtime, I mean, we can go into the details. It's really interesting because the way PHP runs is very similar to how AWS Lambda runs. Making the old PHP way run on Lambda was fairly ... I mean, I don't want to say easy, but it was doable. That's the second approach where, well, people can just start using Lambda as a web host. That's how I host Lambda as a web host instead of functions.

Jeremy: Sure. Right. The custom runtime for just the first part of it, so just being able to run PHP on Lambda, this is something that's really interesting because I know there are others that maintain PHP runtimes out there. You are optimizing it for actual PHP developers, right? You're using PHP-FPM, right?

Matthieu: Exactly. Yes. The FPM runtime, so that the FPM runtime is used for the use case where you want to use AWS Lambda as a web hosting platform. The FPM runtimes actually runs PHP-FPM, which is like PHP web server inside Lambda. Bref has a little bridge that when there is an API gateway event, will take the event, convert it into a request that PHP-FPM understands. This is the first CGI protocol and so Bref does the bridge, provides the first CGI request to PHP. Then PHP runs just as usual.

You know, the PHP execution model is you have a request, a PHP process starts, builds the whole framework, runs the request, processes the request, and returns the response and then `. I mean, this is perfect for AWS Lambda. That's why it's quite easy to integrate FPM with Lambda.

Jeremy: Right. Then you have the ability to actually create individual handlers as functi...

About Joe Duffy

Joe Duffy is cofounder and CEO of Pulumi. Prior to founding Pulumi, Joe was a longtime leader in Microsoft’s Developer Division, Operating Systems Group, and Microsoft Research. Most recently, he was Director of Engineering and Technical Strategy for developer tools, where part of his responsibilities included managing groups building the C#, C++, Visual Basic, and F# languages. Joe created teams for several successful distributed programming platforms, initiated and executed on efforts to take .NET open source and cross-platform, and was instrumental in Microsoft’s company-wide open-source transformation. Joe founded Pulumi in 2018 with Eric Rudder, the former Chief Technical Strategy Officer at Microsoft.

• Twitter: twitter.com/funcofjoe
• Pulumi: www.pulumi.com/
• Pulumi Twitter: twitter.com/PulumiCorp 

Watch this episode on YouTube: https://youtu.be/MYOGfK9PHM8

Transcript

Jeremy: Hi everyone. I'm Jeremy Daly and this is Serverless Chats. Today I'm speaking with Joe Duffy. Hey Joe. Thanks for joining me.

Joe: Hey Jeremy. Thanks for having me.

Jeremy: You are the CEO and founder of Pulumi. Can you give the listeners a little bit about your background and tell us what Pulumi does?

Joe: Yeah, happy to. I founded Pulumi three years ago. Before that, I was an early engineer on the .NET framework at Microsoft. I was actually at Microsoft for a hearty 13 years working in and around developer tools the entire time, managing groups. I actually led the languages team before leaving, helped with the open source transformation at Microsoft, which was really cool to be a part of, and then founded Pulumi. Pulumi is a modern infrastructure as code platform that really brings everything we know and love about application development using great programming languages, great tooling, and actually brings it over to the infrastructure side of the house and really trying to help both infrastructure teams be super productive with great tools but also empower developers to use more of the cloud as part of their application architecture itself.

Jeremy: Awesome. You just mentioned there cloud engineers or the infrastructure team and then serverless developers, so I look at this and I tend to think... especially with smaller organizations... they're almost becoming one and the same. But as you get larger organizations and you start to separate that responsibility... and whether you have separate cloud teams and separate developers or different cells that do that kind of stuff, there is kind of this separation between the infrastructure dev ops people and the serverless developers. Can you explain that difference?

Joe: Yeah. And I agree. I think developers are doing more infrastructure now than they've ever done in the past and I think serverless is really forcing this issue a bit. Is a serverless function infrastructure or is it code? The line's a little blurry. But there's clear things that are still in the infrastructure domain: for example, setting up a virtual private cloud in Amazon, setting up a network, setting up a Kubernetes cluster. Even if you're going to run serverless functions within Kubernetes somebody's got to manage the cluster. Somebody's got to think about security. Somebody's got to think about monitoring. Some of that actually falls on the applications side. A lot of it falls on the infrastructure side.

I think of it as there are deep domain experts in the infrastructure space just like there are deep domain experts in the applications space. I think the magic of what we're seeing with serverless in particular is that the line is getting a little blurry. It's more of a policy decision, I like to say, than a technology decision about who does what. The infrastructure team is going to do the network because that's what they know how to do. They're experts in that. The development team probably doesn't want to become domain experts in how to set up networks. Similarly, the infrastructure team doesn't want to become domain experts in how serverless functions work, so it's better if the developers can be self-serve and really own their own destiny there. I think the tools and workflows really need to support the concept of these two disciplines working closer together going forward and I love that you used the phrase cloud engineering because that's really what I think of. It's the best of developers, the best of infrastructure engineers, really collaborating together.

Jeremy: Right. That's interesting, because I totally agree with that. Where, then, are these challenges, or what are these new challenges that you're seeing both sides face? Because as a developer, like you said, you need to get a little closer to the infrastructure. As a infrastructure person, you need to get a bit closer to the configurations that the developers are setting up. What are the challenges for the developers?

Joe: Infrastructure's hard. For developers specifically, I think no serverless function is an island. A serverless function is only interesting when it's paired with the infrastructure that triggers an event, whether that's a bucket, you want to do something every time a file gets added, or an API gateway where you're actually using serverless functions for infinite scale on the back end. It needs to be connected to infrastructure and historically what that's meant... Actually, one of the reasons we founded Pulumi was I got really excited about serverless and containers and I wanted to create a serverless application and it was great until I had to configure the infrastructure. I wrote 100 lines of JavaScript for a nice little serverless application. I'm like, "All right. I'm ready to go. What do I do next? Oh. For every 10 lines of JavaScript I have to write 100 lines of YAML." That was not pleasant.

That was one of the problems we wanted to solve, was really, "Let's just make it feel like we've got a real programming model, an application model, where we're just building serverless applications and the infrastructure is part of that." Not all of it, again, but a lot of it really should be closer to the applications. Most of the technology today doesn't have that worldview and so there's kind of a fundamental friction and mismatch.

Jeremy: Right. I think that's one of the things that you see more of now, especially with developers. They have to take the infrastructure into consideration now when they write in code. They didn't use to necessarily need to worry about that. Now it's like, "My code I know is going to interact with some other piece of infrastructure," which I think is a challenge there too. So what about the infrastructure side or the dev/ops people? Or I should say the ops people. What are the challenges that they face now that they've got people kind of meddling with their infrastructure?

Joe: It is a challenge to empower developers to manage more of the infrastructure, because there really is this... I think most technologies today and most teams today assume that there's this hard fall between the two sides of the house. As you pointed out, for smaller companies and companies who are born in the cloud, who are starting today, they have the advantage of not creating those silos, but for most teams there is a hard wall between them and for good reasons: because of these domain specializations and expertise. To your point, 10 years ago if I was just doing virtual machines I ta...

Watch this episode on YouTube: https://youtu.be/hFGrKjtWsQg

Transcript:

ON BEST PRACTICES...

Episode #1: Alex DeBrie
Asking about best practices and the reality of implementing them...
@4:24
Alex: I think it's pretty fascinating to see. Like you say, if you're on Twitter and you're following a lot of the big time people doing serverless architectures in this space, they have a lot of great tips around best practices, and this is what you should be doing, all that stuff. But I find, as I'm building serverless applications or as I'm talking to customers and users that are building serverless applications, there are times when there's tension between what the best practices are and what their circumstances are. This could be because maybe they're not coming in with a green field application, or maybe they have a data model that doesn't fit DynamoDB or something like that. It's difficult on how you sort of square that with recommending something that you know isn't the best practice or the most pure serverless application, but you also gotta help people ship products, right? I think balancing that tension can be tough at times.

Episodes #18 & #19: Michael Hart
@30:25 (Ep. 19)
Michael: There's nothing special about Lambda in this respect. It's like, this is just sort of best practices if you were calling any API or if you're writing any API that if you're waiting for many, many, many, many seconds, then you might want to deal with that. And those are the sorts of use cases where I think, okay, fine, that's perhaps not a good practice. You actually, you asked me. We use this at Bustle. So we have a Lambda that renders our frontend HTML code. It's a preact app. It does service side rendering of the HTML, but it delivers to the, you know, to the browser via API Gateway and a CDN and things like that. But it calls our other Lambda directly, which is a GraphQL backend. It calls that to pull in the data that it needs to render the HTML page. Now in the browser, it also will call that GraphQL backend , but it will do it via API Gateway. Because it's coming from the browser, so it needs to make some an authenticated HTTP request into the function. But when you're in the Lambda world, well, that Lambda can just call that Lambda directly, and call the GraphQL Lambda, and that goes to Redis and Elasticsearch wherever it needs to pull the data and send it back. And we just make sure we have the timeouts tuned such that, you know, I mean it responds within milliseconds. It’s not even a thing we would really run into.

ON INSTRUMENTATION...

Episode #2: Nitzan Shapira
Asking about the need to automate instrumentation...
@29:35
Nitzan: Yeah, by the way, it's not just worrying. It's not just the fact that you can forget. It's also just going to take you a certain amount of time - always - that you're going to basically waste instead of writing your own business software. Even if you do remember to do it every time, it's still going to take you some time. Some ways that can work [are] in embedded in your standard libraries that you work with. If you have a library that is commonly used to communicate between services, you want to embed that tracing information or extra information there, so it will always be there. This will kind of automate a lot of the work for you. That's just a matter of what type of tool do you use. If you use X-Ray you're still going have to do some kind of manual work. And it's fine, at first. The problem is when you suddenly grow from 100 functions to 1000 functions — that's where you're going to be probably a little bit annoyed or even lost, because it's going to be just a lot of work and doesn't seem like something that really scales. Anything manual doesn't really scale. This is why you use serverless, because you don't want to scale service manually.

ON APPSYNC DATA OWNERSHIP...

Episode #3: Marcia Villalba
Asking about what service owns the data when using AppSync...
@36:20
Marcia: Well, then, it's the question on who owns the data, and that's something, at least with AppSync, I'm still trying to figure out how to really architect my application, my graph qualifications, because I've been using GraphQL with microservices, and usually I do the filtering in the microservice because the microservice knows the data, knows who can see it, and I don't want to leak that information out. But with AppSync, at least applications and have been building, they are mostly contained into Dynamo tables and Lambdas. So I think when I'm coding this that AppSync is the owner of this data and, then I do the filtering in the resolvers. So I think it's always a question of who owns the data and who is able —where is the level that you want to leak the information out? I don't know if it's clear.

ON WORKFLOW COMPLEXITY...

Episode #4: Chase Douglas
Asking about the complexity of the development workflow...
@6:35
Chase: Yeah, for all the benefits you get from serverless, with its auto scaling and its capabilities of scaling down to zero, which reduces developer cost, you do have some things that you have to manage that are a little different than before. One of the key things is, if I've got, like, a compute resource like a Lambda function in the cloud that has a set of permissions that it's granted and it has some mechanism for locating, the external service is like SQS queue or an SNS topic or an S3 bucket. So it has these two things that it needs to be able to function the permissions and locations. So the challenge that people often hit very early on in serverless development is if I'm writing software on my laptop and I want to test it without having to go through a full deployment cycle, which may take a few minutes to ah to deploy the latest code change. Even if it's, ah one character change up to the cloud service provider. How can I actually test with proper permissions and proper service discovery location mechanisms from my laptop? What mechanisms are there to do that? That's something that we are always evolving.

ON EVENT-DRIVEN ARCHITECTURE...

Episode #5: Mike Deck
When asking about event-driven architecture...
@27:35
Mike: I think that it's probably easiest to understand it when contrasted against kind of a command-driven architecture, which I think is what we're mostly sort of used to. So this idea that I've got some set of APIs that I go out and call and I kind of issue commands there, right? So I maybe have an order service. I'm calling create order or I've got downstream from that. There's some invoicing service now, and so the order service goes out and calls that  and says, "Create the invoice, please." So that's kind of the standard command-oriented model that you typically see with API-driven architectures. An event-driven architecture is kind of, instead of creating specific, directed commands, you're simply publishing these events that talk about facts that have happened, you know these are signals that state has changed within the application. So the order service may publish an event that says, "hey, an order was created." And now it's up to the other downstream services to, they can observe that event and then do the piece of the process that they're responsible for at that point. So it's kind of a subtle difference, but it's really powerful once you really start kind of taking this further down the road in terms of the ability to decouple your services from one another, right? So when you've got a lot of services that need to interact with a number of other ones, you end up kind of with a lot of knowledge about all of those downstre...