AWS Bites

108. How to Solve Lambda Python Cold Starts


Listen Later

In this episode, we discuss how you can use Python for data science workloads on AWS Lambda. We cover the pros and cons of using Lambda for these workloads compared to other AWS services. We benchmark cold start times and performance for different Lambda deployment options like zip packages, layers, and container images. The results show container images can provide faster cold starts than zip packages once the caches are warmed up. We summarize the optimizations AWS has made to enable performant container image deployments. Overall, Lambda can be a good fit for certain data science workloads, especially those that are bursty and need high concurrency.


💰 SPONSORS 💰

AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com !
In this episode, we mentioned the following resources.

  • Our blog post detailing our research on how to optimise Python Data Science in AWS Lambda: https://fourtheorem.com/optimise-python-data-science-aws-lambda/
  • The repository with our benchmarks and related visualizations: https://github.com/fourTheorem/lambda-datasci-perf
  • On-demand Container Loading on AWS Lambda (AWS Paper): https://arxiv.org/abs/2305.13162

  • Do you have any AWS questions you would like us to address?

    Leave a comment here or connect with us on X, formerly Twitter:
    - ⁠⁠https://twitter.com/eoins⁠⁠
    - ⁠⁠https://twitter.com/loige⁠⁠

    ...more
    View all episodesView all episodes
    Download on the App Store

    AWS BitesBy AWS Bites

    • 4.6
    • 4.6
    • 4.6
    • 4.6
    • 4.6

    4.6

    11 ratings


    More shows like AWS Bites

    View all
    Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

    Software Engineering Radio - the podcast for professional software developers

    272 Listeners

    The McKinsey Podcast by McKinsey & Company

    The McKinsey Podcast

    378 Listeners

    Planet Money by NPR

    Planet Money

    30,734 Listeners

    The Changelog: Software Development, Open Source by Changelog Media

    The Changelog: Software Development, Open Source

    284 Listeners

    Thoughtworks Technology Podcast by Thoughtworks

    Thoughtworks Technology Podcast

    41 Listeners

    Talk Python To Me by Michael Kennedy

    Talk Python To Me

    585 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    624 Listeners

    The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

    The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

    435 Listeners

    AWS Podcast by Amazon Web Services

    AWS Podcast

    202 Listeners

    Data Engineering Podcast by Tobias Macey

    Data Engineering Podcast

    140 Listeners

    Kubernetes Podcast from Google by Abdel Sghiouar, Kaslin Fields

    Kubernetes Podcast from Google

    183 Listeners

    The Real Python Podcast by Real Python

    The Real Python Podcast

    138 Listeners

    Big Technology Podcast by Alex Kantrowitz

    Big Technology Podcast

    453 Listeners

    The AWS Developers Podcast by Amazon Web Services

    The AWS Developers Podcast

    22 Listeners

    The Pragmatic Engineer by Gergely Orosz

    The Pragmatic Engineer

    62 Listeners