AWS Bites

110. Why should you use Lambda for Machine Learning?

Listen Later

In this episode, we discuss using AWS Lambda for machine learning inference. We cover the tradeoffs between GPUs and CPUs for ML, tools like ggml and llama.cpp for running models on CPUs, and share examples where we've experimented with Lambda for ML like podcast transcription, medical imaging, and natural language processing. While Lambda ML is still quite experimental, it can be a viable option for certain use cases.


💰 SPONSORS 💰

AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com !
In this episode, we mentioned the following resources.

  • Episode "46. How do you do machine learning on AWS?": https://awsbites.com/46-how-do-you-do-machine-learning-on-aws/
  • Episode "108. How to Solve Lambda Python Cold Starts": https://awsbites.com/108-how-to-solve-lambda-python-cold-starts/
  • ggml (the framework): https://github.com/ggerganov/ggml
  • ggml (the company): https://ggml.ai
  • llama.cpp: https://github.com/ggerganov/llama.cpp
  • whisper.cpp: https://github.com/ggerganov/whisper.cpp
  • whisper.cpp WebAssembly demo: https://whisper.ggerganov.com/
  • ONNX Runtime: https://onnxruntime.ai/
  • An example of using whisper.cpp with the Rust bindings: https://github.com/lmammino/whisper-rs-example
  • Project running Whisper.cpp in a Lambda function: https://github.com/eoinsha/whisper_lambda_cpp
  • AWS Lambda Image Container Chest X-Ray Example: https://github.com/fourTheorem/lambda-image-cxr-detection
  • Episode "103. Building GenAI Features with Bedrock": https://awsbites.com/103-building-genai-features-with-bedrock/⁠


    • Do you have any AWS questions you would like us to address?

    Leave a comment here or connect with us on X, formerly Twitter:
    - ⁠⁠⁠⁠https://twitter.com/eoins⁠⁠⁠⁠
    - ⁠⁠⁠⁠https://twitter.com/loige⁠⁠

    ...more
    View all episodesView all episodes
    Download on the App Store
    AWS BitesBy AWS Bites
    • 4.7
    • 4.7
    • 4.7
    • 4.7
    • 4.7

    4.7

    12 ratings

    More shows like AWS Bites

    View all
    Up First from NPR by NPR

    Up First from NPR

    56,944 Listeners

    AWS Podcast by Amazon Web Services

    AWS Podcast

    204 Listeners

    .NET Rocks! by Carl Franklin and Richard Campbell

    .NET Rocks!

    242 Listeners

    Bad Friends by Bobby Lee & Andrew Santino

    Bad Friends

    14,543 Listeners

    The AWS Developers Podcast by Amazon Web Services

    The AWS Developers Podcast

    26 Listeners