airhacks.fm podcast with adam bien

Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference


Listen Later

An airhacks.fm conversation with Juan Fumero (@snatverk) about:
tornadovm as a Java parallel framework for accelerating data parallelization on GPUs and other hardware,
first GPU experiences with ELSA Winner and Voodoo cards,
explanation of TornadoVM as a plugin to existing JDKs that uses Graal as a library,
TornadoVM's programming model with @parallel and @reduce annotations for parallelizable code,
introduction of kernel API for lower-level GPU programming,
TornadoVM's ability to dynamically reconfigure and select the best hardware for workloads,
implementation of LLM inference acceleration with TornadoVM,
challenges in accelerating Llama models on GPUs,
introduction of tensor types in TornadoVM to support FP8 and FP16 operations,
shared buffer capabilities for GPU memory management,
comparison of Java Vector API performance versus GPU acceleration,
discussion of model quantization as a potential use case for TornadoVM,
exploration of Deep Java Library (DJL) and its ND array implementation,
potential standardization of tensor types in Java,
integration possibilities with Project Babylon and its Code Reflection capabilities,
TornadoVM's execution plans and task graphs for defining accelerated workloads,
ability to run on multiple GPUs with different backends simultaneously,
potential enterprise applications for LLMs in Java including model distillation for domain-specific models,
discussion of Foreign Function & Memory API integration in TornadoVM,
performance comparison between different GPU backends like OpenCL and CUDA,
collaboration with Intel Level Zero oneAPI and integrated graphics support,
future plans for RISC-V support in TornadoVM

Juan Fumero on twitter: @snatverk

...more
View all episodesView all episodes
Download on the App Store

airhacks.fm podcast with adam bienBy Adam Bien

  • 5
  • 5
  • 5
  • 5
  • 5

5

5 ratings


More shows like airhacks.fm podcast with adam bien

View all
Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

Software Engineering Radio - the podcast for professional software developers

272 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

284 Listeners

Thoughtworks Technology Podcast by Thoughtworks

Thoughtworks Technology Podcast

40 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

621 Listeners

Soft Skills Engineering by Jamison Dance and Dave Smith

Soft Skills Engineering

269 Listeners

AWS Podcast by Amazon Web Services

AWS Podcast

201 Listeners

Duke's Corner by Jim Grisanzio

Duke's Corner

8 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

987 Listeners

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

CoRecursive: Coding Stories

189 Listeners

A Bootiful Podcast by Josh Long

A Bootiful Podcast

29 Listeners

The Stack Overflow Podcast by The Stack Overflow Podcast

The Stack Overflow Podcast

62 Listeners

Mac & i - der Apple-Podcast by Mac & i

Mac & i - der Apple-Podcast

0 Listeners

Inside Java by Chad Arimura, David Delabassee

Inside Java

26 Listeners

Foojay.io, the Friends Of OpenJDK! by Foojay.io

Foojay.io, the Friends Of OpenJDK!

0 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

53 Listeners