AI Engineering Podcast

Unlocking AI Potential with AMD's ROCm Stack


Listen Later

Summary
In this episode of the AI Engineering podcast Anush Elangovan, VP of AI software at AMD, discusses the strategic integration of software and hardware at AMD. He emphasizes the open-source nature of their software, fostering innovation and collaboration in the AI ecosystem, and highlights AMD's performance and capability advantages over competitors like NVIDIA. Anush addresses challenges and opportunities in AI development, including quantization, model efficiency, and future deployment across various platforms, while also stressing the importance of open standards and flexible solutions that support efficient CPU-GPU communication and diverse AI workloads.

Announcements
  • Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
  • Your host is Tobias Macey and today I'm interviewing Anush Elangovan about AMD's work to expand the playing field for AI training and inference
Interview
  • Introduction
  • How did you get involved in machine learning?
  • Can you describe what your work at AMD is focused on?
  • A lot of the current attention on hardware for AI training and inference is focused on the raw GPU hardware. What is the role of the software stack in enabling and differentiating that underlying compute?
  • CUDA has gained a significant amount of attention and adoption in the numeric computation space (AI, ML, scientific computing, etc.). What are the elements of platform risk associated with relying on CUDA as a developer or organization?
  • The ROCm stack is the key element in AMD's AI and HPC strategy. What are the elements that comprise that ecosystem?
    • What are the incentives for anyone outside of AMD to contribute to the ROCm project?
  • How would you characterize the current competitive landscape for AMD across the AI/ML lifecycle stages? (pre-training, post-training, inference, fine-tuning)
  • For teams who are focused on inference compute for model serving, what do they need to know/care about in regards to AMD hardware and the ROCm stack?
  • What are the most interesting, innovative, or unexpected ways that you have seen AMD/ROCm used?
  • What are the most interesting, unexpected, or challenging lessons that you have learned while working on AMD's AI software ecosystem?
  • When is AMD/ROCm the wrong choice?
  • What do you have planned for the future of ROCm?
Contact Info
  • LinkedIn
Parting Question
  • From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
  • Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
  • ImageNet
  • AMD
  • ROCm
  • CUDA
  • HuggingFace
  • Llama 3
  • Llama 4
  • Qwen
  • DeepSeek R1
  • MI300X
  • Nokia Symbian
  • UALink Standard
  • Quantization
  • HIPIFY
  • ROCm Triton
  • AMD Strix Halo
  • AMD Epyc
  • Liquid Networks
  • MAMBA Architecture
  • Transformer Architecture
  • NPU == Neural Processing Unit
  • llama.cpp
  • Ollama
  • Perplexity Score
  • NUMA == Non-Uniform Memory Access
  • vLLM
  • SGLang
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
...more
View all episodesView all episodes
Download on the App Store

AI Engineering PodcastBy Tobias Macey

  • 4.3
  • 4.3
  • 4.3
  • 4.3
  • 4.3

4.3

6 ratings


More shows like AI Engineering Podcast

View all
The Cloudcast by Massive Studios

The Cloudcast

154 Listeners

a16z Podcast by Andreessen Horowitz

a16z Podcast

1,043 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

624 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

298 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

340 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

140 Listeners

AI Today Podcast by AI & Data Today

AI Today Podcast

151 Listeners

Practical AI by Practical AI LLC

Practical AI

183 Listeners

The Stack Overflow Podcast by The Stack Overflow Podcast

The Stack Overflow Podcast

63 Listeners

Last Week in AI by Skynet Today

Last Week in AI

298 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

91 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

128 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

72 Listeners

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

496 Listeners

AI + a16z by a16z

AI + a16z

31 Listeners