Sign up to save your podcasts
Or
Share Meta GenAI Infra Blog Review // Special MLOps Podcast
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Meta GenAI Infra Blog Review // Special MLOps Podcast
Meta GenAI Infra Blog Review // Special MLOps Podcast episode by Demetrios.
// Abstract
Demetrios explores Meta's innovative infrastructure for large-scale AI operations, highlighting three blog posts on training large language models, maintaining AI capacity, and building Meta's GenAI infrastructure. The discussion reveals Meta's handling of hundreds of trillions of AI model executions daily, focusing on scalability, cost efficiency, and robust networking. Key elements include the Ops planner work orchestrator, safety protocols, and checkpointing challenges in AI training. Meta's efforts in hardware design, software solutions, and networking optimize GPU performance, with innovations like a custom Linux file system and advanced networking file systems like Hammerspace. The podcast also discusses advancements in PyTorch, network technologies like Roce and Nvidia's Quantum 2 Infiniband fabric, and Meta's commitment to open-source AGI.
// MLOps Jobs board
https://mlops.pallet.xyz/jobs
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Building Meta’s GenAI Infrastructure blog: https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Timestamps:
[00:00] Meta handles trillions of AI model executions
[07:01] Meta creating AGI, ethical and sustainable
[08:13] Concerns about energy use in training models
[12:22] Network, hardware, and job optimization for reliability
[17:21] Highlights of Arista and Nvidia hardware architecture
[20:11] Meta's clusters optimized for efficient fabric
[24:40] Varied steps, careful checkpointing in AI training
[28:46] Meta is maintaining huge GPU clusters for AI
[29:47] AI training is faster and more demanding
[35:27] Ops planner orchestrates a million operations and reduces maintenance
[37:15] Ops planner ensures safety and well-tested changes
View all episodes
By Demetrios
4.6
2323 ratings
Meta GenAI Infra Blog Review // Special MLOps Podcast episode by Demetrios.
// Abstract
Demetrios explores Meta's innovative infrastructure for large-scale AI operations, highlighting three blog posts on training large language models, maintaining AI capacity, and building Meta's GenAI infrastructure. The discussion reveals Meta's handling of hundreds of trillions of AI model executions daily, focusing on scalability, cost efficiency, and robust networking. Key elements include the Ops planner work orchestrator, safety protocols, and checkpointing challenges in AI training. Meta's efforts in hardware design, software solutions, and networking optimize GPU performance, with innovations like a custom Linux file system and advanced networking file systems like Hammerspace. The podcast also discusses advancements in PyTorch, network technologies like Roce and Nvidia's Quantum 2 Infiniband fabric, and Meta's commitment to open-source AGI.
// MLOps Jobs board
https://mlops.pallet.xyz/jobs
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Building Meta’s GenAI Infrastructure blog: https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Timestamps:
[00:00] Meta handles trillions of AI model executions
[07:01] Meta creating AGI, ethical and sustainable
[08:13] Concerns about energy use in training models
[12:22] Network, hardware, and job optimization for reliability
[17:21] Highlights of Arista and Nvidia hardware architecture
[20:11] Meta's clusters optimized for efficient fabric
[24:40] Varied steps, careful checkpointing in AI training
[28:46] Meta is maintaining huge GPU clusters for AI
[29:47] AI training is faster and more demanding
[35:27] Ops planner orchestrates a million operations and reduces maintenance
[37:15] Ops planner ensures safety and well-tested changes
More shows like MLOps.community
View all
The a16z Show
1,097 Listeners
Software Engineering Daily
627 Listeners
Super Data Science: ML & AI Podcast with Jon Krohn
302 Listeners
NVIDIA AI Podcast
346 Listeners
Data Engineering Podcast
146 Listeners
Y Combinator Startup Podcast
226 Listeners
Practical AI
205 Listeners
Machine Learning Street Talk (MLST)
97 Listeners
Dwarkesh Podcast
522 Listeners
No Priors: Artificial Intelligence | Technology | Startups
133 Listeners
This Day in AI Podcast
228 Listeners
AI + a16z
35 Listeners
Lightcone Podcast
22 Listeners
Training Data
42 Listeners
The Pragmatic Engineer
71 Listeners