Sign up to save your podcasts
Or
Share Building GPT-2 in a Spreadsheet — Everything You Wanted to Know About LLMs (But Were Afraid to Ask)
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Building GPT-2 in a Spreadsheet — Everything You Wanted to Know About LLMs (But Were Afraid to Ask)
Learn how to demystify large language models by building GPT-2 from scratch — in a spreadsheet. In this episode, MIT engineer Ishan Anand breaks down the inner workings of transformers in a way that’s visual, interactive, and beginner-friendly, yet deeply technical for experienced builders.
What you’ll learn:
• How GPT-2 became the architectural foundation for modern LLMs like ChatGPT, Claude, Gemini, and LLaMA.
• The three major innovations since GPT-2 — mixture of experts, RoPE (rotary position embeddings), and advances in training — and how they changed AI performance.
• A clear explanation of tokenization, attention, and transformer blocks that you can see and manipulate in real time.
• How to implement GPT-2’s core in ~600 lines of code and why that understanding makes you a better AI builder.
• The role of temperature, top-k, and top-p in controlling model behavior — and how RLHF reshaped the LLM landscape.
• Why hands-on experimentation beats theory when learning cutting-edge AI systems.
Ishan Anand is an engineer, MIT alum, and prolific AI tinkerer who built a fully functional GPT-2 inside a spreadsheet — making it one of the most accessible ways to learn how LLMs work. His work bridges deep technical insight with practical learning tools for the AI community.
Key topics covered:
• Step-by-step breakdown of GPT-2 architecture.
• Transformer math and attention mechanics explained visually.
• How modern LLMs evolved from GPT-2’s original design.
• Practical insights for training and fine-tuning models.
• Why understanding the “old” models makes you better at using the new ones.
This episode of AI Tinkerers One-Shot goes deep under the hood with Ishan to show how LLMs really work — and how you can start building your own.
💡 Resources:
• Ishan Anand LinkedIn – https://www.linkedin.com/in/ishananand/
• AI Tinkerers – https://aitinkerers.org
• One-Shot Podcast – https://one-shot.aitinkerers.org/
👍 Like this video if you found it valuable, and subscribe to AI Tinkerers One-Shot for more conversations with innovators building the future of AI!
View all episodes
By Joe HeitzebergLearn how to demystify large language models by building GPT-2 from scratch — in a spreadsheet. In this episode, MIT engineer Ishan Anand breaks down the inner workings of transformers in a way that’s visual, interactive, and beginner-friendly, yet deeply technical for experienced builders.
What you’ll learn:
• How GPT-2 became the architectural foundation for modern LLMs like ChatGPT, Claude, Gemini, and LLaMA.
• The three major innovations since GPT-2 — mixture of experts, RoPE (rotary position embeddings), and advances in training — and how they changed AI performance.
• A clear explanation of tokenization, attention, and transformer blocks that you can see and manipulate in real time.
• How to implement GPT-2’s core in ~600 lines of code and why that understanding makes you a better AI builder.
• The role of temperature, top-k, and top-p in controlling model behavior — and how RLHF reshaped the LLM landscape.
• Why hands-on experimentation beats theory when learning cutting-edge AI systems.
Ishan Anand is an engineer, MIT alum, and prolific AI tinkerer who built a fully functional GPT-2 inside a spreadsheet — making it one of the most accessible ways to learn how LLMs work. His work bridges deep technical insight with practical learning tools for the AI community.
Key topics covered:
• Step-by-step breakdown of GPT-2 architecture.
• Transformer math and attention mechanics explained visually.
• How modern LLMs evolved from GPT-2’s original design.
• Practical insights for training and fine-tuning models.
• Why understanding the “old” models makes you better at using the new ones.
This episode of AI Tinkerers One-Shot goes deep under the hood with Ishan to show how LLMs really work — and how you can start building your own.
💡 Resources:
• Ishan Anand LinkedIn – https://www.linkedin.com/in/ishananand/
• AI Tinkerers – https://aitinkerers.org
• One-Shot Podcast – https://one-shot.aitinkerers.org/
👍 Like this video if you found it valuable, and subscribe to AI Tinkerers One-Shot for more conversations with innovators building the future of AI!