The GenAI Evolution Atlas

The GenAI Evolution Atlas

Download on the App Store

The GenAI Evolution Atlas episodes

  • How Alignment Turned Autocomplete Into AI Assistants

    A pre-trained model only predicts the next token. Alignment turned that autocomplete into a helpful assistant: InstructGPT's RLHF pipeline, then DPO and Constitutional AI.

    Key takeaways:

    • The three-step pipeline. InstructGPT (2022) used supervised fine-tuning on human demonstrations, a reward model trained on human rankings, then reinforcement learning (PPO) against that reward.
    • A leash on drift. A KL penalty keeps the tuned model close to the original so it doesn't exploit quirks in the reward model.
    • Alignment beats size. Outputs from the 1.3B InstructGPT model were preferred over those of 175B GPT-3.
    • DPO simplifies it. Direct Preference Optimization (2023) learns straight from preference pairs with no separate reward model or RL loop, opening alignment to the open-source community.
    • AI feedback. Anthropic's Constitutional AI has the model critique and revise its own outputs against written principles. Reward hacking and sycophancy remain open problems.

    Chapters:

    00:00 From autocomplete to assistant

    04:33 InstructGPT

    04:51 Supervised fine-tuning

    06:08 The reward model

    07:13 Reinforcement learning with PPO

    07:35 The KL leash

    09:21 Small but aligned beats big

    11:15 Open source and DPO

    13:55 Constitutional AI

    15:58 Reward hacking and sycophancy

    20:45 Closing thought

    Sources:

    • Training language models to follow instructions with human feedback (InstructGPT) (Ouyang et al., 2022)
    • Direct Preference Optimization (Rafailov et al., 2023)
    • Constitutional AI: Harmlessness from AI Feedback (Bai et al., Anthropic, 2022)

    Related episode: S1E8: Why AI Now Reasons and Acts

    Enjoyed it? Follow The GenAI Evolution Atlas (weekly generative AI deep dives and news roundups, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

    Also from Peter: Data Science in the Wild, how real companies ship ML and AI in production, one system per episode.

    Made with AI-generated voices; curated by Peter Liu from the sources above.

    22 min
  • How FlashAttention and MoE Saved AI Scaling

    Bigger models hit hard limits on memory, speed and cost. FlashAttention, mixture-of-experts, RoPE, KV-cache tricks, quantization and LoRA kept AI scaling without changing the Transformer's core design.

    Key takeaways:

    • FlashAttention. Dao et al. (2022) compute exact attention in tiles held in fast on-chip SRAM, avoiding slow trips to GPU main memory (HBM).
    • Mixture-of-experts. A router sends each token to a few expert sub-networks, so total parameters grow while compute per token stays modest, as in Switch Transformers.
    • Position and memory. Rotary position embeddings (RoPE) encode relative position. Grouped-query attention shrinks the KV cache, and vLLM's PagedAttention stops reserved memory from sitting empty.
    • Faster, smaller inference. Speculative decoding uses a small draft model to propose tokens. Quantization stores weights in fewer bits.
    • Cheap customization. LoRA (Hu et al., 2021) freezes the base model and trains small low-rank matrices, so one base model can serve many adapters.

    Chapters:

    00:00 Hitting the scaling wall

    06:59 FlashAttention

    07:30 SRAM vs. HBM

    09:36 Tiling

    12:59 Mixture-of-experts

    14:42 Routing tokens to experts

    21:13 Rotary position embeddings

    25:13 The KV cache problem

    28:03 Grouped-query attention

    30:06 PagedAttention and vLLM

    31:09 Speculative decoding

    32:40 Quantization

    34:51 LoRA

    38:00 Recap

    Sources:

    • FlashAttention (Dao et al., 2022)
    • Switch Transformers (Fedus, Zoph and Shazeer, 2021)
    • RoFormer: Rotary Position Embedding (Su et al., 2021)
    • PagedAttention (vLLM) (Kwon et al., 2023)
    • LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)

    Related episode: S1E2: The Transformer Engine Behind Modern AI

    Enjoyed it? Follow The GenAI Evolution Atlas (weekly generative AI deep dives and news roundups, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

    Also from Peter: Data Science in the Wild, how real companies ship ML and AI in production, one system per episode.

    Made with AI-generated voices; curated by Peter Liu from the sources above.

    41 min
  • How Massive Scale Triggered Emergent AI

    Between 2019 and 2022, AI progress became a matter of scale. GPT-3, scaling laws, the Chinchilla correction, and the debate over whether “emergent” abilities are real or a measurement mirage.

    Key takeaways:

    • In-context learning. GPT-3, at 175 billion parameters, could pick up a task from a few examples in the prompt, with no weight updates.
    • Scaling laws. Kaplan et al. (2020) found loss falls as a smooth power law in model size, data and compute, which made giant training runs predictable.
    • The Chinchilla correction. Hoffmann et al. (2022) showed models like GPT-3 were undertrained: parameters and training tokens should grow together.
    • Emergence, or a mirage? Wei et al. (2022) described abilities that appear suddenly at scale. Schaeffer et al. (2023) argued many of those jumps come from all-or-nothing metrics.
    • Scale isn't alignment. A far smaller model tuned on human feedback was preferred over GPT-3, which sets up the next stage.

    Chapters:

    00:00 When scale became the strategy

    03:47 Why RNNs couldn't scale

    07:14 The 2017 Transformer, briefly

    17:14 GPT-2

    18:07 GPT-3 and in-context learning

    23:12 Kaplan's scaling laws

    27:49 The Chinchilla correction

    32:43 Attention's quadratic cost

    34:02 Emergent abilities

    36:11 The mirage argument

    40:16 Why scale isn't enough

    42:30 Instruction tuning and RLHF

    47:48 Closing thought

    Sources:

    • Language Models are Few-Shot Learners (GPT-3) (Brown et al., 2020)
    • Scaling Laws for Neural Language Models (Kaplan et al., 2020)
    • Training Compute-Optimal Large Language Models (Chinchilla) (Hoffmann et al., 2022)
    • Emergent Abilities of Large Language Models (Wei et al., 2022)
    • Are Emergent Abilities of Large Language Models a Mirage? (Schaeffer et al., 2023)

    Related episode: S1E6: How Alignment Turned Autocomplete Into AI Assistants

    Enjoyed it? Follow The GenAI Evolution Atlas (weekly generative AI deep dives and news roundups, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

    Also from Peter: Data Science in the Wild, how real companies ship ML and AI in production, one system per episode.

    Made with AI-generated voices; curated by Peter Liu from the sources above.

    50 min
  • The Big Bang of Modern AI

    In 2018–2019, AI stopped needing a labeled dataset for every task. Pre-train once on raw text, then adapt: how BERT, GPT and T5 split the field, and why that choice still shapes today's models.

    Key takeaways:

    • The new recipe. Pre-train once on huge amounts of unlabeled text, then fine-tune for each task, instead of collecting labels for everything.
    • The text is the label. Hide part of a sentence and predict it. The model learns grammar, facts and context without anyone labeling anything.
    • The fork. Google's BERT (2018) masks words and reads in both directions, making it a strong understanding model that Google brought into Search. OpenAI's GPT reads left to right and predicts the next token, a natural generator.
    • Everything is text-to-text. Google's T5 reframed every task as text in, text out, using an encoder-decoder.
    • Tokens matter. Subword tokenizers split rare words into reusable pieces, trading vocabulary size against sequence length.

    Chapters:

    00:00 2018: teaching AI to read

    01:48 Pre-train once, adapt everywhere

    07:34 Self-supervision: hide and predict

    11:48 The fork: BERT vs. GPT

    14:04 BERT's bidirectional masking

    17:13 BERT in Google Search

    18:58 GPT's left-to-right bet

    22:42 Why causal models drift

    23:33 T5: everything is text-to-text

    27:40 One model absorbs every task

    29:54 Hardware limits and tokenization

    34:32 Fine-tuning in 2018–2019

    40:11 Closing thought

    Sources:

    • Improving Language Understanding by Generative Pre-Training (GPT) (Radford et al., OpenAI, 2018)
    • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., 2018)
    • Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5) (Raffel et al., 2019)

    Related episode: S1E4: How Massive Scale Triggered Emergent AI

    Enjoyed it? Follow The GenAI Evolution Atlas (weekly generative AI deep dives and news roundups, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

    Also from Peter: Data Science in the Wild, how real companies ship ML and AI in production, one system per episode.

    Made with AI-generated voices; curated by Peter Liu from the sources above.

    41 min
  • The Transformer Engine Behind Modern AI

    How the 2017 Transformer works, piece by piece: queries, keys and values, scaled softmax, multi-head attention, positional encodings, and why its quadratic cost shaped everything that came after.

    Key takeaways:

    • Drop the recurrence. “Attention Is All You Need” (Vaswani et al., 2017) kept attention and threw out the RNN, so every token is processed in parallel.
    • Queries, keys, values. Each token asks a question (query), advertises what it has (key) and carries content (value). Dot products, scaled and passed through a softmax, decide how much each token takes from the others.
    • Many heads, one sense of order. The original model ran eight attention heads in parallel. Because attention has no built-in sense of word order, sine and cosine positional encodings add it.
    • The decoder. Masked attention lets the model generate one token at a time without peeking ahead, the same block later scaled up into GPT-3.
    • The catch. Attention cost grows quadratically with sequence length, which later motivated FlashAttention and mixture-of-experts.

    Chapters:

    00:00 A machine with no visible gears

    03:05 The RNN problem, revisited

    09:18 From encoder-decoder to attention

    11:13 Attention Is All You Need

    12:55 No built-in sense of order

    14:21 Queries, keys and values

    16:41 Scaled dot products and softmax

    22:13 Multi-head attention

    25:01 Positional encodings

    27:01 Residuals and feed-forward layers

    28:54 The decoder and masking

    31:57 The quadratic cost

    35:05 FlashAttention and mixture-of-experts

    39:42 Closing thought

    Sources:

    • Attention Is All You Need (Vaswani et al., 2017)
    • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (Dao et al., 2022)

    Related episode: S1E5: How FlashAttention and MoE Saved AI Scaling

    Enjoyed it? Follow The GenAI Evolution Atlas (weekly generative AI deep dives and news roundups, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

    Also from Peter: Data Science in the Wild, how real companies ship ML and AI in production, one system per episode.

    Made with AI-generated voices; curated by Peter Liu from the sources above.

    41 min
  • How Attention Solved the RNN Bottleneck

    Before the Transformer there was the RNN bottleneck. This episode explains why attention had to be invented: slow sequential math, fading gradients, and the single vector that every translation had to squeeze through.

    Key takeaways:

    • One step at a time. An RNN can't compute step t until step t−1 is finished, which leaves the thousands of parallel cores on a GPU idle.
    • Fading signals. During training the error signal is multiplied through every time step, so it vanishes or explodes and long-range links are hard to learn.
    • LSTMs patch the leak. Hochreiter and Schmidhuber's 1997 LSTM adds gates and a protected memory path that keep information and gradients alive.
    • The bottleneck, then attention. Seq2seq (2014) compressed a whole sentence into one fixed-size vector. Bahdanau, Cho and Bengio let the decoder look back at every encoder state, weighted by a softmax: attention.
    • Words as vectors. Word2Vec (Mikolov et al., 2013) learned embeddings in which related words sit close together.

    Chapters:

    00:00 Why start before the Transformer

    04:22 How RNNs read a sequence

    07:12 Why sequential math wastes GPUs

    09:39 Backpropagation through time

    11:54 Vanishing and exploding gradients

    13:09 LSTMs and gating

    17:33 Machine translation and seq2seq

    20:06 The fixed-size bottleneck

    22:06 Bahdanau attention

    25:26 Softmax alignment weights

    30:32 Word embeddings and Word2Vec

    37:35 Recap

    40:12 Closing thought

    Sources:

    • Long Short-Term Memory (Hochreiter and Schmidhuber, 1997)
    • Sequence to Sequence Learning with Neural Networks (Sutskever, Vinyals and Le, 2014)
    • Neural Machine Translation by Jointly Learning to Align and Translate (Bahdanau, Cho and Bengio, 2014)
    • Efficient Estimation of Word Representations in Vector Space (Mikolov et al., 2013)

    Related episode: S1E2: The Transformer Engine Behind Modern AI

    Enjoyed it? Follow The GenAI Evolution Atlas (weekly generative AI deep dives and news roundups, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

    Also from Peter: Data Science in the Wild, how real companies ship ML and AI in production, one system per episode.

    Made with AI-generated voices; curated by Peter Liu from the sources above.

    42 min

About The GenAI Evolution Atlas

From the publisher's feed

A map of how generative AI got here, and where it's going next.

Season 2 (ongoing): every Wednesday, a deep dive into the most important AI breakthrough of the week, and every Friday, a…