In this inaugural episode of Benchtalks, Snorkel AI co-founder Vincent Chen sits down with Alex Shaw, MTS at Laude Institute and co-creator of Terminal-Bench, to unpack what the rapid hill-climbing on TB2 reveals about the state of AI agent evaluation — and where the field needs to go.

This interview covers: 

• Why TB2 went from 20–30% during development to 75–80% at the frontier today
• The bet on the terminal as the right abstraction for general computer use
• How Harbor became a benchmark factory — and why that matters for RL post-training
• The "benchmaxxing" problem and how the community is keeping TB2 honest
• What Terminal-Bench 3 needs from expert contributors to shape model development for the next year

Full interview/transcript: https://snorkel.ai/blog/benchtalks-alex-shaw-terminal-bench-harbor-building-the-benchmark-factory/