Technically Speaking

For years the recipe for better AI was simple: more data, bigger models.

Then DeepSeek's R1 showed models could learn to reason by practicing on problems with checkable answers, and Reinforcement Learning became the way forward.

But what actually is RL? And is it something every startup should be doing, or just a practice for big AI labs?

Julien Launay, cofounder of Adaptive ML, and Sesh Nalla, VP of Engineering at Datadog (which bought Adaptive), made big bets on RL before it was obvious.

In this episode, Julien, Sesh and Megan Reynolds discuss:
- What RL adds to pre-training ("pre-training is a block of marble")
- How an RL run actually works, step by step
- Why measuring an RL run is "an observability problem"
- Whether you should train your own model ("for 99% of people it doesn't make any sense")
- What scares the people building autonomous agents

Presented by Railway, your agent’s favorite cloud - get started at https://go.technically.dev/railway.

Original music by The Jazz Boys.

Show notes and the full transcript: https://technically.dev/podcast/all-about-reinforcement-learning

• (00:00) - Why is everyone talking about RL?

Every few weeks a new AI model tops the charts, as if dropped out of the sky. But what's the work that actually goes into that?

• AI research is one loop: change the initial setup, run it, see if the numbers improve. "Did those numbers improve and what did we change and how did those numbers change," and then "are those numbers truly meaningful in the things that matter for our customers."
• Reinforcement learning depends on scoring of outcomes. "You give a model a problem to solve, you let it try to solve that problem, you give it feedback on how well it tried to solve that problem."
• Most AI answers can't be marked as simply right or wrong. Databricks builds example companies to test its agents, then grades the answers "the way you would grade a student's essay in school."
• Models learn to game their own tests: "it makes the numbers look really good. And then you actually have the team start interacting with the model and you're like, this sucks."
• Fine-tuning is a commitment, not recommended for most. Jonathan usually says no first: "Once you do this it's a commitment. You're kind of getting married to fine-tuning for a while."
• A better benchmark score doesn't mean a better model for you. "You are not a general-purpose benchmark."

Presented by Railway, your agent’s favorite cloud.

Original music by The Jazz Boys.

Show notes and the full transcript: https://technically.dev/podcast/what-does-an-ai-researcher-actually-do

• (00:00) - What is AI research?

We thought we had access control for software all figured out. You joined a company, and got access to just the tools you needed, and none you didn't.

• Auth is two questions: who are you, and what are you allowed to do? Authentication proves the first, authorization answers the second. Michael's analogy is a driver's licence: the front proves you're you, and "if on the back of your driver's license it says that you have a motorcycle endorsement, you can ride a motorcycle."
• SSO, SAML and SCIM are your work login, the plumbing behind it, and switching it off when you leave. Megan's summary, which he signed off on: "SSO for login, SAML for how it kind of comes together and the protocol works, and then SCIM for when you leave."
• You can't tell what an AI agent needs from what you asked it. Ask an agent to find ways to save the company money and "you don't know what systems it needs access to. In practice, it needs access to everything. But also nothing."
• Approval pop-ups don't scale to agents. What works for one person approving one request breaks when software acts thousands of times a second: "To err is human, but to screw up 10,000 times per second you need a computer."
• What's risky depends on the company, so the AI model can't be the one to decide. "In Company A, it might be totally fine for a model to take a certain action in a certain domain. In a Company B, it might be totally different." At WorkOS, anything under $100 you just buy.
• The old systems don't go away; they get extended for agents. WorkOS's future is "auth for agents and auth for AI and not just auth for people."

Presented by Railway, your agent’s favorite cloud.

Original music by The Jazz Boys.

Show notes and the full transcript: https://technically.dev/podcast/all-about-auth

• (00:00) - What does WorkOS do?