Legendary computer scientist Donald Knuth was recently in for a "shock" when an open mathematical problem he had been working on for several weeks was successfully solved by Anthropic's Claude Opus 4.6. In this episode, we dive into the fascinating story behind "Claude's Cycles," exploring how this generative AI showcased a dramatic advance in automatic deduction and creative problem-solving.

The complex problem, intended for a future volume of The Art of Computer Programming, involved finding a general decomposition of a specific digraph's arcs into three directed Hamiltonian m^3-cycles. After Knuth solved it for m=3, his friend Filip Stappers challenged Claude to find a generalized solution. Guided by strict instructions to document its progress, Claude worked through 31 distinct algorithmic "explorations". Moving from simple depth-first search and simulated annealing to "serpentine patterns" and fiber decomposition, the AI eventually realized it needed "pure math" to discover a working solution for all odd values of m.

Join us as we recount Claude's impressive 60-minute analytical journey, discuss the 760 perfectly valid "Claude-like" decompositions, and see how Knuth rigorously proved the AI's brilliant discovery. Hats off to Claude!

Read Don Knuth's original paper here: https://cs.stanford.edu/~knuth/papers/claude-cycles.pdf

There's an incredible buzz around AI agents, with the prevailing wisdom suggesting that bigger is always better. The industry has poured billions into monolithic, Large Language Models (LLMs) to power these new autonomous systems. But what if this dominant approach is fundamentally misaligned with what agents truly need?

This episode dives deep into compelling new research from Nvidia that makes a powerful case for a paradigm shift: the future of agentic AI isn't bigger, it's smaller. We unpack the core arguments for why Small Language Models (SLMs) are poised to become the new standard, offering superior efficiency, dramatic cost savings, and unprecedented operational flexibility.

Join us as we explore:

• Surprising, real-world examples where compact SLMs are already outperforming massive LLM giants on critical tasks like tool use and code generation.

• The key economic and operational benefits of adopting a modular, "Lego-like" approach with specialized SLMs.

• A clear-eyed look at the practical barriers holding back adoption and the counter-arguments from the "LLM-first" world.

• A concrete, 6-step roadmap for organizations to begin transitioning and harnessing the power of a more agile, cost-effective SLM architecture.

This isn't just an incremental improvement; it's a potential reshaping of the AI landscape. Tune in to understand why the biggest revolution in AI might just be the smallest.

The research paper discussed in this episode, "Small Language Models Are the Future of Agentic AI," can be found on arXiv:
https://arxiv.org/pdf/2506.02153

It looks incredibly impressive when a large language model explains its step-by-step thought process, giving us a window into its "mind." But what if that visible reasoning is a sophisticated illusion? This episode dives deep into a groundbreaking study on the new generation of "Large Reasoning Models" (LRMs)—AIs specifically designed to show their work.

We explore the surprising and counterintuitive findings that challenge our assumptions about machine intelligence. Discover the three distinct performance regimes where these models can "overthink" simple problems, shine on moderately complex tasks, and then experience a complete "performance collapse" when things get too hard. We'll discuss the most shocking discoveries: why models paradoxically reduce their effort when problems get harder, and why their performance doesn't improve even when they're given the exact algorithm to solve a puzzle. Is AI's reasoning ability just advanced pattern matching, or are we on the path to true artificial thought?

Reference:
This discussion is based on the findings from the Apple Machine Learning Research paper, "The Illusion of Thinking: Understanding the Strengths and Limitations of Large Language Models with Pyramids of Thought."
https://machinelearning.apple.com/research/illusion-of-thinking

What happens when AI becomes vastly smarter than humans? It sounds like science fiction, but researchers are grappling with the very real challenge of ensuring Artificial General Intelligence (AGI) is safe for humanity. Join us for a deep dive into the cutting edge of AI safety research, unpacking the technical hurdles and potential solutions. We explore the core risks – from intentional misalignment and misuse to unintentional mistakes – and the crucial assumptions guiding current research, like the pace of AI progress and the "approximate continuity" of its development. Learn about the key strategies being developed, including safer design patterns, robust control measures, and the concept of "informed oversight," as we navigate the complex balance between harnessing AGI's immense potential benefits and mitigating its profound risks.

An Approach to Technical AGI Safety and

Security: https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/evaluating-potential-cybersecurity-threats-of-advanced-ai/An_Approach_to_Technical_AGI_Safety_Apr_2025.pdf

Google Deepmind AGI Safety Course: https://youtube.com/playlist?list=PLw9kjlF6lD5UqaZvMTbhJB8sV-yuXu5eW

Ever tried to understand how AI actually learns, only to get lost in a sea of equations and jargon? This episode is your fast track through the fundamentals of machine learning, breaking down complex concepts into understandable nuggets.

Drawing inspiration from Stanford course materials, we ditch the dense textbook approach and offer a clear, conversational deep dive into the core mechanics of AI learning. Join us as we explore:

• Linear Predictors: The versatile workhorses of early ML, from classifying spam to predicting prices.

• Feature Extraction: The art of turning raw data (like an email) into numbers the algorithm can understand.

• Weights & Scores: How AI weighs different information (like ingredients in a recipe) to make a prediction using the dot product.

• Loss Minimization & Margin: How do we measure when AI gets it wrong, and how does it use that feedback (like the concept of 'margin') to improve?

• Optimization Powerhouses: Unpacking Gradient Descent and its faster cousin, Stochastic Gradient Descent (SGD) – the engines that drive the learning process.

Whether you're curious about AI or need a refresher on the basics, this episode provides a solid foundation, explaining how machines learn without needing an advanced degree. Get ready to understand the building blocks of artificial intelligence!

Stanford's Algorithms for Artificial Intelligence: https://web.stanford.edu/~mossr/pdf/alg4ai.pdf

 It's over 50 years old, yet the C programming language remains a fundamental building block beneath countless technologies we use daily, from operating systems to embedded devices. Join us as we dive deep into the core principles of C, guided by the classic Kernighan & Ritchie text. We explore its history, key syntax, control flow, functions, and the low-level power that explains why C still matters profoundly today.

The C Programming Language (2nd Edition): http://cslabcms.nju.edu.cn/problem_solving/images/c/cc/The_C_Programming_Language_%282nd_Edition_Ritchie_Kernighan%29.pdf

Open source software powers our lives, from our phones to our fridges. We uncover its enormous economic impact, revealing how much businesses save (trillions of dollars!), the importance of different programming languages, and how to ensure its sustainable future.

The Value of Open Source Software: https://www.hbs.edu/faculty/Pages/item.aspx?num=65230

Ever wondered how AI is transforming our lives, from smart homes to the stock market? In this episode, we explore the fascinating world of AI agents – systems that can perceive, reason, and act to achieve goals. We'll unpack the core concepts from "AI Foundations of Computational Agents," delve into real-world examples like trading and tutoring agents, and discuss the blend of neural networks and logical reasoning that powers them. Get ready to understand the building blocks of AI's future!

Artificial Intelligence: Foundations of Computational Agents (3rd Edition): https://mrce.in/ebooks/AI%20Foundations%20of%20Computational%20Agents%203rd%20Ed.pdf

Unlock the secrets behind lightning-fast data retrieval! We explore the inner workings of database storage engines, from simple bash scripts to complex B-trees. Discover how indexes work, the trade-offs between speed and storage space, and how data warehouses organize vast amounts of historical data.

Designing Data-Intensive Applications (free chapters): https://www.scylladb.com/wp-content/uploads/ScyllaDB-Designing-Data-Intensive-Applications.pdf

DDIA book: https://dataintensive.net

Large Language Models are everywhere, but how do they actually work? Join us as we unravel the science and engineering behind these powerful AIs that can write, translate, and even create. From pre-training to fine-tuning, we explore how they learn and what it means for the future of tech.

Foundations of Large Language Models: https://arxiv.org/pdf/2501.09223