Machine Learning Street Talk (MLST)

César Hidalgo has spent years trying to answer a deceptively simple question: What is knowledge, and why is it so hard to move around?

We all have this intuition that knowledge is just... information. Write it down in a book, upload it to GitHub, train an AI on it—done. But César argues that's completely wrong. Knowledge isn't a thing you can copy and paste. It's more like a living organism that needs the right environment, the right people, and constant exercise to survive.

Guest: César Hidalgo, Director of the Center for Collective Learning

1. Knowledge Follows Laws (Like Physics)

2. You Can't Download Expertise

3. Why Big Companies Fail to Adapt

4. The "Infinite Alphabet" of Economies

If you think AI can just "copy" human knowledge, or that development is just about throwing money at poor countries, or that writing things down preserves them forever—this conversation will change your mind. Knowledge is fragile, specific, and collective. It decays fast if you don't use it.

The Infinite Alphabet [César A. Hidalgo]

https://www.penguin.co.uk/books/458054/the-infinite-alphabet-by-hidalgo-cesar-a/9780241655672

https://x.com/cesifoti

Rescript link.

https://app.rescript.info/public/share/eaBHbEo9xamwbwpxzcVVm4NQjMh7lsOQKeWwNxmw0JQ

---

TIMESTAMPS:

00:00:00 The Three Laws of Knowledge

00:02:28 Rival vs. Non-Rival: The Economics of Ideas

00:05:43 Why You Can't Just 'Download' Knowledge

00:08:11 The Detective Novel Analogy

00:11:54 Collective Learning & Organizational Networks

00:16:27 Architectural Innovation: Amazon vs. Barnes & Noble

00:19:15 The First Law: Learning Curves

00:23:05 The Samuel Slater Story: Treason & Memory

00:28:31 Physics of Knowledge: Joule's Cannon

00:32:33 Extensive vs. Intensive Properties

00:35:45 Knowledge Decay: Ise Temple & Polaroid

00:41:20 Absorptive Capacity: Sony & Donetsk

00:47:08 Disruptive Innovation & S-Curves

00:51:23 Team Size & The Cost of Innovation

00:57:13 Geography of Knowledge: Vespa's Origin

01:04:34 Migration, Diversity & 'Planet China'

01:12:02 Institutions vs. Knowledge: The China Story

01:21:27 Economic Complexity & The Infinite Alphabet

01:32:27 Do LLMs Have Knowledge?

---

REFERENCES:

Book:

[00:47:45] The Innovator's Dilemma (Christensen)

https://www.amazon.com/Innovators-Dilemma-Revolutionary-Change-Business/dp/0062060244

[00:55:15] Why Greatness Cannot Be Planned

https://amazon.com/dp/3319155237

[01:35:00] Why Information Grows

https://amazon.com/dp/0465048994

Paper:

[00:03:15] Endogenous Technological Change (Romer, 1990)

https://web.stanford.edu/~klenow/Romer_1990.pdf

[00:03:30] A Model of Growth Through Creative Destruction (Aghion & Howitt, 1992)

https://dash.harvard.edu/server/api/core/bitstreams/7312037d-2b2d-6bd4-e053-0100007fdf3b/content

[00:14:55] Organizational Learning: From Experience to Knowledge (Argote & Miron-Spektor, 2011)

https://www.researchgate.net/publication/228754233_Organizational_Learning_From_Experience_to_Knowledge

[00:17:05] Architectural Innovation (Henderson & Clark, 1990)

https://www.researchgate.net/publication/200465578_Architectural_Innovation_The_Reconfiguration_of_Existing_Product_Technologies_and_the_Failure_of_Established_Firms

[00:19:45] The Learning Curve Equation (Thurstone, 1916)

https://dn790007.ca.archive.org/0/items/learningcurveequ00thurrich/learningcurveequ00thurrich.pdf

[00:21:30] Factors Affecting the Cost of Airplanes (Wright, 1936)

https://pdodds.w3.uvm.edu/research/papers/others/1936/wright1936a.pdf

[00:52:45] Are Ideas Getting Harder to Find? (Bloom et al.)

https://web.stanford.edu/~chadj/IdeaPF.pdf

[01:33:00] LLMs/ Emergence

https://arxiv.org/abs/2506.11135

Person:

[00:25:30] Samuel Slater

https://en.wikipedia.org/wiki/Samuel_Slater

[00:42:05] Masaru Ibuka (Sony)

https://www.sony.com/en/SonyInfo/CorporateInfo/History/SonyHistory/1-02.html

This is a lively, no-holds-barred debate about whether AI can truly be intelligent, conscious, or understand anything at all — and what happens when (or if) machines become smarter than us.

Dr. Mike Israetel is a sports scientist, entrepreneur, and co-founder of RP Strength (a fitness company). He describes himself as a "dilettante" in AI but brings a fascinating outsider's perspective.

Jared Feather (IFBB Pro bodybuilder and exercise physiologist)

The Big Questions:

1. When is superintelligence coming?

2. Does AI actually understand anything?

3. The Simulation Debate (The Spiciest Part)

4. Will AI kill us all? (The Doomer Debate)

5. What happens to human jobs and purpose?

6. Do we need suffering?

Mikes channel: https://www.youtube.com/channel/UCfQgsKhHjSyRLOp9mnffqVg

RESCRIPT INTERACTIVE PLAYER: https://app.rescript.info/public/share/GVMUXHCqctPkXH8WcYtufFG7FQcdJew_RL_MLgMKU1U

---

TIMESTAMPS:

00:00:00 Introduction & Workout Demo

00:04:15 ASI Timelines & Definitions

00:10:24 The Embodiment Debate

00:18:28 Neutrinos & Abstract Knowledge

00:25:56 Can AI Learn From YouTube?

00:31:25 Diversity of Intelligence

00:36:00 AI Slop & Understanding

00:45:18 The Simulation Argument: Fire & Water

00:58:36 Consciousness & Zombies

01:04:30 Do Reasoning Models Actually Reason?

01:12:00 The Live Learning Problem

01:19:15 Superintelligence & Benevolence

01:28:59 What is True Agency?

01:37:20 Game Theory & The "Kill All Humans" Fallacy

01:48:05 Regulation & The China Factor

01:55:52 Mind Uploading & The Future of Love

02:04:41 Economics of ASI: Will We Be Useless?

02:13:35 The Matrix & The Value of Suffering

02:17:30 Transhumanism & Inequality

02:21:28 Debrief: AI Medical Advice & Final Thoughts

---

REFERENCES:

Paper:

[00:10:45] Alchemy and Artificial Intelligence (Dreyfus)

https://www.rand.org/content/dam/rand/pubs/papers/2006/P3244.pdf

[00:10:55] The Chinese Room Argument (John Searle)

https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf

[00:11:05] The Symbol Grounding Problem (Stephen Harnad)

https://arxiv.org/html/cs/9906002

[00:23:00] Attention Is All You Need

https://arxiv.org/abs/1706.03762

[00:45:00] GPT-4 Technical Report

https://arxiv.org/abs/2303.08774

[01:45:00] Anthropic Agentic Misalignment Paper

https://www.anthropic.com/research/agentic-misalignment

[02:17:45] Retatrutide

https://pubmed.ncbi.nlm.nih.gov/37366315/

Organization:

[00:15:50] CERN

https://home.cern/

[01:05:00] METR Long Horizon Evaluations

https://evaluations.metr.org/

MLST Episode:

[00:23:10] MLST: Llion Jones - Inventors' Remorse

https://www.youtube.com/watch?v=DtePicx_kFY

[00:50:30] MLST: Blaise Agüera y Arcas Interview

https://www.youtube.com/watch?v=rMSEqJ_4EBk

[01:10:00] MLST: David Krakauer

https://www.youtube.com/watch?v=dY46YsGWMIc

Event:

[00:23:40] ARC Prize/Challenge

https://arcprize.org/

Book:

[00:24:45] The Brain Abstracted

https://www.amazon.com/Brain-Abstracted-Simplification-Philosophy-Neuroscience/dp/0262548046

[00:47:55] Pamela McCorduck

https://www.amazon.com/Machines-Who-Think-Artificial-Intelligence/dp/1568812051

[01:23:15] The Singularity Is Nearer (Ray Kurzweil)

https://www.amazon.com/Singularity-Nearer-Ray-Kurzweil-ebook/dp/B08Y6FYJVY

[01:27:35] A Fire Upon The Deep (Vernor Vinge)

https://www.amazon.com/Fire-Upon-Deep-S-F-MASTERWORKS-ebook/dp/B00AVUMIZE/

[02:04:50] Deep Utopia (Nick Bostrom)

https://www.amazon.com/Deep-Utopia-Meaning-Solved-World/dp/1646871642

[02:05:00] Technofeudalism (Yanis Varoufakis)

https://www.amazon.com/Technofeudalism-Killed-Capitalism-Yanis-Varoufakis/dp/1685891241

Visual Context Needed:

[00:29:40] AT-AT Walker (Star Wars)

https://starwars.fandom.com/wiki/All_Terrain_Armored_Transport

Person:

[00:33:15] Andrej Karpathy

https://karpathy.ai/

Video:

[01:40:00] Mike Israetel vs Liron Shapira AI Doom Debate

https://www.youtube.com/watch?v=RaDWSPMdM4o

Company:

[02:26:30] Examine.com

https://examine.com/

We often think of Large Language Models (LLMs) as all-knowing, but as the team reveals, they still struggle with the logic of a second-grader. Why can’t ChatGPT reliably add large numbers? Why does it "hallucinate" the laws of physics? The answer lies in the architecture. This episode explores how *Category Theory* —an ultra-abstract branch of mathematics—could provide the "Periodic Table" for neural networks, turning the "alchemy" of modern AI into a rigorous science.

In this deep-dive exploration, *Andrew Dudzik*, *Petar Velichkovich*, *Taco Cohen*, *Bruno Gavranović*, and *Paul Lessard* join host *Tim Scarfe* to discuss the fundamental limitations of today’s AI and the radical mathematical framework that might fix them.

TRANSCRIPT:

https://app.rescript.info/public/share/LMreunA-BUpgP-2AkuEvxA7BAFuA-VJNAp2Ut4MkMWk

---

Key Insights in This Episode:

* *The "Addition" Problem:* *Andrew Dudzik* explains why LLMs don't actually "know" math—they just recognize patterns. When you change a single digit in a long string of numbers, the pattern breaks because the model lacks the internal "machinery" to perform a simple carry operation.

* *Beyond Alchemy:* deep learning is currently in its "alchemy" phase—we have powerful results, but we lack a unifying theory. Category Theory is proposed as the framework to move AI from trial-and-error to principled engineering. [00:13:49]

* *Algebra with Colors:* To make Category Theory accessible, the guests use brilliant analogies—like thinking of matrices as *magnets with colors* that only snap together when the types match. This "partial compositionality" is the secret to building more complex internal reasoning. [00:09:17]

* *Synthetic vs. Analytic Math:* *Paul Lessard* breaks down the philosophical shift needed in AI research: moving from "Analytic" math (what things are made of) to "Synthetic" math [00:23:41]

---

Why This Matters for AGI

If we want AI to solve the world's hardest scientific problems, it can't just be a "stochastic parrot." It needs to internalize the rules of logic and computation. By imbuing neural networks with categorical priors, researchers are attempting to build a future where AI doesn't just predict the next word—it understands the underlying structure of the universe.

---

TIMESTAMPS:

00:00:00 The Failure of LLM Addition & Physics

00:01:26 Tool Use vs Intrinsic Model Quality

00:03:07 Efficiency Gains via Internalization

00:04:28 Geometric Deep Learning & Equivariance

00:07:05 Limitations of Group Theory

00:09:17 Category Theory: Algebra with Colors

00:11:25 The Systematic Guide of Lego-like Math

00:13:49 The Alchemy Analogy & Unifying Theory

00:15:33 Information Destruction & Reasoning

00:18:00 Pathfinding & Monoids in Computation

00:20:15 System 2 Reasoning & Error Awareness

00:23:31 Analytic vs Synthetic Mathematics

00:25:52 Morphisms & Weight Tying Basics

00:26:48 2-Categories & Weight Sharing Theory

00:28:55 Higher Categories & Emergence

00:31:41 Compositionality & Recursive Folds

00:34:05 Syntax vs Semantics in Network Design

00:36:14 Homomorphisms & Multi-Sorted Syntax

00:39:30 The Carrying Problem & Hopf Fibrations

Petar Veličković (GDM)

https://petar-v.com/

Paul Lessard

https://www.linkedin.com/in/paul-roy-lessard/

Bruno Gavranović

https://www.brunogavranovic.com/

Andrew Dudzik (GDM)

https://www.linkedin.com/in/andrew-dudzik-222789142/

---

REFERENCES:

Model:

[00:01:05] Veo

https://deepmind.google/models/veo/

[00:01:10] Genie

https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/

Paper:

[00:04:30] Geometric Deep Learning Blueprint

https://arxiv.org/abs/2104.13478

https://www.youtube.com/watch?v=bIZB1hIJ4u8

[00:16:45] AlphaGeometry

https://arxiv.org/abs/2401.08312

[00:16:55] AlphaCode

https://arxiv.org/abs/2203.07814

[00:17:05] FunSearch

https://www.nature.com/articles/s41586-023-06924-6

[00:37:00] Attention Is All You Need

https://arxiv.org/abs/1706.03762

[00:43:00] Categorical Deep Learning

https://arxiv.org/abs/2402.15332

Is a car that wins a Formula 1 race the best choice for your morning commute? Probably not. In this sponsored deep dive with Prolific, we explore why the same logic applies to Artificial Intelligence. While models are currently shattering records on technical exams, they often fail the most important test of all: **the human experience.**

Why High Benchmark Scores Don’t Mean Better AI

Joining us are **Andrew Gordon** (Staff Researcher in Behavioral Science) and **Nora Petrova** (AI Researcher) from **Prolific**. They reveal the hidden flaws in how we currently rank AI and introduce a more rigorous, "humane" way to measure whether these models are actually helpful, safe, and relatable for real people.

---

Key Insights in This Episode:

* *The F1 Car Analogy:* Andrew explains why a model that excels at the "Humanities Last Exam" might be a nightmare for daily use. Technical benchmarks often ignore the nuances of human communication and adaptability.

* *The "Wild West" of AI Safety:* As users turn to AI for sensitive topics like mental health, Nora highlights the alarming lack of oversight and the "thin veneer" of safety training—citing recent controversial incidents like Grok-3’s "Mecha Hitler."

* *Fixing the "Leaderboard Illusion":* The team critiques current popular rankings like Chatbot Arena, discussing how anonymous, unstratified voting can lead to biased results and how companies can "game" the system.

* *The Xbox Secret to AI Ranking:* Discover how Prolific uses *TrueSkill*—the same algorithm Microsoft developed for Xbox Live matchmaking—to create a fairer, more statistically sound leaderboard for LLMs.

* *The Personality Gap:* Early data from the **Humane Leaderboard** suggests that while AI is getting smarter, it is actually performing *worse* on metrics like personality, culture, and "sycophancy" (the tendency for models to become annoying "people-pleasers").

---

About the HUMAINE Leaderboard

Moving beyond simple "A vs. B" testing, the researchers discuss their new framework that samples participants based on *census data* (Age, Ethnicity, Political Alignment). By using a representative sample of the general public rather than just tech enthusiasts, they are building a standard that reflects the values of the real world.

*Are we building models for benchmarks, or are we building them for humans? It’s time to change the scoreboard.*

Rescript link:

https://app.rescript.info/public/share/IDqwjY9Q43S22qSgL5EkWGFymJwZ3SVxvrfpgHZLXQc

---

TIMESTAMPS:

00:00:00 Introduction & The Benchmarking Problem

00:01:58 The Fractured State of AI Evaluation

00:03:54 AI Safety & Interpretability

00:05:45 Bias in Chatbot Arena

00:06:45 Prolific's Three Pillars Approach

00:09:01 TrueSkill Ranking & Efficient Sampling

00:12:04 Census-Based Representative Sampling

00:13:00 Key Findings: Culture, Personality & Sycophancy

---

REFERENCES:

Paper:

[00:00:15] MMLU

https://arxiv.org/abs/2009.03300

[00:05:10] Constitutional AI

https://arxiv.org/abs/2212.08073

[00:06:45] The Leaderboard Illusion

https://arxiv.org/abs/2504.20879

[00:09:41] HUMAINE Framework Paper

https://huggingface.co/blog/ProlificAI/humaine-framework

Company:

[00:00:30] Prolific

https://www.prolific.com

[00:01:45] Chatbot Arena

https://lmarena.ai/

Person:

[00:00:35] Andrew Gordon

https://www.linkedin.com/in/andrew-gordon-03879919a/

[00:00:45] Nora Petrova

https://www.linkedin.com/in/nora-petrova/

Event:

Algorithm:

[00:09:01] Microsoft TrueSkill

https://www.microsoft.com/en-us/research/project/trueskill-ranking-system/

Leaderboard:

[00:09:21] Prolific HUMAINE Leaderboard

https://www.prolific.com/humaine

[00:09:31] HUMAINE HuggingFace Space

https://huggingface.co/spaces/ProlificAI/humaine-leaderboard

[00:10:21] Prolific AI Leaderboard Portal

https://www.prolific.com/leaderboard

Dataset:

[00:09:51] Prolific Social Reasoning RLHF Dataset

https://huggingface.co/datasets/ProlificAI/social-reasoning-rlhf

Organization:

[00:10:31] MLCommons

https://mlcommons.org/

What if everything we think we know about AI understanding is wrong? Is compression the key to intelligence? Or is there something more—a leap from memorization to true abstraction?

In this fascinating conversation, we sit down with **Professor Yi Ma**—world-renowned expert in deep learning, IEEE/ACM Fellow, and author of the groundbreaking new book *Learning Deep Representations of Data Distributions*. Professor Ma challenges our assumptions about what large language models actually do, reveals why 3D reconstruction isn't the same as understanding, and presents a unified mathematical theory of intelligence built on just two principles: **parsimony** and **self-consistency**.

**SPONSOR MESSAGES START**

—

Prolific - Quality data. From real people. For faster breakthroughs.

https://www.prolific.com/?utm_source=mlst

—

cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy

Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst

Submit investment deck: https://cyber.fund/contact?utm_source=mlst

—

**END**

Key Insights:

**LLMs Don't Understand—They Memorize**

Language models process text (*already* compressed human knowledge) using the same mechanism we use to learn from raw data.

**The Illusion of 3D Vision**

Sora and NeRFs etc that can reconstruct 3D scenes still fail miserably at basic spatial reasoning

**"All Roads Lead to Rome"**

Why adding noise is *necessary* for discovering structure.

**Why Gradient Descent Actually Works**

Natural optimization landscapes are surprisingly smooth—a "blessing of dimensionality"

**Transformers from First Principles**

Transformer architectures can be mathematically derived from compression principles

—

INTERACTIVE AI TRANSCRIPT PLAYER w/REFS (ReScript):

https://app.rescript.info/public/share/Z-dMPiUhXaeMEcdeU6Bz84GOVsvdcfxU_8Ptu6CTKMQ

About Professor Yi Ma

Yi Ma is the inaugural director of the School of Computing and Data Science at Hong Kong University and a visiting professor at UC Berkeley.

https://people.eecs.berkeley.edu/~yima/

https://scholar.google.com/citations?user=XqLiBQMAAAAJ&hl=en

https://x.com/YiMaTweets

**Slides from this conversation:**

https://www.dropbox.com/scl/fi/sbhbyievw7idup8j06mlr/slides.pdf?rlkey=7ptovemezo8bj8tkhfi393fh9&dl=0

**Related Talks by Professor Ma:**

- Pursuing the Nature of Intelligence (ICLR): https://www.youtube.com/watch?v=LT-F0xSNSjo

- Earlier talk at Berkeley: https://www.youtube.com/watch?v=TihaCUjyRLM

TIMESTAMPS:

00:00:00 Introduction

00:02:08 The First Principles Book & Research Vision

00:05:21 Two Pillars: Parsimony & Consistency

00:09:50 Evolution vs. Learning: The Compression Mechanism

00:14:36 LLMs: Memorization Masquerading as Understanding

00:19:55 The Leap to Abstraction: Empirical vs. Scientific

00:27:30 Platonism, Deduction & The ARC Challenge

00:35:57 Specialization & The Cybernetic Legacy

00:41:23 Deriving Maximum Rate Reduction

00:48:21 The Illusion of 3D Understanding: Sora & NeRF

00:54:26 All Roads Lead to Rome: The Role of Noise

00:59:56 All Roads Lead to Rome: The Role of Noise

01:00:14 Benign Non-Convexity: Why Optimization Works

01:06:35 Double Descent & The Myth of Overfitting

01:14:26 Self-Consistency: Closed-Loop Learning

01:21:03 Deriving Transformers from First Principles

01:30:11 Verification & The Kevin Murphy Question

01:34:11 CRATE vs. ViT: White-Box AI & Conclusion

REFERENCES:

Book:

[00:03:04] Learning Deep Representations of Data Distributions

https://ma-lab-berkeley.github.io/deep-representation-learning-book/

[00:18:38] A Brief History of Intelligence

https://www.amazon.co.uk/BRIEF-HISTORY-INTELLIGEN-HB-Evolution/dp/0008560099

[00:38:14] Cybernetics

https://mitpress.mit.edu/9780262730099/cybernetics/

Book (Yi Ma):

[00:03:14] 3-D Vision book

https://link.springer.com/book/10.1007/978-0-387-21779-6

refs on ReScript link/YT

Pedro Domingos, author of the bestselling book "The Master Algorithm," introduces his latest work: Tensor Logic - a new programming language he believes could become the fundamental language for artificial intelligence.

Think of it like this: Physics found its language in calculus. Circuit design found its language in Boolean logic. Pedro argues that AI has been missing its language - until now.

**SPONSOR MESSAGES START**

—

Build your ideas with AI Studio from Google - http://ai.studio/build

—

Prolific - Quality data. From real people. For faster breakthroughs.

https://www.prolific.com/?utm_source=mlst

—

cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy

Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst

Submit investment deck: https://cyber.fund/contact?utm_source=mlst

—

**END**

Current AI is split between two worlds that don't play well together:

Deep Learning (neural networks, transformers, ChatGPT) - great at learning from data, terrible at logical reasoning

Symbolic AI (logic programming, expert systems) - great at logical reasoning, terrible at learning from messy real-world data

Tensor Logic unifies both. It's a single language where you can:

Write logical rules that the system can actually learn and modify

Do transparent, verifiable reasoning (no hallucinations)

Mix "fuzzy" analogical thinking with rock-solid deduction

INTERACTIVE TRANSCRIPT:

https://app.rescript.info/public/share/NP4vZQ-GTETeN_roB2vg64vbEcN7isjJtz4C86WSOhw

TOC:

00:00:00 - Introduction

00:04:41 - What is Tensor Logic?

00:09:59 - Tensor Logic vs PyTorch & Einsum

00:17:50 - The Master Algorithm Connection

00:20:41 - Predicate Invention & Learning New Concepts

00:31:22 - Symmetries in AI & Physics

00:35:30 - Computational Reducibility & The Universe

00:43:34 - Technical Details: RNN Implementation

00:45:35 - Turing Completeness Debate

00:56:45 - Transformers vs Turing Machines

01:02:32 - Reasoning in Embedding Space

01:11:46 - Solving Hallucination with Deductive Modes

01:16:17 - Adoption Strategy & Migration Path

01:21:50 - AI Education & Abstraction

01:24:50 - The Trillion-Dollar Waste

REFS

Tensor Logic: The Language of AI [Pedro Domingos]

https://arxiv.org/abs/2510.12269

The Master Algorithm [Pedro Domingos]

https://www.amazon.co.uk/Master-Algorithm-Ultimate-Learning-Machine/dp/0241004543

Einsum is All you Need (TIM ROCKTÄSCHEL)

https://rockt.ai/2018/04/30/einsum

https://www.youtube.com/watch?v=6DrCq8Ry2cw

Autoregressive Large Language Models are Computationally Universal (Dale Schuurmans et al - GDM)

https://arxiv.org/abs/2410.03170

Memory Augmented Large Language Models are Computationally Universal [Dale Schuurmans]

https://arxiv.org/pdf/2301.04589

On the computational power of NNs [95/Siegelmann]

https://binds.cs.umass.edu/papers/1995_Siegelmann_JComSysSci.pdf

Sebastian Bubeck

https://www.reddit.com/r/OpenAI/comments/1oacp38/openai_researcher_sebastian_bubeck_falsely_claims/

I am a strange loop - Hofstadter

https://www.amazon.co.uk/Am-Strange-Loop-Douglas-Hofstadter/dp/0465030793

Stephen Wolfram

https://www.youtube.com/watch?v=dkpDjd2nHgo

The Complex World: An Introduction to the Foundations of Complexity Science [David C. Krakauer]

https://www.amazon.co.uk/Complex-World-Introduction-Foundations-Complexity/dp/1947864629

Geometric Deep Learning

https://www.youtube.com/watch?v=bIZB1hIJ4u8

Andrew Wilson (NYU)

https://www.youtube.com/watch?v=M-jTeBCEGHc

Yi Ma

https://www.patreon.com/posts/yi-ma-scientific-141953348

Roger Penrose - road to reality

https://www.amazon.co.uk/Road-Reality-Complete-Guide-Universe/dp/0099440687

Artificial Intelligence: A Modern Approach [Russel and Norvig]

https://www.amazon.co.uk/Artificial-Intelligence-Modern-Approach-Global/dp/1292153962

The Transformer architecture (which powers ChatGPT and nearly all modern AI) might be trapping the industry in a localized rut, preventing us from finding true intelligent reasoning, according to the person who co-invented it. Llion Jones and Luke Darlow, key figures at the research lab Sakana AI, join the show to make this provocative argument, and also introduce new research which might lead the way forwards.

**SPONSOR MESSAGES START**

—

Build your ideas with AI Studio from Google - http://ai.studio/build

—

Tufa AI Labs is hiring ML Research Engineers https://tufalabs.ai/

—

cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy

Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst

Submit investment deck: https://cyber.fund/contact?utm_source=mlst

—

**END**

The "Spiral" Problem – Llion uses a striking visual analogy to explain what current AI is missing. If you ask a standard neural network to understand a spiral shape, it solves it by drawing tiny straight lines that just happen to look like a spiral. It "fakes" the shape without understanding the concept of spiraling.

Introducing the Continuous Thought Machine (CTM) Luke Darlow deep dives into their solution: a biology-inspired model that fundamentally changes how AI processes information.

The Maze Analogy: Luke explains that standard AI tries to solve a maze by staring at the whole image and guessing the entire path instantly. Their new machine "walks" through the maze step-by-step.

Thinking Time: This allows the AI to "ponder." If a problem is hard, the model can naturally spend more time thinking about it before answering, effectively allowing it to correct its own mistakes and backtrack—something current Language Models struggle to do genuinely.

https://sakana.ai/

https://x.com/YesThisIsLion

https://x.com/LearningLukeD

TRANSCRIPT:

https://app.rescript.info/public/share/crjzQ-Jo2FQsJc97xsBdfzfOIeMONpg0TFBuCgV2Fu8

TOC:

00:00:00 - Stepping Back from Transformers

00:00:43 - Introduction to Continuous Thought Machines (CTM)

00:01:09 - The Changing Atmosphere of AI Research

00:04:13 - Sakana’s Philosophy: Research Freedom

00:07:45 - The Local Minimum of Large Language Models

00:18:30 - Representation Problems: The Spiral Example

00:29:12 - Technical Deep Dive: CTM Architecture

00:36:00 - Adaptive Computation & Maze Solving

00:47:15 - Model Calibration & Uncertainty

01:00:43 - Sudoku Bench: Measuring True Reasoning

REFS:

Why Greatness Cannot be planned [Kenneth Stanley]

https://www.amazon.co.uk/Why-Greatness-Cannot-Planned-Objective/dp/3319155237

https://www.youtube.com/watch?v=lhYGXYeMq_E

The Hardware Lottery [Sara Hooker]

https://arxiv.org/abs/2009.06489

https://www.youtube.com/watch?v=sQFxbQ7ade0

Continuous Thought Machines [Luke Darlow et al / Sakana]

https://arxiv.org/abs/2505.05522

https://sakana.ai/ctm/

LSTM: The Comeback Story? [Prof. Sepp Hochreiter]

https://www.youtube.com/watch?v=8u2pW2zZLCs

Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis [Kumar/Stanley]

https://arxiv.org/pdf/2505.11581

A Spline Theory of Deep Networks [Randall Balestriero]

https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf

https://www.youtube.com/watch?v=86ib0sfdFtw

https://www.youtube.com/watch?v=l3O2J3LMxqI

On the Biology of a Large Language Model [Anthropic, Jack Lindsey et al]

https://transformer-circuits.pub/2025/attribution-graphs/biology.html

The ARC Prize 2024 Winning Algorithm [Daniel Franzen and Jan Disselhoff] “The ARChitects”

https://www.youtube.com/watch?v=mTX_sAq--zY

Neural Turing Machine [Graves]

https://arxiv.org/pdf/1410.5401

Adaptive Computation Time for Recurrent Neural Networks [Graves]

https://arxiv.org/abs/1603.08983

Sudoko Bench [Sakana]

https://pub.sakana.ai/sudoku/

Ever wonder where AI models actually get their "intelligence"? We reveal the dirty secret of Silicon Valley: behind every impressive AI system are thousands of real humans providing crucial data, feedback, and expertise.Guest: Phelim Bradley, CEO and Co-founder of ProlificPhelim Bradley runs Prolific, a platform that connects AI companies with verified human experts who help train and evaluate their models. Think of it as a sophisticated marketplace matching the right human expertise to the right AI task - whether that's doctors evaluating medical chatbots or coders reviewing AI-generated software.Prolific: https://prolific.com/?utm_source=mlsthttps://uk.linkedin.com/in/phelim-bradley-84300826The discussion dives into:**The human data pipeline**: How AI companies rely on human intelligence to train, refine, and validate their models - something rarely discussed openly**Quality over quantity**: Why paying humans well and treating them as partners (not commodities) produces better AI training data**The matching challenge**: How Prolific solves the complex problem of finding the right expert for each specific task, similar to matching Uber drivers to riders but with deep expertise requirements**Future of work**: What it means when human expertise becomes an on-demand service, and why this might actually create more opportunities rather than fewer**Geopolitical implications**: Why the centralization of AI development in US tech companies should concern Europe and the UK

"What is life?" - asks Chris Kempes, a professor at the Santa Fe Institute.

Chris explains that scientists are moving beyond a purely Earth-based, biological view and are searching for a universal theory of life that could apply to anything, anywhere in the universe. He proposes that things we don't normally consider "alive"—like human culture, language, or even artificial intelligence; could be seen as life forms existing on different "substrates".

To understand this, Chris presents a fascinating three-level framework:

- Materials: The physical stuff life is made of. He argues this could be incredibly diverse across the universe, and we shouldn't expect alien life to share our biochemistry.

- Constraints: The universal laws of physics (like gravity or diffusion) that all life must obey, regardless of what it's made of. This is where different life forms start to look more similar.

- Principles: At the highest level are abstract principles like evolution and learning. Chris suggests these computational or "optimization" rules are what truly define a living system.

A key idea is "convergence" – using the example of the eye. It's such a complex organ that you'd think it evolved only once. However, eyes evolved many separate times across different species. This is because the physics of light provides a clear "target", and evolution found similar solutions to the problem of seeing, even with different starting materials.

**SPONSOR MESSAGES**

—

Prolific - Quality data. From real people. For faster breakthroughs.

https://www.prolific.com/?utm_source=mlst

—

Check out NotebookLM from Google here - https://notebooklm.google.com/ - it’s really good for doing research directly from authoritative source material, minimising hallucinations.

—

cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy

Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst

Submit investment deck: https://cyber.fund/contact?utm_source=mlst

—

Prof. Chris Kempes:

https://www.santafe.edu/people/profile/chris-kempes

TRANSCRIPT:

https://app.rescript.info/public/share/Y2cI1i0nX_-iuZitvlguHvaVLQTwPX1Y_E1EHxV0i9I

TOC:

00:00:00 - Introduction to Chris Kempes and the Santa Fe Institute

00:02:28 - The Three Cultures of Science

00:05:08 - What Makes a Good Scientific Theory?

00:06:50 - The Universal Theory of Life

00:09:40 - The Role of Material in Life

00:12:50 - A Hierarchy for Understanding Life

00:13:55 - How Life Diversifies and Converges

00:17:53 - Adaptive Processes and Defining Life

00:19:28 - Functionalism, Memes, and Phylogenies

00:22:58 - Convergence at Multiple Levels

00:25:45 - The Possibility of Simulating Life

00:28:16 - Intelligence, Parasitism, and Spectrums of Life

00:32:39 - Phase Changes in Evolution

00:36:16 - The Separation of Matter and Logic

00:37:21 - Assembly Theory and Quantifying Complexity

REFS:

Developing a predictive science of the biosphere requires the integration of scientific cultures [Kempes et al]

https://www.pnas.org/doi/10.1073/pnas.2209196121

Seeing with an extra sense (“Dangerous prediction”) [Rob Phillips]

https://www.sciencedirect.com/science/article/pii/S0960982224009035

The Multiple Paths to Multiple Life [Christopher P. Kempes & David C. Krakauer]

https://link.springer.com/article/10.1007/s00239-021-10016-2

The Information Theory of Individuality [David Krakauer et al]

https://arxiv.org/abs/1412.2447

Minds, Brains and Programs [Searle]

https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf

The error threshold

https://www.sciencedirect.com/science/article/abs/pii/S0168170204003843

Assembly theory and its relationship with computational complexity [Kempes et al]

https://arxiv.org/abs/2406.12176

Blaise Agüera y Arcas explores some mind-bending ideas about what intelligence and life really are—and why they might be more similar than we think (filmed at ALIFE conference, 2025 - https://2025.alife.org/).

Life and intelligence are both fundamentally computational (he says). From the very beginning, living things have been running programs. Your DNA? It's literally a computer program, and the ribosomes in your cells are tiny universal computers building you according to those instructions.

**SPONSOR MESSAGES**

—

Prolific - Quality data. From real people. For faster breakthroughs.

https://www.prolific.com/?utm_source=mlst

—

cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy

Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++

Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst

Submit investment deck: https://cyber.fund/contact?utm_source=mlst

—

Blaise argues that there is more to evolution than random mutations (like most people think). The secret to increasing complexity is *merging* i.e. when different organisms or systems come together and combine their histories and capabilities.

Blaise describes his "BFF" experiment where random computer code spontaneously evolved into self-replicating programs, showing how purpose and complexity can emerge from pure randomness through computational processes.

https://en.wikipedia.org/wiki/Blaise_Ag%C3%BCera_y_Arcas

https://x.com/blaiseaguera?lang=en

TRANSCRIPT:

https://app.rescript.info/public/share/VX7Gktfr3_wIn4Bj7cl9StPBO1MN4R5lcJ11NE99hLg

TOC:

00:00:00 Introduction - New book "What is Intelligence?"

00:01:45 Life as computation - Von Neumann's insights

00:12:00 BFF experiment - How purpose emerges

00:26:00 Symbiogenesis and evolutionary complexity

00:40:00 Functionalism and consciousness

00:49:45 AI as part of collective human intelligence

00:57:00 Comparing AI and human cognition

REFS:

What is intelligence [Blaise Agüera y Arcas]

https://whatisintelligence.antikythera.org/ [Read free online, interactive rich media]

https://mitpress.mit.edu/9780262049955/what-is-intelligence/ [MIT Press]

Large Language Models and Emergence: A Complex Systems Perspective

https://arxiv.org/abs/2506.11135

Our first Noam Chomsky MLST interview

https://www.youtube.com/watch?v=axuGfh4UR9Q

Chance and Necessity [Jacques Monod]

https://monoskop.org/images/9/99/Monod_Jacques_Chance_and_Necessity.pdf

Wonderful Life: The Burgess Shale and the History of Nature [Stephen Jay Gould]

https://www.amazon.co.uk/Wonderful-Life-Burgess-Nature-History/dp/0099273454

The major evolutionary transitions [E Szathmáry, J M Smith]

https://wiki.santafe.edu/images/0/0e/Szathmary.MaynardSmith_1995_Nature.pdf

Don't Sleep, There Are Snakes: Life and Language in the Amazonian Jungle [Dan Everett]

https://www.amazon.com/Dont-Sleep-There-Are-Snakes/dp/0307386120

The Nature of Technology: What It Is and How It Evolves [W. Brian Arthur]

https://www.amazon.com/Nature-Technology-What-How-Evolves-ebook/dp/B002RI9W16/

The MANIAC [Benjamin Labatut]

https://www.amazon.com/MANIAC-Benjam%C3%ADn-Labatut/dp/1782279814

When We Cease to Understand the World [Benjamin Labatut]

https://www.amazon.com/When-We-Cease-Understand-World/dp/1681375664/

The Boys in the Boat [Dan Brown]

https://www.amazon.com/Boys-Boat-Americans-Berlin-Olympics/dp/0143125478

[Petter Johansson] (Split brain)

https://www.lucs.lu.se/fileadmin/user_upload/lucs/2011/01/Johansson-et-al.-2006-How-Something-Can-Be-Said-About-Telling-More-Than-We-Can-Know.pdf

If Anyone Builds It, Everyone Dies [Eliezer Yudkowsky, Nate Soares]

https://www.amazon.com/Anyone-Builds-Everyone-Dies-Superhuman/dp/0316595640

The science of cycology

https://link.springer.com/content/pdf/10.3758/bf03195929.pdf