This episode explores KumoRFM, a 2025 proposal for a foundation model that can perform in-context learning directly on relational databases, aiming to handle tasks like churn prediction, fraud detection, recommendation, and forecasting without training a separate model for each schema and label. It explains how the approach represents warehouse data as heterogeneous graphs of rows and foreign-key relationships, using attention over local relational neighborhoods instead of flattening everything into handcrafted feature tables. The discussion focuses on the paper’s strongest claim, zero-shot transfer, and carefully separates true inference-time generalization from easier settings like continued pretraining on the target database or later fine-tuning on the target task. Listeners would find it interesting because the episode gets precise about what this system could change in enterprise ML, while also surfacing the practical caveats around task specification, temporal leakage, infrastructure cost, and how literal the any database, any task promise really is.

Sources:

1. KumoRFM for In-Context Relational Learning

https://kumo.ai/research/kumo_relational_foundation_model.pdf

2. Modeling Relational Data with Graph Convolutional Networks — Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Max Welling, et al., 2017

https://scholar.google.com/scholar?q=Modeling+Relational+Data+with+Graph+Convolutional+Networks

3. Heterogeneous Graph Transformer — Ziniu Hu, Yuxiao Dong, Kuansan Wang, Yizhou Sun, 2020

https://scholar.google.com/scholar?q=Heterogeneous+Graph+Transformer

4. Relational Deep Learning: Graph Representation Learning on Relational Databases — Matthias Fey, Weihua Hu, Kexin Huang, Jan Eric Lenssen, Jure Leskovec, et al., 2023

https://scholar.google.com/scholar?q=Relational+Deep+Learning%3A+Graph+Representation+Learning+on+Relational+Databases

5. Relational Graph Transformer — Vijay Prakash Dwivedi, Sri Jaladi, Yangyi Shen, Federico Lopez, Matthias Fey, Jure Leskovec, et al., 2025 (ICLR 2026)

https://scholar.google.com/scholar?q=Relational+Graph+Transformer

6. One Model to Rule them All: Towards Zero-Shot Learning for Databases — Benjamin Hilprecht, Carsten Binnig, 2021

https://scholar.google.com/scholar?q=One+Model+to+Rule+them+All%3A+Towards+Zero-Shot+Learning+for+Databases

7. Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction — Benjamin Hilprecht, Carsten Binnig, 2022

https://scholar.google.com/scholar?q=Zero-Shot+Cost+Models+for+Out-of-the-box+Learned+Cost+Prediction

8. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second — Noah Hollmann, Samuel Muller, Katharina Eggensperger, Frank Hutter, 2022 (ICLR 2023)

https://scholar.google.com/scholar?q=TabPFN%3A+A+Transformer+That+Solves+Small+Tabular+Classification+Problems+in+a+Second

9. Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data — Rishabh Ranjan, Valter Hudovernik, Mark Znidar, Carlos Guestrin, Jure Leskovec, et al., 2025 (ICLR 2026)

https://scholar.google.com/scholar?q=Relational+Transformer%3A+Toward+Zero-Shot+Foundation+Models+for+Relational+Data

10. Relational In-Context Learning via Synthetic Pre-training with Structural Prior — Yanbo Wang, Jiaxuan You, Chuan Shi, Muhan Zhang, 2026

https://scholar.google.com/scholar?q=Relational+In-Context+Learning+via+Synthetic+Pre-training+with+Structural+Prior

11. OpenRFM: Dissecting Relational In-Context Learning — Zhikai Chen et al., 2026

https://scholar.google.com/scholar?q=OpenRFM%3A+Dissecting+Relational+In-Context+Learning

12. KumoRFM-2: Scaling Foundation Models for Relational Learning — Valter Hudovernik et al., 2026

https://scholar.google.com/scholar?q=KumoRFM-2%3A+Scaling+Foundation+Models+for+Relational+Learning

13. Retrieval & Fine-Tuning for In-Context Tabular Models — Valentin Thomas et al., 2024

https://scholar.google.com/scholar?q=Retrieval+%26+Fine-Tuning+for+In-Context+Tabular+Models

14. Scalable In-Context Learning on Tabular Data via Retrieval-Augmented Large Language Models — Xumeng Wen et al., 2025

https://scholar.google.com/scholar?q=Scalable+In-Context+Learning+on+Tabular+Data+via+Retrieval-Augmented+Large+Language+Models

15. Exploring Fine-Tuning for Tabular Foundation Models — Aditya Tanna et al., 2026

https://scholar.google.com/scholar?q=Exploring+Fine-Tuning+for+Tabular+Foundation+Models

16. AI Post Transformers: Predictive Query Language for Relational Databases — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-06-08-predictive-query-language-for-relational-103e68.mp3

17. AI Post Transformers: KumoRFM-2 for Relational Learning at Scale — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-06-08-kumorfm-2-for-relational-learning-at-sca-13c996.mp3

18. AI Post Transformers: Why LightGBM Made Boosted Trees Fast — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-05-05-why-lightgbm-made-boosted-trees-fast-286a89.mp3

19. AI Post Transformers: How Induction Heads Emerge in Transformers — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-05-03-how-induction-heads-emerge-in-transforme-a7bfcb.mp3