AI in Tokenization: Byte‑Latent Transformers, H‑Net & Bolmo Hosted by Nathan Rigoni | Guest: Jordan Conragan – Research Engineer at 11 Labs (formerly a Lockheed Martin colleague)

How can we make language models treat every byte of text as efficiently as a byte‑level transformer, and what does that mean for the future of AI‑driven computation? Could dynamic patching and entropy‑based chunking finally solve the tokenization bottleneck that limits model size, speed, and math reasoning?

What you will learn

• The motivation behind Byte‑Latent Transformers and why plain byte‑to‑token mapping explodes sequence lengths.
• How entropy‑driven patching groups low‑information bytes into larger tokens, shrinking effective sequence length while preserving information density.
• The design of H‑Net’s hierarchical dynamic chunking: a learned, end‑to‑end routing module that replaces a separate tokenizer‑training step.
• Bolmo’s approach of using a non‑causal LSTM boundary predictor to adapt existing LLMs for byte‑level input with minimal compute.
• Practical implications for math‑heavy workloads, model compression, and the bits‑per‑parameter efficiency debate.

Resources mentioned

• Byte‑Latent Transformer paper – “Byte‑Latent Transformers: Patches Scale Better Than Tokens.” https://arxiv.org/abs/2412.09871
• H‑Net (Hierarchical Net) paper – “Dynamic Chunking for End‑to‑End Hierarchical Sequence Modeling.” https://github.com/goombalab/hnet
• Bolmo (Allen AI) blog post on OLMO 3 and its byte‑level adaptation. https://allenai.org/blog/bolmo
• 11 Labs Scribe V2 (mentioned as Jordan’s current project).

Why this episode matters
Tokenization is the hidden cost driver behind today’s trillion‑parameter models. By moving from fixed sub‑word vocabularies to entropy‑aware, dynamically sized patches, we can dramatically reduce sequence length, lower compute budgets, and improve numerical reasoning—key steps toward making large language models more accessible, faster, and better at math. The discussion also surfaces the trade‑offs of maintaining a separate tokenizer versus learning chunking jointly with the model, a design choice that will shape the next generation of efficient AI systems.

Subscribe for more deep dives, visit www.phronesis-analytics.com, or email [email protected].

Keywords: Byte latent transformer, byte‑level tokenization, entropy‑based patching, dynamic chunking, H‑Net, Bolmo, model compression, bits‑per‑parameter, AI efficiency, mathematical reasoning.