Most ML training bugs don't announce themselves. They hide in floating-point rounding, batch distribution drift, and concurrency issues that only emerge across thousands of GPUs. The hard part isn't writing the algorithm — it's building infrastructure disciplined enough to tell the difference between a numerically equivalent optimization and a quietly corrupted model.
We speak with Tianshu Yu, member of the technical staff at Liquid AI, about the real work of building reliable ML systems at scale. Tianshu previously led reinforcement learning infrastructure at ByteDance, where he was a core contributor to VERL — one of the most widely used open-source RL training frameworks. He now works on pre-training and architecture design for edge-deployed vision language models.
The conversation covers the three-phase structure of reinforcement learning, why checkpoints serve debugging, fault recovery, and model selection simultaneously, and how capability collapse emerges when data distributions concentrate during fine-tuning. We also get into what changes when you design models for edge hardware: the shift from attention to convolution-dominant ops, latency budgets as a first-class architectural constraint, and why a 350M parameter model has meaningfully different engineering requirements than a frontier model.
KEY TAKEAWAYS
00:00 Introduction.
03:10 Reinforcement learning trains models against reward signals rather than ground truth, requiring the system to generate its own training data through rollout.
06:00 Capability collapse is a real risk during RL fine-tuning: optimizing on a concentrated data distribution can degrade unrelated model capabilities, including multilingual performance.
09:00 Checkpoints serve three distinct purposes in large-scale training: monitoring quality curves, enabling fault recovery after hardware or software failures, and supporting model selection across training steps.
13:30 A smooth reward curve is not sufficient evidence of a healthy training run — benchmark evaluations at checkpoint boundaries frequently reveal instabilities invisible in aggregate metrics.
17:30 Training system bugs decompose into three categories: data pipeline issues, infrastructure logic errors, and algorithmic design flaws — and disentangling them requires iterative hypothesis testing, not a single diagnostic pass.
20:30 ML infrastructure correctness is fundamentally different from software correctness: because everything operates in floating point, a mathematically equivalent optimization can still produce divergent numerics.
24:00 Eval benchmarks are a late and noisy signal — 30-minute evaluation cycles and LLM-judge variance make them unsuitable as a primary debugging instrument during active training.
32:00 Open-source training frameworks like VERL provide two compounding benefits: a battle-tested abstraction layer that separates algorithm from infrastructure, and community validation that reduces the prior probability of fundamental kernel bugs.
39:00 Multimodal architectures unify vision and text by mapping continuous image patches into the same embedding space as discrete text tokens, enabling a shared transformer backbone across modalities.
45:00 Edge model architecture is constrained not just by parameter count but by hardware topology: devices without large tensor cores favor convolution-dominant operations over attention, requiring architecture decisions to be made alongside inference profiling.
Thanks for listening to “Hello Agent!: The podcast at the intersection of data & agents.” If you loved this episode, let us know with a 5-star review! Remember to subscribe so you don’t miss an episode. To learn more about Redpanda, visit redpanda.com
RESOURCES MENTIONED
VERL
github.com/verl-project/verl
Robust LLM Training Infrastructure at ByteDance
arxiv.org/abs/2509.16293
#RealTimeData #DataStreaming #Redpanda
