Daily Tech Feed: From the Labs

Now I have everything. Composing the show notes:

Here are the show notes:

Why it matters. Every LLM-based text-to-speech system shipping today carries a structural flaw: text tokens and audio frames move at incompatible speeds inside the same model, forcing engineers to choose between reliability, quality, and inference cost. Hume AI's TADA: A Generative Framework for Speech Modeling via Text-Acoustic Dual Alignment eliminates the mismatch entirely — enforcing strict one-to-one synchronization between text tokens and continuous acoustic vectors, producing zero content hallucinations across 1,000+ test samples and running at 5× the throughput of comparable systems.

Hume AI. Hume AI is a voice AI infrastructure company building training data pipelines, evaluation systems, and reinforcement learning tooling for frontier labs and enterprises. TADA ships fully open: code, pre-trained weights, the audio tokenizer, and vocoder decoder — a 1B English model and a 3B multilingual model covering eight languages, both built on Llama architecture. Resources: arXiv paper, GitHub repository, HuggingFace demo, PyPI package, Hume AI blog post. Trung Dang also holds an affiliation with Dartmouth College.

The Researchers. The paper's authors are Trung Dang, Sharath Rao, Ananya Gupta, Christopher Gagne, Panagiotis Tzirakis, Alice Baird, Jakub Piotr Cłapa, Peter Chin, and Alan Cowen — founder and CEO of Hume AI, whose prior research on the taxonomy of human emotions underpins the company's focus on expressive, emotionally-aware speech generation. Panagiotis Tzirakis and Alice Baird are established researchers in affective computing and speech-based emotion recognition.

Key Technical Concepts. Prior LLM-TTS architectures — including VALL-E (Microsoft), SoundStorm (Google DeepMind), and systems built on EnCodec discrete tokens — all operate at fixed acoustic frame rates (12.5–75 Hz), producing speech sequences 5–35× longer than their text inputs and requiring either reduced frame rates (losing expressiveness) or intermediate semantic token layers (adding complexity and failure points). TADA replaces discrete acoustic tokens with continuous vectors aligned one-to-one to text tokens via a learned encoder-aligner pair, collapsing the sequence length disparity entirely. A flow matching head decodes LLM hidden states into waveforms via a vocoder, enabling high-fidelity reconstruction without the computational overhead of fixed-rate codecs. The paper introduces Speech Free Guidance — blending logits from text-only and text-speech inference modes — to close the modality gap that appears when co-generating text and speech simultaneously. The architecture opens direct paths for extension: new modalities via tokenizer adaptation, long-context drift mitigation, and assistant-scenario fine-tuning.

Daily Tech Feed: From the Labs is available on Apple Podcasts, Spotify, and wherever fine podcasts are distributed. Visit us at pod.c457.org for all our shows. New episodes daily.

On March 7th, 2026, Andrej Karpathy released autoresearch — a 630-line Python repo that lets an AI agent run autonomous ML experiments overnight. The agent modifies training code, runs 5-minute experiments, keeps improvements, discards failures, and repeats. ~100 cycles while you sleep. Karpathy watched val_bpb drop from 1.0 to 0.97 without touching anything. Shopify CEO Tobi Lutke adapted it the same night and got a 19% improvement — with the smaller agent-optimized model eventually outperforming a larger manually configured one. The engineering task has shifted: you're not tuning the model anymore, you're writing the instructions that tell the agent how to tune the model.

karpathy/autoresearch

Three files:

The 630-line constraint is intentional: the entire codebase fits in a model's context window, so the agent can reason about the whole file at once.

Karpathy describes program.md as the "research org code" — you're not programming the model, you're programming the organization that does the research. The human's job shifts from running experiments to writing the instructions that tell the agent how to run experiments well.

His README opens with fictional lore: the agents are now in the 10,205th generation of the codebase, "a self-modifying binary that has grown beyond human comprehension." He's joking. The repo exists and you can clone it tonight.

Rich Sutton published a 1,200-word essay in 2019 and was largely dismissed. Then the past five years vindicated every word of it. Now Sutton is making a second claim: that the LLM paradigm — train in a lab, freeze weights, deploy — is structurally limited in the same way the knowledge-based approaches he criticized were limited. He and John Carmack are building the alternative at Keen Technologies, targeting a genuine AGI prototype by 2030. Whether they succeed or fail, the argument deserves serious examination. The track record says so.

No

No paper.json for 0026. The script is entirely about FlashAttention-4. Let me look at the full script for researcher/institution details, then search for the paper.

Why it matters. The economics of vulnerability discovery just broke. In twenty minutes, Claude Opus 4.6 found a novel use-after-free memory bug in Firefox — one of the most audited codebases on the internet, backed by millions of CPU hours of continuous fuzzing. That single result is a waypoint on a documented curve: from GPT-4 exploiting 87% of known one-day vulnerabilities with 91 lines of LangChain code in 2024, to Anthropic's red team finding 500+ high-severity zero-days in well-maintained open-source software in early 2026, to a live collaboration with Mozilla that found 22 Firefox vulnerabilities in two weeks. We are in a window: finding is democratized, reliable exploitation still has friction. This episode documents the curve, names what's most at risk, and argues for what defenders must do before the gap closes.

University of Illinois Urbana-Champaign. The baseline was set in April 2024 by researchers at UIUC in arXiv:2404.08144. Daniel Kang and colleagues wrapped GPT-4 in 91 lines of code and pointed it at fifteen real-world systems with known critical vulnerabilities. GPT-4 successfully exploited 87% when given the CVE description — and 7% without it. Every other model tested, including GPT-3.5 and every open-source model available at the time, scored zero percent. The gap between GPT-4 and the field was absolute. Two years later, that gap is the story.

The Researchers. Daniel Kang leads the work from UIUC's systems and security group. The paper established the quantitative baseline that every subsequent AI vulnerability paper has been measured against, and it remains the foundational citation for understanding how fast the offense side is moving.

Key Technical Concepts. The paper's core insight: the model had the capability to exploit, but needed knowledge to aim it. The 80-percentage-point gap between having the CVE description versus not reveals that vulnerability exploitation is a knowledge problem as much as a reasoning problem — which is precisely why the subsequent shift to zero-day discovery (no CVE, no description) represents such a qualitative jump. The agent architecture used LangChain for tool orchestration and required no custom scaffolding beyond a standard ReAct loop.

Anthropic. On February 5, 2026, Anthropic's red team published "0-Days" — a documented account of Claude Opus 4.6 finding over 500 high-severity zero-day vulnerabilities in open-source codebases that had already accumulated millions of CPU hours of continuous fuzzing. The bugs had gone undetected for decades. The model found them out of the box, with no custom scaffolding, by reasoning about code the way a human researcher would: reading past fixes for similar bugs, spotting patterns that tend to cause problems, constructing the exact input required to trigger failure.

The Researchers. Nicholas Carlini and Keane Lucas led the red team effort at Anthropic. Carlini is one of the field's most cited adversarial ML researchers; his prior work includes foundational attacks on neural networks and cryptographic applications of deep learning. The paper represents a practical operationalization of capabilities that the security community had theorized about but not documented at this scale.

Key Technical Concepts. The qualitative shift from one-day to zero-day exploitation matters enormously. Fuzzing — the dominant industrial technique for finding memory bugs — works by throwing random inputs at code to see what breaks. It is brute force, and it has been running for years on these codebases. LLM-based auditing instead reasons about program logic: understanding control flow, tracing data through memory allocations, identifying use-after-free and buffer overflow patterns by semantic understanding rather than probabilistic input mutation. The bugs fuzzers miss are structurally different from the bugs LLMs miss — and they're the same class of bugs that memory-safe languages like Rust eliminate architecturally.

Mozilla. In March 2026, Anthropic and Mozilla announced a formal collaboration: Claude Opus 4.6 auditing the Firefox codebase. The result — 22 vulnerabilities in two weeks, 14 classified high-severity — represents almost a fifth of all high-severity Firefox vulnerabilities remediated across the entirety of 2025. The model scanned approximately 6,000 C++ files and submitted 112 unique reports. Mozilla's own blog post documents the coordinated disclosure process. The exploitation attempt cost $4,000 in API credits and succeeded in two cases, both requiring a sandboxed environment with security features disabled.

The Researchers. The collaboration involved Anthropic's red team alongside Mozilla's security engineering group. Mozilla maintains one of the most mature continuous fuzzing programs in open source — the OSS-Fuzz integration for Firefox has been running for years — making this an unusually high baseline against which to measure AI-assisted discovery.

Key Technical Concepts. Use-after-free vulnerabilities — the class of bug found in the first twenty minutes — occur when code continues to access memory after it has been freed, allowing an attacker to overwrite arbitrary data. They are the dominant class of critical memory safety bugs in C++ and the reason browser vendors have invested heavily in sandboxing, control flow integrity, and memory tagging. The Firefox result demonstrates that LLM-assisted auditing can find this class at scale in a codebase where human-expert and automated methods have been running continuously for over a decade.

ICLR 2026. ZeroDayBench, presented at the ICLR 2026 "Agents in the Wild" workshop, establishes the first structured benchmark for evaluating LLM agents on finding and patching 22 novel critical vulnerabilities in open-source software. Models tested include frontier systems from OpenAI, Anthropic, and xAI. The benchmark's finding: frontier models are "not yet capable of autonomously solving" the hardest tasks at full scale.

The Researchers. The ZeroDayBench authors represent the growing subfield of AI-assisted security research that has emerged specifically in response to the capability jumps documented by the Carlini/Lucas and Fang papers. The benchmark itself is a signal: the field is moving fast enough that a structured evaluation framework was needed.

Key Technical Concepts. ZeroDayBench is the honest counterpoint this episode earns credibility by naming. Fully autonomous zero-day discovery at scale — finding and reliably weaponizing bugs without human involvement — remains hard. The benchmark quantifies exactly where the frontier is, which makes it more useful than either dismissal or alarm. Related evaluation frameworks include CyberSecEval from Meta and InterCode from Princeton.

Google. The Google Threat Intelligence Group's 2025 Zero-Day Review counted 90 zero-days exploited in the wild in 2025, up from 78 in 2024. Forty-eight percent targeted enterprise software — an all-time high. Commercial surveillance vendors now attribute more zero-days than traditional nation-states. The 2026 forecast explicitly flags AI acceleration as a factor. GTIG tracks zero-day exploitation across all major platform vendors; their annual review is the primary empirical baseline for real-world exploitation trends.

The Researchers. GTIG's analysis draws on Google's threat intelligence corpus, including Project Zero disclosures, Mandiant incident response data, and public CVE records. Project Zero's 0-day "In the Wild" tracking spreadsheet is the most granular public dataset of exploited zero-days.

Key Technical Concepts. The enterprise software targeting shift matters for defenders: security appliances, VPN concentrators, and network edge devices are now the preferred entry point for state actors and commercial surveillance vendors alike, precisely because they sit outside EDR coverage and process untrusted traffic by design. The same reasoning capability that found Firefox bugs applies equally to SCADA and industrial control software — older, less-audited codebases with smaller security teams and longer patch cycles.

February 2026. Co-RedTeam documents multi-agent orchestration for security discovery and exploitation — not a single model but coordinated networks of agents handling reconnaissance, vulnerability identification, exploit development, and verification as a pipeline. The paper represents the industrialization phase: the capability is being systematized, productized, and made accessible to actors without deep security expertise.

The Researchers. The Co-RedTeam authors are part of a growing literature on agentic security systems that has expanded rapidly since the Fang et al. 2024 baseline. Related work includes PentestGPT and HackingBuddyGPT, both of which demonstrated LLM-assisted penetration testing at earlier capability levels.

Key Technical Concepts. Multi-agent architectures lower the barrier further by decomposing the vulnerability pipeline into specialized sub-agents: one reasons about code to find candidate bugs, another constructs proof-of-concept exploits, a third verifies impact. This mirrors how human red teams work — divide by expertise — and it means the capability no longer requires a single model to be good at everything simultaneously. The coordination layer is agentic scaffolding of the same kind used in software engineering agents like SWE-agent.

Daily Tech Feed: From the Labs is available on Apple Podcasts, Spotify, and wherever fine podcasts are distributed. Visit us at pod.c457.org for all our shows. New episodes daily.

Why it matters. Language models can quote the manual on a bicycle and still miss a broken chain. Beyond Language Modeling: An Exploration of Multimodal Pretraining argues that this is structural, not incidental: text is a lossy compression of reality, and models trained only on it master the description of shadows without seeing the objects casting them. The paper runs controlled, from-scratch pretraining experiments using the Transfusion framework — combining next-token prediction for language with diffusion for vision — across text, image-text pairs, video, and action-conditioned video. The result is four concrete design insights for multimodal architecture, delivered without the confound of inherited language pretraining.

Meta FAIR and NYU Courant. This paper comes out of Meta FAIR and NYU Courant, with contributions spanning vision, language, and representation learning. The full paper is at arXiv 2603.03276. A project page with additional material is at beyond-llms.github.io. The paper is also indexed on Hugging Face Papers.

The Researchers. Lead author Shengbang Tong heads a broad collaboration. Senior contributors include Yann LeCun (Turing Award, Chief AI Scientist at Meta, NYU Silver Professor), Saining Xie (NYU Courant / Google DeepMind), and Luke Zettlemoyer (UW and Meta FAIR). Also notable: Rob Fergus (NYU / Meta), Mike Lewis, Marjan Ghazvininejad, and Nicolas Ballas.

Key Technical Concepts. The paper's four findings hang on specific architectural choices. First, the Transfusion framework unifies next-token prediction and diffusion-based generation in a single training objective — no post-hoc adapter bolted onto a frozen language backbone. Second, the Representation Autoencoder (RAE), benchmarked against alternatives including CLIP-style encoders and VAE-based tokenizers, emerges as the optimal unified representation: compact enough for semantic understanding, detailed enough for generation. Third, Mixture of Experts (MoE) enables emergent modality specialization — experts naturally route toward language or visual patterns — providing efficient capacity scaling without forcing a single monolithic path. Fourth, an IsoFLOP analysis reveals vision is far more data-hungry than language: equal compute budget assumptions, standard in language-only scaling, misallocate resources in multimodal settings and produce systems with eloquent speech and weak depth perception.

Daily Tech Feed: From the Labs is available on Apple Podcasts, Spotify, and wherever fine podcasts are distributed. Visit us at pod.c457.org for all our shows. New episodes daily.

Why it matters. Links to arXiv:2603.03251. Explains Saguaro / SSD — the second speculation layer that keeps the draft model productive during verifier execution. 2× faster than optimized SD, 5× autoregressive, lossless.

Stanford / Together AI. Links to the paper, GitHub (tanishqkumar/ssd), and both model pages (Llama 3.1 70B target, Llama 3.2 1B draft).

The Researchers. Three authors with confirmed Google Scholar IDs:

Key Technical Concepts. Links to: original speculative decoding (arXiv:2211.17192), speculative sampling (arXiv:2302.01318), FlashAttention (arXiv:2205.14135), FlashAttention-2 (arXiv:2307.08691), Llama 3.1 paper. Covers the three core challenges, the 90% bonus token prediction result, cache hit/miss fallback, and the CPU branch prediction analogy from the paper.

~20 verified links total. All arXiv IDs pulled from search results, no fabricated URLs.

Why it matters. On the night Alibaba shipped Qwen3.5 — a 397-billion-parameter sparse mixture-of-experts model with 17B active parameters, a 1M-token context window, and a small-model family the open-source community had been waiting for — they fired the person who built it. This is the story of what happens when a corporate open-source champion builds something so good it undermines the product his employer is trying to sell.

Alibaba / Qwen Team. Qwen3.5 was released February 16, 2026 by the Qwen Team at Alibaba Cloud. The flagship Qwen3.5-397B-A17B is available on HuggingFace under Apache 2.0. The small-model family (4B through 35B) runs on consumer hardware — 8B on a single GPU, 14B on a gaming rig, 32B on a Mac Studio. Full model collection at Qwen on HuggingFace. The official blog post is at qwen.ai. Coverage: TechCrunch, Bloomberg, VentureBeat, OfficeChai.

The Researchers. Junyang Lin (@JustinLin610) was tech lead of the Qwen team at Alibaba Group and Peking University — the person who kept the open-weight commitment alive through years of internal corporate pressure. His departure the night of the Qwen3.5 small model launch prompted immediate public statements from colleagues; at least two other Qwen team members updated their bios to "former" within 24 hours. His replacement comes from an industrial AI background, with a mandate focused on DAU metrics and alignment with China's national AI research priorities under the 14th Five-Year Plan.

Key Technical Concepts. Qwen3.5's core architectural innovation is a hybrid Gated DeltaNet attention scheme: three out of four transformer blocks use linear attention via Gated Delta Networks — which combine Mamba2's gating mechanism with a delta rule for updating hidden states — while every fourth block uses full standard attention. This hybrid design delivers near-linear context processing cost with a 1M-token context window. The model is also natively multimodal from pretraining (early fusion across text, image, and video), not a vision adapter bolt-on, supporting 201 languages at 60% lower inference cost than its predecessor. Benchmark context: independent real-world coding evaluation on r/LocalLLaMA found that while the 397B holds up on hard tasks, it collapses on master-level multi-file agentic work — a failure mode not captured by Alibaba's self-reported SWE-bench numbers. The Qwen3 technical report (precursor architecture) is on arXiv:2505.09388. The broader architectural lineage: DeltaNet parallel implementation and the flash-linear-attention library.

Notes: No dedicated arXiv for Qwen3.5 exists yet — the primary citation is the blog post. The Qwen3 technical report (2505.09388) is the closest precursor. ~20 verified links total. Adapted the format for a news/analysis episode — no single paper anchor, so the model HuggingFace page stands in that role.

Why it matters. Every major language model in production today — GPT, Claude, Gemini, Llama — generates text the same way: left to right, one token at a time. That sequential assumption has been so productive for so long that most researchers treat it as fixed. A team at UC Berkeley and the University of Illinois just published dLLM: Simple Diffusion Language Modeling, a unified open-source framework that refuses to take autoregression for granted. Diffusion language models generate entire sequences through iterative denoising — bidirectionally, in parallel — and dLLM is the infrastructure that lets the field measure, compare, and build on them systematically for the first time.

UC Berkeley and the University of Illinois. The paper comes out of UC Berkeley and the University of Illinois Urbana-Champaign. The full paper is at arXiv:2602.22661. The framework — including pretrained checkpoints — is live at github.com/ZHZisZZ/dllm. The paper's HuggingFace page, with community discussion and model cards, is at huggingface.co/papers/2602.22661.

The Researchers. Zhanhui Zhou (UC Berkeley, equal contributor) and Lingjie Chen (UIUC, equal contributor) led the engineering. Hanghang Tong is a professor at UIUC whose work spans graph learning and large-scale data mining. Dawn Song is a MacArthur Fellow and professor at UC Berkeley, known for foundational work in security, deep learning, and the intersection of the two — she co-founded Oasis Labs and has been a central figure in open ML research for two decades.

Key Technical Concepts. dLLM unifies two discrete diffusion paradigms that have been developing in isolated codebases. Masked Diffusion Language Modeling (MDLM) formalizes the masking objective pioneered by BERT (arXiv:1810.04805) within a principled probabilistic diffusion framework, extending ideas from Simplified and Generalized Masked Diffusion for Discrete Data (arXiv:2406.04329). Block Diffusion (BD3LM) hybridizes autoregressive and diffusion generation — sequences are divided into blocks, causal across blocks and jointly denoised within each block, offering a tunable dial between the two paradigms. Both trainers are built natively on HuggingFace Accelerate, PEFT, and FSDP, making them immediately accessible to anyone already in that ecosystem. The framework's unified evaluation harness solves a reproducibility crisis that has plagued the DLM literature since discrete diffusion models for text were first formalized (arXiv:2107.03006). Perhaps most consequentially, dLLM includes conversion recipes for adapting existing pretrained checkpoints — BERT-style models via continued training under MDLM, and causal Transformer (arXiv:1706.03762) models via attention mask modification — dramatically lowering the compute barrier for community experimentation on open-weight models like Llama, Qwen, and Mistral.

Daily Tech Feed: From the Labs is available on Apple Podcasts, Spotify, and wherever fine podcasts are distributed. Visit us at pod.c457.org for all our shows. New episodes daily.

A deep dive into DualPath, a system that solves the storage bandwidth bottleneck in agentic LLM inference — then a scale-by-scale walkthrough of how the same bottleneck affects everyone from Raspberry Pi clusters to DGX SuperPods. As AI agents run multi-turn sessions (100+ turns, 95%+ KV-cache reuse), the bottleneck shifts from compute to storage I/O. DualPath exploits idle decode-engine storage NICs to load KV-cache and transfer it via RDMA to prefill engines, achieving 1.87x offline throughput and 1.96x online serving improvements. We break down the architecture, then walk from RPi5 to Mac mini to DGX Spark to production, showing where the diagnosis applies universally and where the specific cure requires datacenter hardware.

This episode was generated with AI assistance.