High Signal: Data Science | Career | AI

Noah Smith, economist and author of Noahpinion, joins High Signal to look at what AI is already changing… and what it isn’t. The conversation moves beyond the usual productivity hype to ask harder questions: Is agentic coding actually increasing revenue per hour worked? Will software remain a high-margin business if AI makes it easy to clone? And what happens when generating content, code, vendors, applications, and companies becomes much cheaper than verifying them?

Noah argues that one of the biggest near-term shifts is not simply automation, but trust. AI is beginning to replace parts of the internet’s knowledge infrastructure — search, Stack Overflow, Reddit, and how-to content — while also flooding markets with new forms of slop. For AI builders and leaders, the central challenge may become less about producing more and more about knowing what is real, valuable, and worth trusting…. In a word, verification.

LINKS

Noah's writing we discuss:

Books, essays, and reports mentioned:

Steve Tadelis, Professor of Economics at UC Berkeley and former senior economist at eBay and Amazon, joins High Signal to bridge the gap between economic theory and the high-stakes reality of data science and AI. Drawing on his experience at the forefront of the world’s largest marketplaces, Steve discusses the "invisible friction" that prevents organizations from acting on data: a combination of misaligned incentives, organizational inertia, and the "Upton Sinclair problem," where leaders are effectively paid not to understand new paradigms.

The conversation moves from the "frustratingly obvious" opportunities left on the floor during eBay’s early years to the relentlessly scientific culture of Amazon. Steve explains why surface-level metrics like conversion rates often mask underlying rot in user retention and how rigorous experimentation, such as his famous $20 million search-ad experiment, can expose the difference between genuine growth and mere navigational intent. We also explore the structural shifts of the AI era, where Steve offers an important counter-narrative: rather than leveling the playing field, AI may act as an "unequalizer" that exponentially rewards those with the deepest critical thinking skills.

LINKS

Chris Fonnesbeck, veteran analyst for the Yankees and Mets and creator of the open-source Bayesian modeling library PyMC, joins to unpack why baseball has been a leading indicator for data science and analytics for over a century, and why builders and AI leaders need to pay attention now. The reason it has led is simple: huge incentives and a culture that treats decisions as quantifiable. With wins worth about eight to ten million dollars apiece and front offices built around probabilistic reasoning, baseball has had every reason to push the methods further and faster than industry.

The skillset and culture that built this lead is what AI teams now need to adopt more of: probabilistic thinking, hierarchical models, integrating expert judgment, reasoning carefully under uncertainty, and increasingly causal inference. The conversation traces the throughline from those early statistical innovations to the decisions driving multi-million dollar contracts today, with concrete patterns AI builders can take back to their own work: how to handle small samples and high stakes, why outcomes are the wrong thing to measure, what changes when you push uncertainty all the way through your model, and why robust causal inference needs to be the next frontier.

LINKS

Noah Bruegmann, President of Data CRT, joins High Signal to discuss how to move your data function from a cost center to a strategic "value center". He explains how AI amplifies your existing data culture, the importance of "no-assistance" reporting, and how rebranding documentation as "Context" can finally secure executive buy-in. Drawing on 15 years of experience spanning trading floors and Silicon Valley startups, Noah argues that for too long, data teams have been submerged under an "iceberg" of invisible data preparation. He details how the arrival of LLMs and agentic tools is fundamentally shifting this landscape, automating technical drudgery and allowing data professionals to transition into what he calls "Jack Ryan" mode: acting as high-level intelligence analysts rather than mere number crunchers.

We dig into the architectural and psychological shifts required to navigate this new era and why the most valuable skill in an AI-augmented world is no longer mastering SQL syntax, but "problem framing": the ability to reduce business ambiguity into high-leverage insights. Noah cautions that while AI offers a dopamine hit of instant answers, it demands a new discipline of rigorous verification to avoid automated hallucinations. The conversation provides a clear directive for executives: move past the "ticket-taker" model and start treating the data team as the essential "left-side brain" for organizational decision-making.

LINKS

Jordan Morrow, SVP of Data & AI Transformation at AgileOne and the "Godfather of Data Literacy," joins High Signal to discuss the shift from being "data-driven" to becoming "AI-enabled." Jordan warns that many organizations are walking into the same traps that derailed the data science era a decade ago: prioritizing expensive tooling and hype over the cultural change and literacy required to actually move the needle. The pattern is already visible: enterprise AI projects are failing at around 90%, while individuals doing shadow AI are quietly thriving. The catch is that shadow AI brings its own risks, with people feeding sensitive data into public models without governance. He argues that because AI is probabilistic rather than deterministic, the bottleneck for success has shifted from technical coding ability to a user’s ability to apply "Engineered Intelligence," a blend of machine capability and human emotional intelligence.

LINKS

Dawn Woodard (Distinguished Engineer, LinkedIn), Andrés Bucchi (LATAM Airlines), and Jeremy Hermann (CEO & Co-Founder, Delphina) join High Signal for a deep dive into the shifting architecture of data science & analytics in the era of AI. As the industry moves from static dashboards to vibe coding and conversational querying, this panel of industry veterans explores why traditional data fundamentals—strict cataloging, verifiable outputs, and a single source of truth—are suddenly the most critical bottlenecks in the AI era.

We dig into the sobering reality of the "source of truth" problem, where the speed of AI-generated code far outpaces our ability to define what "correct" actually looks like in a complex enterprise. The conversation reveals how AI is breaking legacy experimentation platforms, the transition of the data analyst into a "verifier" of AI-generated workflows, and why "headless" security architectures are essential for the next generation of autonomous agents. From the limitations of LLMs in causal reasoning to the challenges of integrating AI into "brownfield" enterprise codebases, this discussion provides a grounded framework for leaders navigating the gap between AI hype and operational reality.

LINKS

Martin Tingley, Head of Windows Experimentation at Microsoft and former Head of the Experimentation Platform Analysis Team at Netflix, talks about why humans are the bottleneck in experimentation, and how a five-level maturity framework points the way toward self-optimizing software.

Our conversation traces the path from basic hypothesis testing to a frontier where Generative AI creates, evaluates, and refines product variants in a closed loop. We explore the architectural shift required to move from testing single variants to optimizing entire parameter spaces, and how startups are already using AI to generate production-ready landing pages for Fortune 500 companies in hours rather than weeks. Tingley also shares a strategic lens on "experimentation programs," explaining how plotting the distribution of treatment effects across different product areas can serve as a powerful tool for capital allocation and high-level strategy.

LINKS

Bozena Pajak, VP of Learning at Duolingo, joins High Signal to discuss the evolution of AI at Duolingo: from personalized difficulty models to the current generative frontier where AI characters provide low-stakes and high impact conversational practice. We discuss the role of AI in overcoming one of the biggest hurdles in language acquisition, speaking anxiety. We also talk about how Bozena's team leverages agentic workflows to scale content and why the next wave of personalization involves shifting from difficulty levels to "thematic lenses" tailored to specific user interests.

LINKS

Benn Stancil, writer and co-founder of Mode, joins High Signal to ask some uncomfortable questions about the current AI moment. Is now actually a terrible time to start a company? If the tools you build on today are obsolete in six months, at what point does the head start stop mattering? Is all that context engineering you're doing a waste of time, destined to go the way of Boolean search syntax in the 90s?

Benn argues that AI is turning us all into Steve Jobs, not the visionary who delegated, but the one who berated people over pixel placement. As AI takes over the doing, our job becomes obsessing over the polish. He makes the case that technical debt may be self-healing: if future models can untangle the mess today's models made, then messy code isn't debt…it's a spec for a clean rewrite.

We also dig into why Claude Cowork can't work. AI has these uncanny ticks you can't beat out, so anything it writes "as you" will smell like AI. The solution isn't better AI writing—it's to stop pretending we write to each other at all. Benn envisions a future where communication is radically intermediated: I dump facts into a shared repository, your AI reads them, and nobody bothers with the social decoration in between.

LINKS

Nicholas Moy, former Head of Research at Windsurf & now at Google DeepMind, joins High Signal to discuss the shift from "co-driving" to a truly "agentic" era of development. We discuss Windsurf's journey from early prototypes that struggled with compounding errors to the successful launch of their agentic coding product. Nick explains that building a startup in the current climate requires a strategy of "disrupting yourself" to avoid the innovator’s dilemma; companies must be ready to pivot as soon as a new frontier model makes previously impossible features viable. He argues that traditional technical moats are increasingly fragile, and true defensibility now comes from real-world usage data, brand reputation, and a deep intuition for what users need at the frontier of these capabilities.

LINKS