MIT’s Manya Ghobadi on AI Economics: Why Token Pricing Misses the Point

Host Dan Mitchell interviews Manya Ghobadi, MIT CSAIL associate professor and founder/CEO of Systalyze, about making AI practical, efficient, and economically viable at scale. Ghobadi explains that AI “models are just math,” with prompt cost driven by model architecture and matrix multiplications, and offers a simple view of LLMs as next-token probability predictors. She argues efficiency should be a first-principles focus, critiques token-based pricing as a flawed proxy for value, and advocates resource-based thinking centered on GPU requirements, deterministic performance, and self-hosted open-source models for enterprises. Drawing on conversations with over 1,000 Fortune 500 AI leads, she notes organizations struggle to measure ROI and maintain efficient GPU clusters as models and libraries change. She contrasts academia’s long-term innovation with industry’s resource-driven progress, calls agentic AI overhyped, and highlights performance and energy efficiency as underestimated, urging listeners to challenge the status quo and choose tools wisely.

00:00 Mainly AI Episode 3 - Models Are Just Math: The Economics of AI at Scale

01:26 AI Is Just Math

04:59 The Economics of AI

14:14 Academia vs. Industry

21:34 Most Overhyped AI Trend

24:19 Most Underestimated AI Trend

27:25 5-Year Prediction

Mainly AI Episode 2: AI Snake Oil - are they correct?

Welcome back to Mainly AI, where we dive deep into groundbreaking perspectives on mainstream technology. In this episode, we explore a thought-provoking paper that reframes artificial intelligence (AI) as a general-purpose technology, akin to electricity or the internet, rather than a mystical or terrifying force. We discuss the slow but impactful diffusion of AI into society, its business applications, societal and cultural impacts, and the necessary regulations and frameworks needed to guide its responsible use. Understanding AI's limitations and its mirrored reflection of human values is crucial for its future development. Join us as we dispel myths and promote a realistic view of AI's role in our lives.

00:00 Introduction to Mainly AI

01:07 Reframing AI: Not Magic, Just Technology

01:50 Historical Comparisons: Electricity and the Internet

03:24 AI Adoption and Diffusion

04:49 Three Layers of AI Progress

06:54 Regulation and Cultural Acceptance

10:11 Trust and Safety in AI

18:05 AI as a Mirror: Reflections and Responsibilities

21:44 Conclusion and Call to Action

In this episode of Mainly AI, hosts Dan Mitchell and Viby Jacob dive into the topic of agentic AI with guests Neetu Pathak, CEO and co-founder of SkyMel, and Drew Dimmick, CTO and co-founder of Prompt 360. They discuss the necessity of check-in balances for autonomous agents, the implications for various industries, and real-world use cases for these agents, including customer support and enterprise IT management. Ethical considerations and potential ramifications such as the need for governance frameworks and liability issues are explored. The conversation also touches on the future of AI agents in enhancing day-to-day tasks, reducing menial work, and creating new types of jobs. The episode concludes with a discussion on the Model Context Protocol (MCP) and A2A (Agent to Agent) communication protocols that facilitate the cooperation between agents, making systems efficient and flexible.