Token Engineering

Token Engineering

By NeuroMetric AITechnology
Download on the App Store

Token Engineering episodes

  • TMax: Closing the Frontier Gap With Open Data

    In this episode of Token Engineering, Cooper sits down with Yash, Head of AI Research at Neurometric AI, to break down TMax from AI2 (Allen Institute for AI)—a fully open dataset and training recipe that pushes small open-weight models like Qwen to near-frontier performance on terminal agent tasks. The paper argues that diversity and difficulty matter more than where the data comes from—even when that data is entirely invented by Gemini rather than pulled from real-world code.



    We talked about:

     

    • What TMax is, and why AI2 built a fully open recipe instead of a closed benchmark
    • Why Gemini-invented data beat real GitHub repos on diversity
    • Same recipe, opposite results: boosting one Qwen version, degrading the next
    • How small models “cheat” when a task is beyond them
    • Why a lighter harness outperformed a more complex one on Claude Haiku
    • Gains that spilled past Terminal-Bench into SWE-bench and AIME math
    • The compute cost of agentic RL training, and how runaway tool calls blow the budget
    • Neurometric’s “Harbor Master,” built to run evals past AI2’s narrow benchmark set



      • Resources Mentioned:

      TMax: A recipe for terminal agents: https://arxiv.org/abs/2606.23321 




      Connect with Neurometric:

      Website: https://www.neurometric.ai/ 

      Substack: https://neurometric.substack.com/ 

      X: https://x.com/neurometricai/ 

      Bluesky: https://bsky.app/profile/neurometric.bsky.social

       

      Host/s:

      Calvin Cooper

      https://x.com/cooper_nyc_ 

      https://www.linkedin.com/in/coopernyc

       

      Guest/s:

      Yash Sharma

      https://x.com/yash_j_sharma 

      https://www.linkedin.com/in/yashjsharma

      34 min
    • From Tokenmaxxing to Tokenminning: The Case for Token Engineering

      In this episode, Cooper sits down with Neurometric CEO, Rob May, to talk through a big-news month: the show's rebrand from Inference Time Tactics to Token Engineering, the launch of tokenminning.com as a response to the "tokenmaxxing" trend, Neurometric's new token engineering platform and fundraising announcement. But the bigger conversation is about why Rob thinks "token engineering" is about to become a discipline every AI company needs—the same way DevOps or UX became non-negotiable roles—and why he's betting the company on getting there first.



      We talked about:

       

      • The origin of tokenminning.com—started as an April Fools' joke and Reddit group—and why it caught fire as a real counterpoint to "tokenmaxxing."
      • Specialized, fine-tuned models beat frontier models on individual tasks. 
      • New token engineering platform unifies fine-tuning, distillation, and caching in one system. 
      • COGS vs. OPEX: how Neurometric spots which customers feel AI costs most. 
      • Integrating with Harbor to run 15,000 agentic evaluation tasks across models like Gemma, Qwen, Mistral, and Granite.
      • Token engineering is becoming a C-suite responsibility, not just an engineering one.



        • Resources Mentioned:

        Tokenminning Manifest: https://www.tokenminning.com/ 




        Connect with Neurometric:

        Website: https://www.neurometric.ai/ 

        Substack: https://neurometric.substack.com/ 

        X: https://x.com/neurometricai/ 

        Bluesky: https://bsky.app/profile/neurometric.bsky.social

         

        Hosts:

        Calvin Cooper

        https://x.com/cooper_nyc_ 

        https://www.linkedin.com/in/coopernyc

         

        Rob May

        https://x.com/robmay 

        https://www.linkedin.com/in/robmay

        29 min
      • Automating the SLM Development Loop: Inside the Pioneer Agent Paper with Yash Sharma of Neurometric AI

        In this episode of Inference Time Tactics, Cooper sits down with Yash Sharma, Head of AI Research at Neurometric AI, to break down the Pioneer Agent paper from Fastino Labs—a system that uses Claude Sonnet as an ML engineer in a box to build and improve small language models end-to-end. From cold start data curation to production failure diagnosis, the paper argues the real bottleneck in SLM creation isn't training—it's everything around it.

         

        We talked about:

         

        • What Pioneer Agent actually is and why Fastino built a 25-page system paper instead of just a training paper. 
        • How Neurometric is already tackling the same problems—and where the paper maps onto our approach. 
        • Why naive retraining on production data quietly degrades your model—and how an agentic loop fixes it. 
        • What the benchmarks reveal about where SLMs work out of the box versus where they need intervention. 
        • Where Pioneer Agent hits its limits—and how Neurometric is pushing SLMs toward harder agentic tasks.



          • Connect with Neurometric:

          Website: https://www.neurometric.ai/ 

          Substack: https://neurometric.substack.com/ 

          X: https://x.com/neurometricai/ 

          Bluesky: https://bsky.app/profile/neurometric.bsky.social

           

          Host/s:

          Calvin Cooper

          https://x.com/cooper_nyc_ 

          https://www.linkedin.com/in/coopernyc

           

          Guest/s:

          Yash Sharma

          https://x.com/yash_j_sharma 

          https://www.linkedin.com/in/yashjsharma

          33 min
        • SLMs Beat GPT-4o: How AI Is Being Used in Hiring with Fletcher Wimbush, CEO of Discovered AI

          In this episode of Inference Time Tactics, Cooper sits down with Fletcher Wimbush, CEO of Discovered AI, to get into what’s actually broken about hiring—and how 13 years of bootstrapped recruiting, 10,000 interviews, and four peer-reviewed journal articles became the foundation for a platform now outperforming its biggest competitors. With hundreds of applicants flooding every open role and general AI tools hitting the ceiling at 70-80% resume screening accuracy, Discovered is combining behavioral science with fine-tuned small language models to get hiring right the first time.

           

          We talked about:

           

          • The hidden cost of over-relying on qualifications over attitude and integrity—and why Warren Buffett says a smart person with low integrity is your biggest threat.
          • How 10,000 interviews and four peer-reviewed journal articles with Bowling Green State University gave Discovered’s hiring system its scientific backbone.
          • Fine-tuned small language models hitting 98.5% accuracy on resume screening where GPT-4o topped out at 70-80%—at a fraction of the cost.
          • What’s coming next on the Discovered roadmap: smarter resume database search, AI-powered parsing, and an AI interviewer built to ask the right follow-up questions.

            •  

            Connect with Discovered AI:

            Website: https://discovered.ai 

            LinkedIn: https://www.linkedin.com/in/fletcher-wimbush 



            Connect with Neurometric:

            Website: https://www.neurometric.ai/ 

            Substack: https://neurometric.substack.com/ 

            X: https://x.com/neurometricai/ 

            Bluesky: https://bsky.app/profile/neurometric.bsky.social

             

            Hosts:

            Calvin Cooper

            https://x.com/cooper_nyc_ 

            https://www.linkedin.com/in/coopernyc

            29 min
          • Voice Intelligence at Scale: From Call of Duty to Fraud Detection with Modulate AI

            Every day billions of voice conversations happen across games, customer service calls, and financial transactions. Almost none of them are understood by machines. In this episode of Inference Time Tactics, Calvin Cooper and Yash Sharma sit down with Carter Huffman, CTO and co founder of Modulate, to explore the AI systems that can finally understand voice conversations in real time.

             

            Modulate’s model Velma 2.0 powers voice intelligence across industries. From moderating voice chat in games like Call of Duty to detecting fraud in financial calls and analyzing customer support conversations, their system uses ensembles of specialized models to capture tone, intent, emotion, and conversational dynamics. Instead of relying on giant foundation models, Velma orchestrates over 100 specialized models to deliver higher accuracy at dramatically lower cost.

             

            We talked about:

             

            • The challenge of processing a trillion hours of annual global voice traffic.
            • Scaling real-time moderation for massive platforms like Call of Duty.
            • Capturing nuance, tone, and sarcasm beyond basic text transcripts.
            • Ensemble architecture utilizing over 100 specialized models.
            • Orchestration layers that trim compute costs by identifying optimal model subsets.
            • Achieving order-of-magnitude cost savings compared to large foundational models.
            • Applying "exploration vs. exploitation" optimization to shifting conversation data.
            • Future development of "context graphs" to map participant intent and causality.



              • Resources Mentioned:

              NeuroMetric Audio Leaderboard: https://leaderboard.neurometric.ai/?leaderboard=audio 



              Connect with Modulate:

              Website: https://www.modulate.ai/ 

              LinkedIn: https://www.linkedin.com/in/carter-huffman-a9aba05b 

              Velma: https://www.modulate.ai/velma 



              Connect with Neurometric:

              Website: https://www.neurometric.ai/ 

              Substack: https://neurometric.substack.com/ 

              X: https://x.com/neurometricai/ 

              Bluesky: https://bsky.app/profile/neurometric.bsky.social



              Hosts:

              Calvin Cooper

              https://x.com/cooper_nyc_ 

              https://www.linkedin.com/in/coopernyc

               

              Yash Sharma

              https://x.com/yash_j_sharma 

              https://www.linkedin.com/in/yashjsharma/

              33 min
            • From GPU Scarcity to GPU Waste: Solving the Utilization Crisis

              In this episode of Inference Time Tactics, Cooper and Byron sit down with Charlie and Anil from Rapt AI to tackle one of the industry's most expensive problems: GPU underutilization. With half a trillion dollars invested in GPU infrastructure running at just 20-30% utilization, Rapt AI is building AI-powered orchestration that automatically analyzes workloads and matches them to the right compute resources—no guesswork required.

               

              We talked about:

               

              • Why half a trillion dollars in GPU infrastructure runs at only 20-30% utilization—and how a 5% drop costs $200,000 per $2M investment. 
              • How Rapt AI's platform continuously analyzes workloads and auto-optimizes GPU allocation, letting customers run 4-14 models per GPU. 
              • Real results: moving workloads from H100s to A100s at 40% of the cost, and reducing GPU footprints from 184 to under 50 while improving performance. 
              • Why 2026 becomes the year of inference as agentic workloads create unprecedented infrastructure chaos. 
              • The shift from supply problems to optimization problems—and why abstraction layers matter across multi-vendor environments. 
              • Power as the next crisis: tokens-per-watt emerging as the critical metric alongside tokens-per-dollar. 
              • How intelligent orchestration frees up data scientists and ML ops teams from infrastructure tuning to focus on AI innovation.



                • Connect with Rapt AI:

                Website: https://www.rapt.ai/ 

                LinkedIn (Anil Ravindranath): https://www.linkedin.com/in/anilravindranath 

                LinkedIn (Charlie Leeming): https://www.linkedin.com/in/charlieleeming/ 



                Connect with Neurometric:

                Website: https://www.neurometric.ai/ 

                Substack: https://neurometric.substack.com/ 

                X: https://x.com/neurometricai/ 

                Bluesky: https://bsky.app/profile/neurometric.bsky.social



                Hosts:

                Calvin Cooper

                https://x.com/cooper_nyc_ 

                https://www.linkedin.com/in/coopernyc

                 

                Byron Galbraith

                https://x.com/bgalbraith 

                https://www.linkedin.com/in/byrongalbraith

                41 min
              • Lessons from the Leading Edge: What 420 AI Deployments Reveal About Enterprise Success

                In this episode of Inference Time Tactics, Rob, Cooper, and Byron sit down with Shawn Rogers, CEO of BARC US to unpack fresh data from 421 organizations actively deploying AI in production. Shawn shares what separates the 20% of AI leaders from everyone else, why cost surprises are hitting harder than expected, and how the pressure to "just do AI" is causing companies to skip critical foundations—often to their detriment.



                We talked about:

                 

                • Why multi-model strategies and small language models are becoming essential for enterprise AI.
                • The seven foundational areas that help AI leaders deploy twice as many projects as everyone else. 
                • Why 51% of deployments face unexpected cost overruns—and which expenses hit hardest. 
                • Data quality jumping to the #1 challenge, affecting 44% of production deployments. 
                • The IT satisfaction paradox: top resource at the start, lowest satisfaction scores at scale. 
                • How responsible AI priorities shifted as human-in-the-loop dropped from 36% to 21%. 



                  • Resources Mentioned:

                  Lessons from the Leading Edge: Successful Delivery of AI/GenAI

                  https://barc.com/research/successful-ai-genai-delivery/



                  Connect with BARC:

                  Website: https://barc.com/ 

                  LinkedIn (Shawn Rogers): https://www.linkedin.com/in/shawnrogers/ 



                  Connect with Neurometric:

                  Website: https://www.neurometric.ai/ 

                  Substack: https://neurometric.substack.com/ 

                  X: https://x.com/neurometricai/ 

                  Bluesky: https://bsky.app/profile/neurometric.bsky.social



                  Hosts:

                  Rob May

                  https://x.com/robmay 

                  https://www.linkedin.com/in/robmay

                   

                  Calvin Cooper

                  https://x.com/cooper_nyc_ 

                  https://www.linkedin.com/in/coopernyc

                   

                  Byron Galbraith

                  https://x.com/bgalbraith 

                  https://www.linkedin.com/in/byrongalbraith

                  45 min
                • The Thinking Algorithm Leaderboard: Why No Single Model Wins

                  In this episode of Inference Time Tactics, Cooper and Byron break down NeuroMetric's Thinking Algorithm Leaderboard and what it reveals about building production-ready AI agents. They share why prompt engineering with a single model won't cut it for enterprise use cases, explore the impact of inference-time compute strategies, and discuss what they learned from testing 10 models across real CRM tasks—from surprising token inefficiency to catastrophic failures in SQL generation.

                   

                  We talked about:

                   

                  • Why NeuroMetric built the first leaderboard combining models with inference-time compute strategies. 
                  • How Salesforce's CRMArena-Pro reflects real multi-step business tasks better than pure reasoning benchmarks. 
                  • The jagged frontier: no single model or technique dominates across all tasks. 
                  • Why GPT 20B was surprisingly token inefficient—twice as slow as GPT 120B for similar accuracy. 
                  • How GPT-5 nano's conversational style broke SQL generation tasks completely. 
                  • Trading accuracy for speed: two-model ensembles versus five, and saving 20+ seconds per task. 
                  • Throughput constraints as a hidden bottleneck when scaling to production volumes. 
                  • Future directions: LLM-guided search, task clustering, and compression to specialized small models.



                    • Resources Mentioned:

                    CRMArena-Pro from Saleforce:

                    https://www.salesforce.com/blog/crmarena-pro/

                    Thinking Algorithm Leaderboard: 

                    https://leaderboard.neurometric.ai/ 



                    Connect with Neurometric:

                    Website: https://www.neurometric.ai/ 

                    Substack: https://neurometric.substack.com/ 

                    X: https://x.com/neurometricai/ 

                    Bluesky: https://bsky.app/profile/neurometric.bsky.social

                     

                    Hosts:

                    Calvin Cooper

                    https://x.com/cooper_nyc_ 

                    https://www.linkedin.com/in/coopernyc

                     

                    Guest/s:

                    Byron Galbraith

                    https://x.com/bgalbraith 

                    https://www.linkedin.com/in/byrongalbraith 

                    29 min
                  • Benchmarking Generalization: How AI Learns Beyond Training Data

                    In this episode of Inference Time Tactics, Rob and Cooper from Neurometric sit down with Yash Sharma, an AI researcher whose work is reshaping how we understand model generalization. Yash recently completed his PhD at the Max Planck Institute for Intelligent Systems and has held research roles at Google Brain, Meta AI, Amazon, Borealis AI, and IBM Research. His studies on compositional generalization, adversarial robustness, and long-tail benchmarks reveal when and why models succeed—or fail—at reasoning beyond their training data.

                    If you’re designing inference-time systems, building agents that need reliability, or just want to understand what “generalization” actually means in practice, this conversation bridges deep theory with actionable insight—clear, technical, and strategically grounded.

                    Key Topics
                    • What it really means for AI systems to generalize beyond their training data
                    • Why large language models still fail in novel or unpredictable scenarios
                    • How inference-time compute can both amplify and reveal generalization limits
                    • What these limits mean for building reliable, agentic AI systems
                    • How to benchmark generalization in real-world settings
                    • Yash’s “Let It Wag!” benchmark for testing long-tail and under-represented concepts
                    • Why genuine scientific breakthroughs (like curing cancer) require more than scaling test-time compute

                      • Connect with Yash Sharma:

                      • Yash Sharma
                      • Let It Wag! Benchmark
                      • Paper: Pretraining Frequency Predicts Compositional Generalization of CLIP (NeurIPS 2024 Workshop)

                        • Connect with Neurometric:

                        Website: https://www.neurometric.ai/ 

                        Substack: https://neurometric.substack.com/ 

                        X: https://x.com/neurometricai/ 

                        Bluesky: https://bsky.app/profile/neurometric.bsky.social

                         

                        Rob May

                        https://x.com/robmay 

                        https://www.linkedin.com/in/robmay

                         

                        Calvin Cooper

                        https://x.com/cooper_nyc_ 

                        https://www.linkedin.com/in/coopernyc

                        37 min
                      • Solving the Cold Start Problem in AI Inference

                        In this episode of Inference Time Tactics, Rob, Cooper, and Byron sit down with Prashanth Velidandi, co-founder of InferX, to explore how serverless inference is tackling the AI “cold start problem.” They dig into why 90% of the model lifecycle happens at inference—not training—and how cold starts and idle GPUs are crippling efficiency. Prashanth explains InferX’s snapshot technology, what it takes to deliver sub-second cold starts, and why inference infrastructure—not just models—will define the next era of AI.



                        We talked about:

                         

                        • Why inference represents 90% of the model lifecycle, compared to the training focus most of the industry has.
                        • How cold starts and idle GPUs create massive inefficiencies in AI infrastructure.
                        • InferX’s snapshot technology that enables sub-second model loading and higher GPU utilization.
                        • The challenges of explaining and selling deeply technical infrastructure to the market.
                        • Why enterprises care about inference efficiency, cost, and reliability more than model size.
                        • How serverless inference abstracts away infrastructure complexity for developers.
                        • The coming explosion of multi-agent systems and billions of specialized models.
                        • Why sustainable innovation in AI will come from inference infrastructure.



                          • Connect with InferX

                          Prashanth Velidandi

                          https://inferx.net 

                          https://x.com/pmv_inferx 

                          https://www.linkedin.com/in/prashanth-velidandi-98629b115



                          Connect with Neurometric:

                          Website: https://www.neurometric.ai/ 

                          Substack: https://neurometric.substack.com/ 

                          X: https://x.com/neurometricai/ 

                          Bluesky: https://bsky.app/profile/neurometric.bsky.social

                           

                          Rob May

                          https://x.com/robmay 

                          https://www.linkedin.com/in/robmay

                           

                          Calvin Cooper

                          https://x.com/cooper_nyc_ 

                          https://www.linkedin.com/in/coopernyc

                           

                          Byron Galbraith

                          https://x.com/bgalbraith 

                          https://www.linkedin.com/in/byrongalbraith

                          35 min

                        About Token Engineering

                        From the publisher's feed

                        A show about AI economics and the cost of thinking. From Neurometric, on the tokenminning movement, the discipline emerging to make every unit of AI work cheaper, faster, and more reliable.