Last winter, Groq cofounder and CEO Jonathan Ross walked into a meeting with Nvidia CEO Jensen Huang with a pitch for the companies’ tech to work together. He now describes the synergy with a logistics analogy: stop building AI data centers as if every workload wants the same hardware. Training is bulk hauling; inference is last-mile delivery. GPUs can do both, but using the 18-wheeler even when you just need a van can be a lot slower. So: Nvidia’s general-purpose GPUs are the big trucks. Groq’s specialized chips—LPUs, or language processing units, designed to run models fast—are the smaller vans. “If you were building out a logistics network for the entire United States, and I told you your two options were all 18-wheelers or just delivery vans, which one would you pick?” Ross said. “The best answer is both.” 

Ross wasn’t just pitching a worldview. He wanted Nvidia’s permission to buy around 100,000 Blackwell chips, likely worth billions. Huang grilled him on the technical details, and then the meeting ended.

 When Huang called back three days later, Ross expected a discussion about his GPU purchase order. Instead, the Nvidia CEO cut to the chase. “We should probably move really fast,” Ross recalled him saying.

Learn more about your ad choices. Visit megaphone.fm/adchoices