In this MonkCast conversation, RedMonk's James Governor talks with Paul Brookes, a senior AI engineer at TurinTech, about making AI inference faster and cheaper. TurinTech predates the generative AI boom, having spent years optimizing complex code with genetic algorithms, and now points those tools at the models themselves. Brookes walks through techniques like kernel fusion and model compilation that squeeze more tokens per second out of specific hardware, drawing on the company's work with Intel on OpenVINO and vLLM. The two get into running capable open models such as Qwen on local machines, the spiraling cost of AI , and why judging engineers by tokens burned misses the point. Brookes also describes Artemis and its discovery harness, which lets agents learn from past results, and traces his own route from quantum physics into low-level performance engineering.

This RedMonk conversation is sponsored by TurinTech.

Show notes: https://redmonk.com/videos/paul-brookes/

Chapters
00:00 Introduction to AI and TurinTech

01:42 Optimization Challenges in AI

04:30 Working with Semiconductor Companies

08:46 The Cost of AI and Local Model Deployment

11:29 Local Model Performance and Infrastructure

14:44 Partnerships and Breakthroughs with Intel

16:54 Key Techniques for Optimizing Inference

19:39 The Role of AI Engineers

21:50 Bridging the Gap with Customers

23:21 Harnessing AI for Continuous Improvement