What if the fastest path to efficient edge AI isn’t a bigger CPU, but a smarter stream of data? We pull back the curtain on NeuralArt—the flexible, stream‑based accelerator inside the STM32N6—and show how a decade of prototypes led us to rethink how tensors move, how layers are scheduled, and how much work a compiler can save when memory is the real bottleneck. Instead of shuttling activations back and forth, our architecture routes data through specialized units in tightly orchestrated “epochs,” keeping compute hot and bandwidth cool.

From there, we tackle the hard limits of standard‑cell designs on practical MCU nodes. Power efficiency stuck around 1–5 TOPS/W and density near 0.1–2 TOPS/mm² pushed us to explore in‑memory computing. We break down digital versus analog IMC—determinism and integration on one side, approximate but highly efficient compute on the other—and share prototype results that hit roughly 40 TOPS/W and about 10 TOPS/mm² at 1 GHz. Along the way, we dig into why half of system power can vanish into data movement and how weight‑stationary strategies change the game.

We also get candid about trade‑offs. Embedded phase change memory (PCM) brings remarkable density and multi‑level storage, but demands strict weight‑stationary mapping and drift compensation. No single technology wins every metric, so we lay out a heterogeneous 2D mesh that blends digital IMC, analog IMC, and classical stream units. Our compiler assigns each subgraph to the node that fits its accuracy, throughput, and energy needs, and our NeoSoC research effort moves this vision toward silicon with an upcoming 80‑nm tapeout.

If you care about edge inference, memory bandwidth, quantization, and real‑world efficiency beyond spec‑sheet peaks, this conversation is for you. Subscribe, share with a teammate who’s wrestling with on‑device AI, and leave a review with the biggest bottleneck you want us to tackle next.

Send us Fan Mail

Support the show

Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org