
Sign up to save your podcasts
Or


Ryan and Luca welcome back Sebastian Boblest and Marco Mader Mendes from Bosch to discuss the challenges of deploying AI in safety-critical embedded systems. The conversation explores two key aspects: ensuring the generated code itself is safe (following standards like ISO 26262), and guaranteeing the trained model behaves correctly in production (covered by ISO PAS 8800).
The guests share practical strategies including fallback algorithms, model monitoring, redundancy approaches, and the importance of constraining what AI systems can do. They discuss real-world challenges teams face—from organizational resistance to unrealistic expectations about quick AI wins. Sebastian and Marco emphasize that while getting to 80% accuracy is often fast, reaching production-ready robustness requires significant effort. The episode offers a grounded look at what it actually takes to deploy ML safely in automotive, aerospace, and other domains where failures can threaten human life.
Key Topics:
Notable Quotes:
"You have to really think what you do. There's no off-the-shelf strategy. You have to really think about how you ensure that everything is working correctly." — Sebastian Boblest
"We as AI engineers are not 'hey everything has to be replaced by AI.' Maybe it's better to have a hybrid model where you still trust physical equations that have been working for 100 years." — Marco Mader Mendes
"You can solve problems with AI to the 80% level very quickly. People are super thrilled with these quick results. But the last 10 to 20% to get to product level—that gets quite tough." — Sebastian Boblest
Resources Mentioned:
We talk with Mark Deutel and Dr. Axel Plinge from Fraunhofer IIS about the art and science of running AI models on resource-constrained devices. Mark walks us through his PhD work on getting neural networks to run on microcontrollers with just 256KB of memory - a feat that requires clever optimization tricks, from neural architecture search to instruction-level tuning. Axel shares insights on why edge AI matters for robotics, where battery life and response time make local processing essential.
The conversation ranges from their multilingual airport robot (complete with culturally appropriate gestures) to the practical challenges of teaching robots to grasp objects. We explore the shift from hand-engineered features to data-driven approaches, discuss why German privacy laws favor edge processing, and learn why the simplest microcontrollers sometimes give the most predictable performance. Mark and Axel make a compelling case that not every problem needs the biggest AI model - sometimes a specialized edge model is exactly the right tool.
Key Topics:
Notable Quotes:
"You're trying to run something that is way more compute-heavy on something that shouldn't actually be able to run it. It's kind of like the magic that guys in the 90s had when they wanted to run 3D games on old computers." — Mark Deutel
"You cannot optimize without leaving stuff out. Mark says it's a magic trick, but it's not that much magic - yes, he got YOLO running on a system, but he also removed a lot of categories. Specialization is always part of the deal." — Dr. Axel Plinge
"When you start to think about it, it's actually insane what considerations you make when you grasp something. You consider: is it fragile? Where's the best point to pick it? Should I pick it at the top or bottom? Are my hands sweaty? There's all kind of stuff you do without really thinking about it." — Mark Deutel
Resources Mentioned:
We explore the resurgence of self-hosted LLMs and why they're worth considering despite the convenience of frontier models. From data privacy concerns to cost management and consistent performance, there are solid reasons to run your own models. We break down the hardware requirements (spoiler: it's all about memory bandwidth), compare different runtime environments from beginner-friendly LM Studio to production-ready VLLM, and discuss practical model choices. Luca shares his experience running dual RTX 3090s in his basement, while Ryan questions whether his 2004 Athlon is still viable (it might be). We also touch on the reality that local models lag frontier models by about six months—which means they're roughly where Claude Opus 4A was in February, and that's perfectly usable for many tasks.
Key Topics:
Notable Quotes:
"You need memory bandwidth. All of the memory. Every single token filters through all of those neural network layers—potentially 8 billion parameters per token." — Luca
"The marketing behind these tools is very much geared toward 'don't worry what's happening behind the curtain.' But there are more efficient ways of solving the same problem without incurring the same cost." — Ryan
"I really feel like the most rational choice is a second-hand 3090 still. They have 24 gigabytes of VRAM, they're nice and fast, and they have about half the bandwidth of a 5090—which is still pretty good and very much usable." — Luca
Resources Mentioned:
Ryan and Luca pull back the curtain on large language models, explaining what's really happening when you chat with an AI. They break down tokenization, context windows, and the stateless nature of LLMs—revealing why these tools aren't actually thinking or remembering, just generating the most likely next token based on massive matrices of weights. The conversation covers practical implications like why long sessions deteriorate, how caching affects costs, and why that apologetic "I won't do it again" from your LLM is meaningless. They also tackle the misleading anthropomorphization in AI marketing and share frustrations with features like Claude's opaque memory system. If you've ever wondered why your LLM seems to forget instructions or why it confidently states nonsense, this episode explains the mathematical reality behind the conversational illusion.
Key Topics:
Notable Quotes:
"The LLM doesn't even know what truth is, so it can't lie to you. It's not lying to the truth either. It's bullshitting—it doesn't care which way is true or false." — Ryan
"All LLMs know how to do is hallucinate. They just generate chains of tokens. If you're lucky, those chains have some connection to the real world and are actually helpful." — Luca
"I wish it wasn't programmed to just lie to me. If the LLM says 'I won't do it again,' yes it will, because it has no memory of this incident. It will behave the same way tomorrow." — Luca
Resources Mentioned:
We talk with Sebastian Boblest and Marco Mader Mendes from Bosch about the surprisingly challenging art of deploying AI models on resource-constrained embedded devices. They share insights from their work on ETAS Embedded AI Coder, a tool that generates optimized C code from neural network models for microcontrollers—sometimes with as little as 100 bytes of RAM available.
The conversation covers practical strategies for model compression (often by factors of 100x or more), the counterintuitive benefits of float models over quantized ones on tiny devices, and why feature engineering still matters. Sebastian and Marco explain how they navigate the trade-offs between RAM, compute time, and accuracy, and why each project presents unique constraints—from confidential hardware specs to compiler quirks. They also discuss real-world applications, including Bosch's AI-powered wall scanner that uses radar and neural networks to detect cables in walls.
Key Topics:
Notable Quotes:
"We go down to applications where we use 100 bytes of RAM and you can still do something useful with this on a Cortex-M0. It's very surprising how small you can get with neural networks." — Sebastian Boblest
"Sometimes we have to squeeze it down not by one or two X, sometimes it's up to 100 X and more. This is really a regular task for us." — Marco Mader Mendes
"One thing that's super counterintuitive for many people on these smaller devices is to go from a quantized model to a float model. It can actually help you. It's exactly the opposite of what people do on these larger systems." — Sebastian Boblest
Resources Mentioned:
Ryan and Luca tackle the hot topic of AI-driven security research, sparked by the release (and brief containment) of Anthropic's Mythos tool. Ryan, drawing on his 20+ years in cybersecurity, delivers a surprisingly reassuring message: if you've been doing proper engineering, AI-found vulnerabilities aren't the existential threat the hype suggests.
We explore how Mythos and similar tools are flooding the CVE database with thousands of new vulnerabilities, but discuss why this doesn't automatically mean more successful attacks. Ryan explains the crucial difference between finding a bug and actually exploiting it, and why embedded systems developers shouldn't panic—but should definitely have their update processes sorted. The conversation covers everything from botnet refrigerators to Ukrainian security cameras, threat modeling with AI assistance, and why AI makes such a relentlessly effective hacker (spoiler: it doesn't get bored). Bottom line: the weapons haven't changed, you just need to install the bulletproof glass you should have had all along.
Key Topics:
Notable Quotes:
"Just because there's more CVEs doesn't really change the defensive posturing that you need to do. If you have a solid engineering process to provide updates to your system and test and verify those updates actually work, it doesn't matter how many bugs you find. You can find two. You can find 6,000." — Ryan Torvik
"AI doesn't have social norms. AI will just go and do things. And yes, there are guardrails that they're trying to put on, but like, I don't know if you've seen AI just ignore parts of your prompt before." — Ryan Torvik
"There's not a death star. This is not an existential crisis. This is not something new. It changes the game but not in a way that we can't handle. The weapons haven't changed, you just need to install the bulletproof glass." — Ryan Torvik
Resources Mentioned:
We tackle the elephant in the room: will AI take our jobs? Spoiler alert - probably not, but your job will definitely change shape. Ryan and Luca dig into what software development actually is (hint: it's not just mashing keyboards), why embedded systems might be particularly safe from AI disruption, and what we can learn from the Luddites and steam engines.
Drawing on Luca's experience with DevOps transformations and Ryan's work running a company, we explore why engineers keep solving the wrong problems, what Black & Decker actually sells (it's not drill bits), and why vibe-coding your email server is a terrible idea. The real question isn't whether AI will replace you - it's whether you understand what your actual job is in the first place. Plus: Pokemon evolution as career advice, and why AI is the new Agile fairy dust.
Key Topics:
Notable Quotes:
"Your job is not writing code. Your job is making product. If you think about the Luddites, their job was not operating a loom. Their job was making clothes." — Ryan Torvik
"Writing code is really the smallest part of software development. Most of it is sitting in front of the screen, looking up and to the left, and figuring out what code to write." — Luca Ingianni
"The CEO of Black & Decker was once quoted as saying: We are not in the business of selling drill bits. We are in the business of selling holes in the wall. If we had laser cannons that made holes in walls, people would buy the laser cannons instead." — Luca Ingianni
Resources Mentioned:
Luca unveils WhittleSpec, his new open-source framework for AI-assisted development that challenges the waterfall assumptions baked into most coding tools. Born from frustration with frameworks like SpecKit that treat specifications as static documents, WhittleSpec emphasizes continuous learning through retros, refinement, and test-driven development. We explore why most AI coding frameworks make typing faster but don't help with the hard part—the thinking—and discuss how proper feedback loops and vertical slicing can lead to more trustworthy software.
The conversation ranges from the philosophy of whittling away what doesn't fit (versus plowing ahead blindly) to practical implementation details: specification, planning, task breakdown, TDD cycles, and retrospectives. Luca argues that professional software engineering requires systematic processes that support learning at every step, not just tools that generate code quickly. We also touch on the missing pieces in current frameworks: support for safety-critical development, long-term roadmaps, and embedded systems considerations.
Key Topics:
Notable Quotes:
"No engineer ever said, 'I wish I could type curly brackets faster.' That was never quite the bottleneck. The hard part is sitting in front of your screen and going, 'hmm.'" — Luca Ingianni
"SpecKit is just plain old horrendous waterfall. There are no provisions at all in it for learning. The idea is you specify something well enough and then you just walk away, sip a coffee, the machine does its thing. That approach has never ever worked." — Luca Ingianni
"Your initial specification is not going to be right. As you implement the actual solution, you're going to learn things that's going to change what you need to accomplish with the spec. You might need to change completely what your expectations are." — Ryan Torvik
Resources Mentioned:
Ryan and Luca sit down with Shawn Hymel, an educator and course creator focused on edge AI and embedded systems. We explore what's changed in the past few years—from basic keyword spotting to full object detection on microcontrollers, thanks to integrated NPUs. Shawn walks us through the messy reality of deploying ML models to embedded hardware: vendor-specific toolchains, dependency hell, and the ongoing challenge of making edge AI accessible. We discuss what students and experienced engineers need to learn (or unlearn) to work effectively in this space, and look ahead to exciting developments like reinforcement learning on tiny devices and neuromorphic computing. It's a candid, technical conversation about where edge AI stands today and where it's headed.
Key Topics:
Notable Quotes:
"Five, six years ago, we didn't have full object detection on microcontrollers. Now with NPUs, we can do things like full YOLO on a 320x320 image—milliwatts of power, full object detection. That was not a thing five years ago." — Shawn Hymel
"Expect to spend a day or two getting inference to actually run. The docs are still new, the graph compilers are fairly new. You're going to end up in dependency hell—both on the Python side and when you bring it over to the embedded side." — Shawn Hymel
"If a PID loop solves your need, there is absolutely no reason to put AI on there. That is a solved problem. Don't do it—just use a PID loop." — Shawn Hymel
Resources Mentioned:
Ryan and Luca tackle a challenge many AI users are facing: what happens when your AI provider starts acting up? Drawing from recent experiences with Anthropic's capacity issues, secret billing practices, and model degradation, we explore practical strategies for avoiding vendor lock-in.
We discuss the three layers of complexity: the model itself, the harness (like Claude Code or GitHub Copilot), and your authored content (skills, MCP servers, prompts). Each layer presents different challenges when switching providers. Ryan shares his approach of stepping back to simpler, more granular prompting to stay provider-agnostic, while Luca experiments with maintaining escape hatches to other platforms. We also look at the realities of running local models and the tradeoffs between convenience and control. The bottom line? Pick one system, get proficient, but prepare your exit strategy - because in this volatile landscape, you'll likely need it sooner than you think.
Key Topics:
Notable Quotes:
"The more you actually make use of AI in your work, the more you use it as a force multiplier, the more painful it becomes if that force multiplier goes away." — Luca
"It's about total clock time. If you get a one shot and then have to redo it again, how much of that clock time is being used effectively?" — Ryan
"Pick one, stick with it, be proficient in it. But prepare yourself to have to escape eventually, because the situation is so volatile." — Luca
Resources Mentioned:
From the publisher's feed