Grumpy old Coders

David and Thomas dive into the inner workings of the Transformer architecture behind modern Large Language Models. They break down foundational concepts including vectors, matrices, function approximations, artificial neural networks, and backpropagation. Supported by a live Python code demonstration, they walk through tokenization, word embeddings, sinusoidal positional encodings, and Query-Key-Value attention dynamics. The episode wraps up with a discussion on scaling context windows, KV caching, and the philosophical debate around AI consciousness versus human self-awareness.

Outline:

00:21 – Episode intro, podcast banter, & topic overview

02:37 – Mathematical foundations: Vectors, matrices, & functions

06:18 – Artificial Neural Networks: Layers, weights, & backpropagation

11:42 – ANN practical example: Image classification mechanics

14:51 – Tokenization & vocabulary constraints

18:40 – Word embeddings & GloVe vector spaces

21:00 – Positional encoding & sinusoidal matrices

27:00 – Self-attention components: Queries (Q), Keys (K), and Values (V)

31:37 – Live Python code walkthrough & script execution

37:25 – Attention matrix calculations & Softmax normalization

45:56 – Multi-head attention & multi-stage learning dynamics

49:44 – Large Language Model context window scaling

55:13 – KV caching & inference optimization

58:28 – AI consciousness debate

1:14:24 – Guardrails, sampling temperature, & LLM watermarking

1:20:51 – Closing thoughts and sign-off

David and Thomas recap their multi-country motorcycle journey through Germany, Austria, Slovenia, and Italy. They share key moments from the road, including navigating winding mountain serpentines. Along the way, they reflect on awkward luggage setups and unpredictable alpine weather. The episode wraps up with a discussion on the tech job market, the realities of AI "vibe coding" versus software engineering fundamentals, and preliminary planning for their next road trip.

Outline:

00:20 – Episode intro, camera/mic troubleshooting, & tour premise

01:00 – Bike comparison: Suzuki SV650 vs. BMW R 1250 GS

03:11 – Lake Chiemsee stop, rain, and duck watching

06:08 – Crossing into Austria via Kufstein & taking biker photos

08:20 – Triassic Park landmark & tour snacking habits

10:32 – Riding through Slovenia: Serpentines

15:00 – Wörthersee stopover

20:04 – Tom's leg injury

24:25 – High-altitude dining & Slovenian road marking confusion

29:13 – Luggage setup challenges

35:00 – Nolan helmet visor breakdown

40:14 – Trattoria al Norgé: Homemade pasta & Italian hospitality

45:37 – Narrow mountain pass & wet bitumen hazards

52:30 – Staying in Innervillgraten & Innsbruck hospital visit

1:01:54 – Returning to Germany

1:11:00 – Tech industry state, LLMs, & "vibe coding" vs. engineering

1:19:10 – Future trip planning

1:22:50 – Closing thoughts and sign-off

David and Thomas dive into the critical role of context management and agentic memory with large language models. They explore the core mechanics of AI agents, breaking down the differences between low-level tools, Model Context Protocol (MCP) servers, and high-level skills, before tackling the token cost challenges of stateless LLMs. The hosts analyze human-inspired cognitive strategies like context "compaction" and background "dreaming". The episode closes with thoughts on codebase visualization using Graphify and Obsidian, and a humorous nod to Inception-style agent recursion.

Outline:

00:21 – Heatwave talk & audio troubleshooting

07:16 – Chatbots vs. AI agents

12:20 – Defining tools, MCP, and skills

20:51 – Context boundaries & compaction

33:12 – Cognitive concepts and "dreaming"

40:14 – Exploring several tools

1:11:15 – Closing

After a year-long break, David and Thomas return to discuss CHAOSS (Community Health Analytics in Open Source Software), a Linux Foundation framework for evaluating open source project health. They explore metrics like compliance/security, governance, and community activity. The hosts emphasize that while metrics provide valuable data, community trust and authentic collaboration matter most. They also share motorcycle adventures—David's new Suzuki SV650 and their Black Forest tour. The episode closes with plans for more frequent releases and inviting guest speakers.

Outline:

00:00 – Reunion & motorcycle talk

08:20 – What is CHAOSS?

15:26 – Metric groups (compliance, governance, community)

22:27 – Bus factor & sustainability

25:00 – GrimoireLab monitoring tool

28:14 – When to adopt CHAOSS

34:26 – Future plans & closing

In this episode, David and Thomas delve into the concept of Product-Led Growth (PLG). They discuss the differences between sales-led and product-led approaches, emphasizing the importance of user experience and collaboration across teams. The conversation highlights the benefits of PLG, particularly for startups, and the necessity of understanding customer needs and feedback to ensure product success. In this conversation, Thomas and David explore the intricacies of product-led growth (PLG) and the journey users take from being strangers to becoming champions of a product.

Chapters:

00:00

Introduction to Product-Led Growth

02:55

Understanding Go-to-Market Strategies

10:05

The Role of User Experience in PLG

15:09

Benefits and Challenges of PLG

21:16

Collaboration Across Teams for Success

36:36

The Genesis of Open Source Products

37:58

Understanding Customer Acquisition Costs

38:55

The Product-Led Growth Framework

40:37

Transitioning from Strangers to Explorers

43:41

The Evaluation Stage: From Stranger to Explorer

46:51

Activation: The Moment of Value Realization

51:20

From Explorer to Beginner: The Activation Journey

53:11

The Importance of Documentation and Training

56:47

Onboarding Experiences and User Guidance

01:01:29

Driving Adoption and Cultivating Regular Users

01:06:41

The Role of Champions in Product Advocacy

01:08:00

The Role of Champions in Product Growth

01:10:11

Understanding the Flywheel Effect in SaaS

01:14:02

Brand Loyalty and User Experience

01:15:56

The Competitive Landscape of Cloud Services

01:19:02

Open Source vs. Proprietary Software

Keywords:

Product-Led Growth, Go-to-Market Strategy, User Experience, Sales, Marketing, SaaS, Customer Success, PLG Benefits, Collaboration, Startup, open source, product-led growth, customer acquisition, evaluation stage, activation, onboarding, user experience, documentation, champions, SaaS, Product Growth, SaaS, Champions, Flywheel Effect, Brand Loyalty, Cloud Services, Open Source, PLG, Software Licensing

In this episode, Thomas and David discuss the evolving landscape of e-mobility, focusing on electric vehicles (EVs) and their implications for consumers and the environment. They explore various types of electric vehicles, including hybrids and fully electric models, and delve into battery technology, efficiency, and the cost of ownership. The conversation also touches on the charging infrastructure, everyday usage experiences, and the longevity of EV batteries. Additionally, they examine the competitive landscape, particularly the rise of Chinese manufacturers in the EV market.

Chapters:

00:00

Types of Electric Vehicles

06:09

Efficiency of Electric Vehicles

12:28

Comparing Electric and Traditional Vehicles

19:30

Pricing and Competition in the Market

25:17

Range and Charging Infrastructure

29:12

Benefits and Considerations of E-Mobility

34:32

Understanding Charging Options

36:26

Availability and Pricing of Charging Stations

44:02

Different Prices for Charging

46:38

Charging at Home and Battery Lifespan

53:18

Lifespan of Electric Car Batteries

57:30

Replacing the Battery

01:01:07

Efficiency and Maintenance of Electric Cars

01:03:23

Financial Considerations and Environmental Impact

01:13:28

Chinese Car Manufacturers in the Global Market

Keywords: e-mobility, electric vehicles, battery technology, charging infrastructure, environmental impact, cost considerations, market trends, efficiency, hybrid vehicles, Chinese competition

In this episode, David and Thomas discuss their use of a large language model to build a chatbot based on documentation. They explain the concept of prompt engineering and how it can be used to guide the model's responses. They also introduce the concept of retrieval augmented generation (RAG). They also discuss the use of knowledge base articles and documentation to influence the behavior of LLMs. Additionally, they touch on the use of graph databases and embeddings to identify redundancies in documentation and improve search results. The conversation concludes with a discussion on the challenges and limitations of working with LLMs.

Merry Christmas (or just a happy holiday season)! Thomas and David discuss how technology is part of a wider definition of culture and how techies can contribute to technical education of kids.

Thomas and David talk about Technical Enablement, how David came to work on Technical Enablement projects, how it can help compensating the pains of scaling quickly, Learning Experience Design and more.

Thomas and David talk about how to produce a podcast. You will learn which equipment is needed, which software can be used and how the prep work and post-production looks like. Have fun!