Programming Throwdown

Intro topic:  Buying a Car

News/Links:

• Cognitive Load is what Matters
  • https://github.com/zakirullin/cognitive-load
• Diffusion models are Real-Time Game Engines
  • https://gamengen.github.io/
• Your Company Needs Junior Devs
  • https://softwaredoug.com/blog/2024/09/07/your-team-needs-juniors
• Seamless Streaming / Fish Speech / LLaMA Omni
  • Seamless: https://huggingface.co/facebook/seamless-streaming
  • Fish: https://github.com/fishaudio/fish-speech 
  • LLaMA Omni: https://github.com/ictnlp/LLaMA-Omni 

Book of the Show

• Patrick: 
  • Thought Emporium Youtube
    • https://youtu.be/8X1_HEJk2Hw?si=T8EaHul-QMahyUvQ
• Jason: 
  • Novel Minds
    • https://www.novelminds.ai/
      
      

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Escape Simulator
    • https://pinestudio.com/games/escape-simulator/
• Jason: 
  • Cursor IDE
    • https://www.cursor.com/
      
      

Topic: Vector Databases (~54 min)

• How computers represent data traditionally
  • ASCII values
  • RGB values
• How traditional compression works
  • Huffman encoding (tree structure)
  • Lossy example: Fourier Transform & store coefficients
• How embeddings are computed
  • Pairwise (contrastive) methods
  • Forward models (self-supervised)
• Similarity metrics
• Approximate Nearest Neighbors (ANN)
• Sub-Linear ANN
  • Clustering
  • Space Partitioning (e.g. K-D Trees)
• What a vector database does
  • Perform nearest-neighbors with many different similarity metrics
  • Store the vectors and the data structures to support sub-linear ANN
  • Handle updates, deletes, rebalancing/reclustering, backups/restores
• Examples
  • pgvector: a vector-database plugin for postgres
  • Weaviate, Pinecone 
  • Milvus

• James Morse: Software Engineer at Cisco
  • System Administrator to DevOps
  • Difference between DevOps and MLOps
  • Getting Started with DevOps
• Luke Marsden: CEO of Helix ML
  • How to start a business at 15 years old
  • BTRFS vs ZFS
  • MLOps: the intersection of software, DevOps and AI
  • Fine-tuning AI on the Cloud
  • Some advice for folks interested in ML Ops
• Yuval Fernbach: CTO MLOps & JFrog
  • Starting Qwak
  • Going from a jupyter notebook to production
  • ML Supply Chain
  • Getting started in Machine Learning
• Stephen Chin: VP of DevRel at Neo4J
  • Developer Relations: The Job
  • What is a Large Language Model?
  • Knowledge graphs and the Linkage Model
  • How to Use Graph databases in Enterprise
  • How to get into ML Ops

175: Resume Writing

Intro topic:  DSLR Photography vs Camera Phone

News/Links:

• Free Internet while flying by abusing edits to your profile name
  • https://robertheaton.com/pyskywifi/
• Making Animated Characters with AI Art
  • https://www.youtube.com/watch?v=zSN76gb_Z28
• On 10x Engineers
  • https://stackoverflow.blog/2024/06/19/the-real-10x-developer-makes-their-whole-team-better/
• The Beauty and Challenges of AI-Generated Artistic Gymnastics
  • https://www.youtube.com/watch?v=YwJIYj3hPAU
    
    

Book of the Show

• Patrick: 
  • The Three Body Problem by Cixin Liu
    • https://amzn.to/3xNEoRB
• Jason: 
  • The Checklist Manifesto
    • https://amzn.to/3W2JjpM

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Super Mario Bros. Wonder (Nintendo Switch)
  • https://amzn.to/3S9VJLf
• Jason: 
  • Amazon Q
  • https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.amazon-q-vscode
    
    

Topic: Resume Writing (Courtesy of Matthew C.)

• Why have a resume?
  • Many jobs require it to get into the consideration
  • Today many are screened for keywords automatically
  • Log for future you
• What is a resume?
  • One-page description
  • Key accomplishments & experiences
  • Comparison to CV
  • References
• How to write a good resume?
  • Do’s
    • Include your github if it has good contributions
    • Be specific (dates, locations, skills)
    • Isolate your specific contributions
    • Be accurate/honest
    • Be concise
    • Be ready to discuss any point you have on the resume
    • List hobbies/activities/extracurriculars
  • Don’ts
    • Have mistakes (especially dates)
    • Use images (most companies use text extraction)
    • Use it as a design portfolio
    • Put social qualities (e.gs. hard-working, motivated, friendly)
    • Use fancy templates/tools
  • Resources
    • Manager Tools: How to scan resumes https://www.manager-tools.com/2016/05/how-scan-resume-part-1 
    • Google docs
    • Latex/Lyx for CVs
• How to think about your career and how it impacts your future resume writing (career planning)
  • Technologies and architectures more than specifics of project details
  • How various choices may age over time

Intro topic:  Social Media Auto Responder LLM

News/Links:

• Amazon releases Amazon Q
  • https://press.aboutamazon.com/2024/4/aws-announces-general-availability-of-amazon-q-the-most-capable-generative-ai-powered-assistant-for-accelerating-software-development-and-leveraging-companies-internal-data
• Cheap RiscV “Super Cluster” from Bitluni
  • DIY 256-Core RISC-V super computer
  • https://www.youtube.com/watch?v=-4d3PgEXhdY
  • CH32V203
• Phi 3 Vision Released
  • https://azure.microsoft.com/en-us/blog/new-models-added-to-the-phi-3-family-available-on-microsoft-azure/
  • Ollama
• ChatGPT 4o
  • https://openai.com/index/hello-gpt-4o/
    
    

Book of the Show

• Patrick: MyFirstMillion Podcast
  • https://www.mfmpod.com/
• Jason: A Path Towards Autonomous Machine Intelligence
  • https://openreview.net/pdf?id=BZ5a1r-kVsf

Patreon https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: Dave the Diver
  • https://store.steampowered.com/app/1868140/DAVE_THE_DIVER/
• Jason: Turing Complete
  • https://store.steampowered.com/app/1444480/Turing_Complete/ 

Topic: DevOps

• What is DevOps
  • DevOps vs SRE
• Why DevOps is important
  • Engineering time is expensive
  • Outages can hurt company metrics & reputation
• Build & Testing Infrastructure
  • Bazel & Build/Test Idempotency
  • Build/Test Farms
  • BuildBarn
  • Github Actions
  • Jenkins
• Infrastructure as code
  • Terraform
  • Blue Green Deployment
• Continuous Everything!
  • Continuous Integration
  • Continuous Deployment
• How to Measure DevOps
  • Build Times
  • Release cadence
  • Bug tracking / round trip times
  • Engineer Surveys
    • Time spent doing what you enjoy
• DevOps Horror Stories

173: Mocking and Unit Tests

Intro topic:  Headphones

News/Links:

• Texas A&M University Physics Festival
  • https://physicsfestival.tamu.edu/
• Rust vs Cpp at Google
  • Lars Bergstrom (Google Director of Engineering): Rust teams at Google are as productive as the ones using Go and 2x those using Cpp
  • https://youtu.be/6mZRWFQRvmw?t=27012
• Is Cosine Similarity Really About Similarity
  • https://arxiv.org/abs/2403.05440
• Xz utils supply chain attack
  • Andres Freund at Microsoft
  • https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/
    
    

Book of the Show

• Patrick:
  • 80/20 Running by Matt Fitzgerald
  • https://amzn.to/3xyEKLo
• Jason: 
  • A Movie Making Nerd
  • https://amzn.to/49ycDJj

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Shapez Android: https://play.google.com/store/apps/details?id=com.playdigious.shapez&hl=en_US&gl=US
  • Shapez iOS: https://apps.apple.com/us/app/shapez-factory-game/id6450830779
• Jason: 
  • Dwarf Fortress
  • https://store.steampowered.com/app/975370/Dwarf_Fortress/
    
    

Topic: Mocking and Unit Tests

• What are Unit Tests
  • Balance between utility, maintenance, and coverage
  • Unit Test: testing small functions
  • Regression Test: Testing larger functions
  • System Test: End-to-end testing of programs
• What are mocks & fakes
• When to use mock vs. fake
• Mocking libraries in various languages
  • Python: https://docs.python.org/3/library/unittest.mock.html
  • Java: https://github.com/mockito/mockito
  • C++:  https://github.com/google/googletest
    
    

172: Transformers and Large Language Models

Intro topic: Is WFH actually WFC?

News/Links:

• Falsehoods Junior Developers Believe about Becoming Senior
  • https://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/
• Pure Pursuit
  • Tutorial with python code: https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit 
  • Video example: https://www.youtube.com/watch?v=qYR7mmcwT2w 
• PID without a PHD
  • https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf
• Google releases Gemma
  • https://blog.google/technology/developers/gemma-open-models/
    
    

Book of the Show

• Patrick: The Eye of the World by Robert Jordan (Wheel of Time)
  • https://amzn.to/3uEhg6v
• Jason: How to Make a Video Game All By Yourself
  • https://amzn.to/3UZtP7b

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: Stadia Controller Wifi to Bluetooth Unlock
  • https://stadia.google.com/controller/index_en_US.html
• Jason: FUSE and SSHFS
  • https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh
    
    

Topic: Transformers and Large Language Models

• How neural networks store information
  • Latent variables
• Transformers
  • Encoders & Decoders
• Attention Layers
  • History
    • RNN
      • Vanishing Gradient Problem
    • LSTM
      • Short term (gradient explodes), Long term (gradient vanishes)
  • Differentiable algebra
  • Key-Query-Value
  • Self Attention
• Self-Supervised Learning & Forward Models
• Human Feedback
  • Reinforcement Learning from Human Feedback
  • Direct Policy Optimization (Pairwise Ranking)

Intro topic: Monitor setups

News/Links:

• BlueScuti, Willis, beats Tetris
  • https://www.youtube.com/watch?v=GuJ5UuknsHU
• PalWorld accused of being an AI Product
  • https://www.forbes.com/sites/paultassi/2024/01/22/palworld-accused-of-using-genai-with-no-evidence-so-far/?sh=26a9651b4239
• 4 Billion if-statements to determine if a number is even or odd
  • https://andreasjhkarlsson.github.io/jekyll/update/2023/12/27/4-billion-if-statements.html
• Seamless M4T
  • https://ai.meta.com/blog/seamless-m4t/
    
    

Book of the Show

• Patrick:
  • Foundation by Isaac Asimov
    • https://amzn.to/3SrmgnP
• Jason: 
  • Propaganda by Edward Bernays
    • https://amzn.to/47JUCXJ

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • The Room Game
    • https://www.fireproofgames.com/games/the-room
• Jason:
  • Incredibuild
    • https://www.incredibuild.com/
      
      

Topic: Compilers and Interpreters (Request by Jessica W.)

• Machine Code
  • Architecture Specific
• Assembly
  • Single vs Two Pass Compiler
• High level Languages
  • Intermediate Representation
  • JVM ByteCode vs Machine Code for portability
• Scripting/Interpreters
• JIT
• Profile Guided Optimization
• Resources
  • https://www.craftinginterpreters.com/
  • https://nandgame.com/
  • Turing Complete

Predictions:
Jason 

• VR for Work
• Lowering AI training cost/ improved efficiency
• RISC-V takeoff

Patrick

• Ai claim of AGI
• Ai peer reviewer
• Ai Video Generator
• More space vehicles reaching orbit

Early career, finding role at FAANG, liaising vs shipping code. Startup?

3 part. 1. How and when current hype for AI will end? 2. Shape of the show 3. Upcoming in tech

What are essential programmer knowledge items?

CS Student, how to organize life and goals? What purpose life should serve?

What kind of programmer were you in college?

Happy Holidays!

Intro topic: Testing your car battery

News/Links:

• Tech Layoffs still going on
  • https://www.sfchronicle.com/tech/article/google-layoffs-california-companies-18465600.php 
• Real-time dreamy Cloudscapes with Volumetric Raymarching
  • https://blog.maximeheckel.com/posts/real-time-cloudscapes-with-volumetric-raymarching/
• Robot Rascals
  • https://en.wikipedia.org/wiki/Robot_Rascals 
• Meta Quest 3 
  • https://www.theverge.com/23906313/meta-quest-3-review-vr-mixed-reality-headset
    
    

Book of the Show

• Patrick:
  • HyperLogLog Paper
    • https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf 
• Jason: 
  • Eureka! NVIDIA Research Breakthrough Puts New Spin on Robot Learning 
    • https://blogs.nvidia.com/blog/2023/10/20/eureka-robotics-research/ 

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Techtonica: https://store.steampowered.com/app/1457320/Techtonica/ 
• Jason:
  • ESP32 development board: https://amzn.to/3Qpmb20 
  • WEMOS

Topic: HyperLogLog

• Motivation
  • Cardinality Counting
• LinearCounting
  • Hash + expectation of collision based on how full
  • Bloom Filter
• LogLog
  • Use first N bits as bucket
  • Use max sequential 0s in each bucket
  • Average
• HyperLogLog
  • Handle empty buckets
  • Use correction factor like linear counting for low counts (number of empty buckets) and high counts
• Distributing
  • Transfer bucket counts

Intro topic: What are expectations on developers that have nothing to do with programming

News/Links:

• Pushing for a lower dev estimate is like negotiating weather with a Meteorologist
  • https://smartguess.is/blog/your-estimate-is-less-than-that/
• Announcing python in excel
  • https://techcommunity.microsoft.com/t5/excel-blog/announcing-python-in-excel-combining-the-power-of-python-and-the/ba-p/3893439
• 7 Habits of Highly Effective Software Engineers
  • https://makingsmallercircles.com/articles/7-habits-of-highly-effective-software-engineers/
• Raspberry pi 5 begins shipping
  • https://www.phoronix.com/news/Raspberry-Pi-5-Shipping
    
    

Book of the Show

• Patrick:
  • Harry Potter and the Sorcerer's Stone Illustrated Edition
    • https://amzn.to/3St3L35
• Jason: 
  • The Pete and Sebastian Show
    • https://podcasts.apple.com/us/podcast/the-pete-and-sebastian-show/id570256898

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Obsidian
    • obsidian.md
• Jason:
  • Ink by Inkle
    • https://github.com/inkle/ink
      
      

Topic: Godot

• What is a game engine?
  • Graphics, animation, particle effects
  • Tilemaps
  • Entity component systems
  • Physics
  • Sound
  • I/O
  • Input handling (touchscreen, joystick)
  • GDScript
• Why use a game engine?
  • Portability
  • Allow for testing individual components of the game
  • Libraries for ads, in-app-purchases
• Godot
  • Open source
  • Focused on 2-D but now many 3-D features
  • Great development environment
• AI Hero
  • Started in phaser (development rut)
  • Moved to Godot, rapid prototyping