Programming Throwdown

Intro topic:  Social Media Auto Responder LLM

News/Links:

• Amazon releases Amazon Q
  • https://press.aboutamazon.com/2024/4/aws-announces-general-availability-of-amazon-q-the-most-capable-generative-ai-powered-assistant-for-accelerating-software-development-and-leveraging-companies-internal-data
• Cheap RiscV “Super Cluster” from Bitluni
  • DIY 256-Core RISC-V super computer
  • https://www.youtube.com/watch?v=-4d3PgEXhdY
  • CH32V203
• Phi 3 Vision Released
  • https://azure.microsoft.com/en-us/blog/new-models-added-to-the-phi-3-family-available-on-microsoft-azure/
  • Ollama
• ChatGPT 4o
  • https://openai.com/index/hello-gpt-4o/
    
    

Book of the Show

• Patrick: MyFirstMillion Podcast
  • https://www.mfmpod.com/
• Jason: A Path Towards Autonomous Machine Intelligence
  • https://openreview.net/pdf?id=BZ5a1r-kVsf

Patreon https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: Dave the Diver
  • https://store.steampowered.com/app/1868140/DAVE_THE_DIVER/
• Jason: Turing Complete
  • https://store.steampowered.com/app/1444480/Turing_Complete/ 

Topic: DevOps

• What is DevOps
  • DevOps vs SRE
• Why DevOps is important
  • Engineering time is expensive
  • Outages can hurt company metrics & reputation
• Build & Testing Infrastructure
  • Bazel & Build/Test Idempotency
  • Build/Test Farms
  • BuildBarn
  • Github Actions
  • Jenkins
• Infrastructure as code
  • Terraform
  • Blue Green Deployment
• Continuous Everything!
  • Continuous Integration
  • Continuous Deployment
• How to Measure DevOps
  • Build Times
  • Release cadence
  • Bug tracking / round trip times
  • Engineer Surveys
    • Time spent doing what you enjoy
• DevOps Horror Stories

173: Mocking and Unit Tests

Intro topic:  Headphones

News/Links:

• Texas A&M University Physics Festival
  • https://physicsfestival.tamu.edu/
• Rust vs Cpp at Google
  • Lars Bergstrom (Google Director of Engineering): Rust teams at Google are as productive as the ones using Go and 2x those using Cpp
  • https://youtu.be/6mZRWFQRvmw?t=27012
• Is Cosine Similarity Really About Similarity
  • https://arxiv.org/abs/2403.05440
• Xz utils supply chain attack
  • Andres Freund at Microsoft
  • https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/
    
    

Book of the Show

• Patrick:
  • 80/20 Running by Matt Fitzgerald
  • https://amzn.to/3xyEKLo
• Jason: 
  • A Movie Making Nerd
  • https://amzn.to/49ycDJj

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Shapez Android: https://play.google.com/store/apps/details?id=com.playdigious.shapez&hl=en_US&gl=US
  • Shapez iOS: https://apps.apple.com/us/app/shapez-factory-game/id6450830779
• Jason: 
  • Dwarf Fortress
  • https://store.steampowered.com/app/975370/Dwarf_Fortress/
    
    

Topic: Mocking and Unit Tests

• What are Unit Tests
  • Balance between utility, maintenance, and coverage
  • Unit Test: testing small functions
  • Regression Test: Testing larger functions
  • System Test: End-to-end testing of programs
• What are mocks & fakes
• When to use mock vs. fake
• Mocking libraries in various languages
  • Python: https://docs.python.org/3/library/unittest.mock.html
  • Java: https://github.com/mockito/mockito
  • C++:  https://github.com/google/googletest
    
    

172: Transformers and Large Language Models

Intro topic: Is WFH actually WFC?

News/Links:

• Falsehoods Junior Developers Believe about Becoming Senior
  • https://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/
• Pure Pursuit
  • Tutorial with python code: https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit 
  • Video example: https://www.youtube.com/watch?v=qYR7mmcwT2w 
• PID without a PHD
  • https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf
• Google releases Gemma
  • https://blog.google/technology/developers/gemma-open-models/
    
    

Book of the Show

• Patrick: The Eye of the World by Robert Jordan (Wheel of Time)
  • https://amzn.to/3uEhg6v
• Jason: How to Make a Video Game All By Yourself
  • https://amzn.to/3UZtP7b

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: Stadia Controller Wifi to Bluetooth Unlock
  • https://stadia.google.com/controller/index_en_US.html
• Jason: FUSE and SSHFS
  • https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh
    
    

Topic: Transformers and Large Language Models

• How neural networks store information
  • Latent variables
• Transformers
  • Encoders & Decoders
• Attention Layers
  • History
    • RNN
      • Vanishing Gradient Problem
    • LSTM
      • Short term (gradient explodes), Long term (gradient vanishes)
  • Differentiable algebra
  • Key-Query-Value
  • Self Attention
• Self-Supervised Learning & Forward Models
• Human Feedback
  • Reinforcement Learning from Human Feedback
  • Direct Policy Optimization (Pairwise Ranking)

Intro topic: Monitor setups

News/Links:

• BlueScuti, Willis, beats Tetris
  • https://www.youtube.com/watch?v=GuJ5UuknsHU
• PalWorld accused of being an AI Product
  • https://www.forbes.com/sites/paultassi/2024/01/22/palworld-accused-of-using-genai-with-no-evidence-so-far/?sh=26a9651b4239
• 4 Billion if-statements to determine if a number is even or odd
  • https://andreasjhkarlsson.github.io/jekyll/update/2023/12/27/4-billion-if-statements.html
• Seamless M4T
  • https://ai.meta.com/blog/seamless-m4t/
    
    

Book of the Show

• Patrick:
  • Foundation by Isaac Asimov
    • https://amzn.to/3SrmgnP
• Jason: 
  • Propaganda by Edward Bernays
    • https://amzn.to/47JUCXJ

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • The Room Game
    • https://www.fireproofgames.com/games/the-room
• Jason:
  • Incredibuild
    • https://www.incredibuild.com/
      
      

Topic: Compilers and Interpreters (Request by Jessica W.)

• Machine Code
  • Architecture Specific
• Assembly
  • Single vs Two Pass Compiler
• High level Languages
  • Intermediate Representation
  • JVM ByteCode vs Machine Code for portability
• Scripting/Interpreters
• JIT
• Profile Guided Optimization
• Resources
  • https://www.craftinginterpreters.com/
  • https://nandgame.com/
  • Turing Complete

Predictions:
Jason 

• VR for Work
• Lowering AI training cost/ improved efficiency
• RISC-V takeoff

Patrick

• Ai claim of AGI
• Ai peer reviewer
• Ai Video Generator
• More space vehicles reaching orbit

Early career, finding role at FAANG, liaising vs shipping code. Startup?

3 part. 1. How and when current hype for AI will end? 2. Shape of the show 3. Upcoming in tech

What are essential programmer knowledge items?

CS Student, how to organize life and goals? What purpose life should serve?

What kind of programmer were you in college?

Happy Holidays!

Intro topic: Testing your car battery

News/Links:

• Tech Layoffs still going on
  • https://www.sfchronicle.com/tech/article/google-layoffs-california-companies-18465600.php 
• Real-time dreamy Cloudscapes with Volumetric Raymarching
  • https://blog.maximeheckel.com/posts/real-time-cloudscapes-with-volumetric-raymarching/
• Robot Rascals
  • https://en.wikipedia.org/wiki/Robot_Rascals 
• Meta Quest 3 
  • https://www.theverge.com/23906313/meta-quest-3-review-vr-mixed-reality-headset
    
    

Book of the Show

• Patrick:
  • HyperLogLog Paper
    • https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf 
• Jason: 
  • Eureka! NVIDIA Research Breakthrough Puts New Spin on Robot Learning 
    • https://blogs.nvidia.com/blog/2023/10/20/eureka-robotics-research/ 

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Techtonica: https://store.steampowered.com/app/1457320/Techtonica/ 
• Jason:
  • ESP32 development board: https://amzn.to/3Qpmb20 
  • WEMOS

Topic: HyperLogLog

• Motivation
  • Cardinality Counting
• LinearCounting
  • Hash + expectation of collision based on how full
  • Bloom Filter
• LogLog
  • Use first N bits as bucket
  • Use max sequential 0s in each bucket
  • Average
• HyperLogLog
  • Handle empty buckets
  • Use correction factor like linear counting for low counts (number of empty buckets) and high counts
• Distributing
  • Transfer bucket counts

Intro topic: What are expectations on developers that have nothing to do with programming

News/Links:

• Pushing for a lower dev estimate is like negotiating weather with a Meteorologist
  • https://smartguess.is/blog/your-estimate-is-less-than-that/
• Announcing python in excel
  • https://techcommunity.microsoft.com/t5/excel-blog/announcing-python-in-excel-combining-the-power-of-python-and-the/ba-p/3893439
• 7 Habits of Highly Effective Software Engineers
  • https://makingsmallercircles.com/articles/7-habits-of-highly-effective-software-engineers/
• Raspberry pi 5 begins shipping
  • https://www.phoronix.com/news/Raspberry-Pi-5-Shipping
    
    

Book of the Show

• Patrick:
  • Harry Potter and the Sorcerer's Stone Illustrated Edition
    • https://amzn.to/3St3L35
• Jason: 
  • The Pete and Sebastian Show
    • https://podcasts.apple.com/us/podcast/the-pete-and-sebastian-show/id570256898

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Obsidian
    • obsidian.md
• Jason:
  • Ink by Inkle
    • https://github.com/inkle/ink
      
      

Topic: Godot

• What is a game engine?
  • Graphics, animation, particle effects
  • Tilemaps
  • Entity component systems
  • Physics
  • Sound
  • I/O
  • Input handling (touchscreen, joystick)
  • GDScript
• Why use a game engine?
  • Portability
  • Allow for testing individual components of the game
  • Libraries for ads, in-app-purchases
• Godot
  • Open source
  • Focused on 2-D but now many 3-D features
  • Great development environment
• AI Hero
  • Started in phaser (development rut)
  • Moved to Godot, rapid prototyping

Intro topic: Jogging Metrics

News/Links:

• Unholy LLM
  • https://huggingface.co/Undi95/Unholy-v1-12L-13B 
• The reverse red herring  
  • https://www.blameless.com/blog/the-reverse-red-herring
• The "ens–tification" of TikTok
  • https://www.wired.com/story/tiktok-platforms-cory-doctorow/ 
• Response letter to Godot is not the new Unity
  • https://sampruden.github.io/posts/godot-is-not-the-new-unity/
  • https://gist.github.com/reduz/cb05fe96079e46785f08a79ec3b0ef21
    
    

Book of the Show

• Patrick:
  • Math Games with Bad Drawings by Ben Orlin
  • https://amzn.to/48qlg9A
• Jason: 
  • The Invisible Hook: The Hidden Economics of Pirates
  • https://amzn.to/3LChBff

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • Factorio (Desktop Game)
• Jason:
  • AI Hero (iOS and Android)

Topic: Desktop user interfaces

• What is a user interface?
• Web and Mobile UI toolkits
• Desktop options
  • Qt
    • Cross platform, custom UI elements
    • Qt creator, code generator
  • WxWidgets
    • Cross platform, uses native UI elements 
    • WxFormDesigner, code generation
    • Can look different on different operating systems
  • Electron
    • Local nodejs webserver
    • Html/JavaScript technology
    • Requires interprocess communication to use other languages  
  • Jupyter notebooks
    • Mathematica-like notebook 
    • Not for distribution 
  • Streamlit
    • Python to web compiler
  • Game Engines
    • Unity, Godot, Unreal
• Tips for building desktop UI
  • UI is slow (startup time, interaction time)
  • Separate the UI from the engine & business logic

- pganalyze: https://pganalyze.com/

- Weekly series "5mins of Postgres": https://www.youtube.com/channel/UCDV_1Dz2Ixgl1nT_3DUZVFw

- How Postgres chooses which index to use: https://pganalyze.com/blog/how-postgres-chooses-index

- CMU databases courses: https://db.cs.cmu.edu/courses/

- Postgres community: https://www.postgresql.org/community/

As well as social links:

- Mastodon: https://hachyderm.io/@lukas

- Twitter/X: @pganalyze, @LukasFittl

- GitHub: @pganalyze, @lfittl

- LinkedIn: https://www.linkedin.com/in/lfittl/ , https://www.linkedin.com/company/pganalyze/

Intro topic: Revisiting the power of Spreadsheets

News/Links:

• LK-99 Isn’t a Superconductor
  • https://www.nature.com/articles/d41586-023-02585-7
• Normalizing Flows
  • https://pyro.ai/examples/normalizing_flows_i.html
• How is llama.cpp possible?
  • https://finbarr.ca/how-is-llama-cpp-possible/
• Chat with open source large language models
  • https://chat.lmsys.org/
    
    

Book of the Show

• Patrick: Math with Bad Drawings by Ben Orlin
  • https://amzn.to/44dsgDz
• Jason: 
  • Overboard! https://play.google.com/store/apps/details?id=com.InkleLtd.Overboard Nhl=en_US&gl=US 

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

• Patrick: 
  • ffmprovisr https://amiaopensource.github.io/ffmprovisr/
• Jason:
  • Pandas read_ods() read_excel()

Topic: Differential Equations

• Why should programmers learn about DiffEq
  • Law of Large Numbers
• What are differential equations?
  • When you know the rate of change
  • EigenVectors & EigenValues
  • What is Jacobian What is Jacobian? | The right way of thinking derivatives and integrals
    
    
• Special cases
  • Partial Differential Equations
  • Ordinary Differential Equations
• Why solvers are important
  • Numerical Stability at larger step sizes
  • Example: https://medium.com/@pukumarathe/eulers-method-and-runge-kutta-4th-order-method-in-python-b4a0068a8ebe 
• Fun Examples
  • Predator-Prey relationships in scipy
    • https://scientific-python.readthedocs.io/en/latest/notebooks_rst/3_Ordinary_Differential_Equations/02_Examples/Lotka_Volterra_model.html 
  • Physics Engines for games
    • https://youtu.be/52n2qKgwW_Q 
  • PageRank
    • https://arxiv.org/pdf/2001.08973.pdf