Test & Code

Nicole is a software engineer and writer, and recently wrote about the trade-offs we make when deciding which tests to write and how much testing is enough.

We talk about:

• Balancing schedule vs testing
• How much testing is the right about of testing
• Should code coverage be measured and tracked
• Good refactoring can reduce code coverage
• Is it worth testing error conditions?
• Are rare error codes ok to just monitor?
• API drift and autospec
• Mitigating risk
• Deciding what to test and what not to test
• Focus testing on key money-making features 
• If there's a bug in this part of the code, how much business impact is there?
• Performance testing needs to approximately match real world workloads
• Cost of a service breaking vs the cost of creating, maintaining, and running tests
• Keeping test suites quick to minimize getting distracted

Links:

• Too much of a good thing: the trade-off we make with tests 
• Load testing is hard, and the tools are... not great. But why?
• Yet Another Rust Resource (YARR!)
• Goodhart's law - "When a measure becomes a target, it ceases to be a good measure"

If you've ever thought about starting a podcast or a SaaS project, you'll want to listen to this episode.
 
Justin is one of the people who motivated me to get started podcasting.
He's also running a successful SaaS company, transistor.fm, which hosts this podcast.

Topics:

• Podcasting
• Building new SaaS (software as a service) products
• Balancing work, side hustle, and family
• Great places to snowboard in British Columbia

BTW.

Links from the show:

• Transistor.fm - excellent podcast hosting, Justin is a co-founder
• How to start a podcast in 2024
• Podcasts from Justin
  • Build your SaaS - current
  • Build & Launch - an older one, but great
  • MegaMaker - from 2021 / 2022

Charlie Marsh and team are using Rust to make Python tooling faster.

• Ruff can take the place of Flake8, isort, and Black, and so much more.
• uv can take the place of pip, pip-tools, and virtualenv
• Astral is Charlie's venture backed company, and what they have with `ruff` and `uv` is just the start.

Since uv is the newest tool, there's quite a bit of the discussion diving into uv.

Links:

• ruff
• Astral
• uv

Software engineers that move into leadership roles have a struggle between learning leadership skills, maintaining technical skills, and learning new leadership and technical skills. 
Matt Makai went from individual contributor to developer relations to leadership in devrel. 
We discuss how to stay technical, as well as dive into some results of his studies in how companies use developer relationship channels.

If you haven't tried running automated tests, especially with pytest,  in VS Code recently, you should take another look.
The Python for VS Code interface for testing, especially for pytest, has changed recently. 
On this episode we discuss the change with the software engineer working on the pytest changes, Eleanor Boyd, and the product manager, Courtney Webster. 

Links from the episode:

• Blog post announcing the rewrite  
• Code repo for questions, comments, issues etc

If a test fails in a test suite, I'm going to want to re-run the test. I may even want to re-run a test, or a subset of the suite, a bunch of times.  
There are a few pytest plugins that help with this:

• pytest-repeat
• pytest-rerunfailures
• pytest-flakefinder
• pytest-instafail

We talk about each of these in this episode.

In 2002, Kent Beck released a book called  "Test Driven Development by Example".
In December of 2023, Kent wrote an article called "Canon TDD".
With Kent's permission, this episode contains the full content of the article.

Brian's commentary is saved for a followup episode.

Links:

• Canon TDD
• Test Driven Development by Example

We want to be able to run tests in a suite, and debug them in isolation, and have the behavior be the same.  
If the behavior is different in isolation vs in a suite, it's a nightmare to debug. 

In this episode, we'll talk about:

• Causes of dependence
• Testing for dependencies using plugins
• Debugging test dependencies

Plugins discussed:

• pytest-randomly
• pytest-reverse
• pytest-random-order

Test Driven Development. Red, Green, Refactor. 

• Do we have to do the refactor part? 
• Does the refactor at the end include tests? 
• Or can I refactor the tests at any time?
• Why is refactor at the end? 

This episode is to talk about this with a an example.

How do you test the argument parsing bit of an application that uses argparse?

This episode covers:

• Design for Test: Structuring your app or script so it's easier to test.
• pytest & capsys for testing stdout
• Adding debug and preview flags for debugging and testing
• And reverting to subprocess.run if you can't modify the code under test

Also, there's a full writeup and code samples available:

• Blog post: Testing argparse Applications 
• Code Repo