Test & Code

"There are five practical reasons that we write tests. Whether we realize it or not, our personal testing philosophy is based on how we judge the relative importance of these reasons." - Sarah Mei

This episode discusses the factors.

Sarah's order:

1. Verify the code is working correctly
2. Prevent future regressions
3. Document the code’s behavior
4. Provide design guidance
5. Support refactoring

Brian's order:

1. Verify the code is working correctly
2. Prevent future regressions
3. Support refactoring
4. Provide design guidance
5. Document the code’s behavior

The episode includes reasons why I've re-ordered them.

Links:

• Five Factor Testing - Sarah Mei

A recent Twitter thread by Simon Willison reminded me that I've been meaning to do an episode on the testing trophy.
 This discussion is about the distinction between unit and integration tests, what those terms mean, and where we should spend our testing time.

Links:

• Simon Willison's Twitter Thread
• The Testing Trophy and Testing Classifications — Kent C Dodds
• Write tests. Not too many. Mostly integration. — Kent C Dodds
• On the Diverse And Fantastical Shapes of Testing — Martin Fowler

The idea of having a software as a service product sound great, doesn't it?

 Solve a problem with software. Have a nice looking landing page and website. Get paying customers.

 Eventually have it make enough revenue so you can turn it into your primary source of income. 

There's a lot of software talent out there. We could solve lots of problems. 
 But going from idea to product to first customer is non-trivial. 
 Especially as a side hustle. 
 This episode discusses some of the hurdles from idea to first customer. 

Brandon Braner is building Released.sh. It's a cool idea, but it's not done yet. 

Brandon and I talk about building side projects:

• finding a target audience
• limiting scope to something doable by one person
• building a great looking landing page
• finding time to work on things
• prioritizing and planning
• learning while building
• even utilizing third party services to allow you to launch faster
• and last, but not least, having fun

Special Guest: Brandon Braner.

Links:

• Released
• Tailwind CSS 
• Tailwind UI
• Figma
• Heroku
• Google App Engine

• Who should do QA?
• How does that change with different projects and teams?
• What does "doing QA" mean, anyway?

Answering these questions are the goals of this episode.

Links:

• Test Automation - Who Should be Involved? | Thoughtworks

In this episode, I talk with Paul Ganssle about a fun workflow that he calls pseudo-TDD.
 Pseudo-TDD is a way to keep your commit history clean and your tests passing with each commit.
 This workflow includes using pytest xfail and some semi-advanced version control features.

Some strict forms of TDD include something like this:

• write a failing test that demonstrates a lacking feature or defect
• write the source code to get the test to pass
• refactor if necessary
• repeat

In reality, at least for me, the software development process is way more messy than this, and not so smooth and linear.

Pauls workflow allow you to develop non-linearly, but commit cleanly.

Links:

• A pseudo-TDD workflow using expected failures
• episode 171: How and why I use pytest's xfail - Paul Ganssle
• episode 165: pytest xfail policy and workflow
• episode 162: Flavors of TDD

In the preface of "Python Testing with pytest" I list some reasons to use pytest, under a section called "why pytest?". Someone asked me recently, a different but related question "why NOT unittest?".

unittest is an xUnit style framework. For me, xUnit style frameworks are fatally flawed for software testing.

That's what this episode is about, my opinion of 

• "Why NOT unittest?", or more broadly, 
• "What are the fatal flaws of xUnit?"

Links:

• Python Testing with pytest, Second Edition
• unittest docs
• unittest assert methods
• xUnit - Wikipedia

A prototype is a a preliminary model of something, from which other forms are developed or copied.
 In software, we think of prototypes as early things, or a proof of concept.
 We don't often think of prototyping during daily software development or maintenance. I think we should.
 This episode is about growing better designed software with the help of a prototype mindset.

Links:

• Selecting a programming language can be a form of premature optimization — Brett Cannon's blog post

Paul Ganssle, is a software developer at Google, core Python dev, and open source maintainer for many projects, has some thoughts about pytest's xfail.
 He was an early skeptic of using xfail, and is now an proponent of the feature.
 In this episode, we talk about some open source workflows that are possible because of xfail. 

Special Guest: Paul Ganssle.

Links:

• How and why I use pytest's xfail — Paul's blog post mentioned in the episode
• Craft Minimal Bug Reports — Matthew Rocklin's article
• episode 111: Subtests in Python with unittest and pytest - Paul Ganssle
• episode 165: pytest xfail policy and workflow
• episode 166: unittest expectedFailure and xfail

Prayson Daniel, a principle data scientist, discusses testing machine learning pipelines with pytest.

Prayson is using pytest for some pretty cool stuff, including:

• unit tests, of course
• testing pipeline stages
• counterfactual testing
• performance testing

All with pytest. So cool.

Special Guest: Prayson Daniel.

Links:

• Python Bytes 250, with Prayson Daniel — Listen to this for more of an introduction to Prayson

Performance monitoring and error detection is just as important with services and microservices as with any system, but with added complexity. Omri Sass joins the show to explain telemetry and monitoring of services and of systems with services. 

Special Guest: Omri Sass.