Test & Code

Code Coverage or Test Coverage is a way to measure what lines of code and branches in your code that are utilized during testing. 
 Coverage tools are an important part of software engineering. 

But there's also lots of different opinions about using it. 

• Should you try for 100% coverage? 
• What code can and should you exclude? 
• What about targets?

I've been asked many times what I think about code coverage or test coverage.
 This episode is a train of thought brain dump on what I think about code coverage.

We'll talk about:

• how I use code coverage to help me write source code
• line coverage and branch coverage
• behavior coverage
• using tests to ask and answer questions about the system under test
• how to target coverage just to the code you care about
• excluding code
• good reasons and bad reasons to exclude code

And also the Pareto Principle or 80/20 rule, and the law of diminishing returns and how that applies (or doesn't) to test coverage.

Links:

• Coverage.py
• pytest-cov

The Python extension for VS Code is most downloaded extension for VS Code. 
 Brett Cannon is the manager for the distributed development team of the Python extension for VS Code.

In this episode, Brett and I discuss the Python extension and VS Code, including:

• pytest support
• virtual environment support
• how settings work, including user and workspace settings
• multi root projects
• testing Python in VS Code
• debugging and pydevd
• jump to cursor feature
• upcoming features

Special Guest: Brett Cannon.

Links:

• Brett Cannon on Changelog
• John Wick
• Ballerina
• Frank Willison Award
• Python extension for VS Code
• episode 117 : How IDEs can make software testing easier - Paul Everitt
• User and Workspace Settings
• virtual environments
• Testing Python in VS Code
• pydevd
• Jump to Cursor in Feb VS Code Python blog

pytest plugins are an amazing way to supercharge your test suites, leveraging great solutions from people solving test problems all over the world. In this episode Michael and I discuss 15 favorite plugins that you should know about.

We also discuss fixtures and plugins and other testing tools that work great with pytest

• tox
• GitHub Actions
• Coverage.py
• Selenium + splinter with pytest-splinter
• Hypothesis

And then our list of pytest plugins:

1. pytest-sugar 
2. pytest-cov 
3. pytest-stress 
4. pytest-repeat 
5. pytest-instafail 
6. pytest-metadata
7. pytest-randomly
8. pytest-xdist 
9. pytest-flake8 
10. pytest-timeout 
11. pytest-spec 
12. pytest-picked 
13. pytest-freezegun 
14. pytest-check 
15. fluentcheck

That last one isn't a plugin, but we also talked about pytest-splinter at the beginning. So I think it still counts as 15.

Special Guest: Michael Kennedy.

Links:

• pytest-sugar — changes the default look and feel of pyptest (e.g. progressbar, show tests that fail instantly)
• pytest-cov — run coverage.py from pytest
• pytest-stress — allows you to loop tests for a user defined amount of time.
• pytest-repeat — for repeating test execution
• pytest-instafail — shows failures and errors instantly instead of waiting until the end of test session.
• pytest-metadata — for accessing test session metadata
• pytest-randomly — randomly order tests and control random.seed
• pytest-xdist — distributed testing
• pytest-flake8 — pytest plugin to run flake8
• pytest-timeout — terminate tests after a certain timeout
• pytest-spec — display test execution output like a specification
• pytest-picked — run the tests related to the changed files (according to Git)
• pytest-freezegun — easily freeze time
• pytest-check — allows multiple failures per test
• fluentcheck — fluent assertions
• episode 104 — Top 28 pytest plugins with Anthony Sottile
• Python Testing with pytest — The easiest way to get up to speed with pytest fast. There's also a chapter dedicated to plugins that also teaches you how to write and test your own plugins.
• tox
• GitHub Actions
• Coverage.py
• pytest-splinter — provides a set of fixtures to use splinter for browser testing
• splinter — makes it easy to write automated tests of web applications
• hypothesis — property-based testing
• Talk Python Episode #267 — This episode is a cross post with Talk Python

One of the great things about attending in person coding conferences, such as PyCon, is the hallway track, where you can catch up with people you haven't seen for possibly a year, or maybe even the first time you've met in person. 

Nina is starting something like the hallway track, online, on twitch, and it's already going, so check out the first episode of Python Tea.

Interesting coincidence is that this episode is kind of like a hallway track discussion between Nina and Brian.

We've had Nina on the show a couple times before, but it's been a while.

 In 2018, we talked about Mentoring on episode 44.
In 2019, we talked about giving Memorable Tech Talks in episode 71.

In this episode, we catch up with Nina, find out what she's doing, and talk about a bunch of stuff, including:

• Live Coding
• Online Conferences
• Microsoft Python team
• Python Tea, an online hallway track
• Q&A with Python for VS Code team
• Python on hardware
• Adafruit
• Device Simulator Express
• CircuitPython
• Tricking out your command prompt
• Zsh and Oh My Zsh
• Emacs vs vi key bindings for shells 
• Working from home

Special Guest: Nina Zakharenko.

Links:

• nnjaio - Twitch
• Nina Zakharenko 💜🐍 (@nnja) / Twitter
• Device Simulator Express - Visual Studio Marketplace
• Initial code for Microsoft's PyBadge at PyCon 2020
• Goodbye Print, Hello Debugger! - Nina Zakharenko - PyCon 2020 Talk 
• 𝙽𝚒𝚗𝚊 𝚉𝚊𝚔𝚑𝚊𝚛𝚎𝚗𝚔𝚘 💜🐍 - DEV.to
• Python Tea Announcement
• The Live Coders Conference

"The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers."

That's a lot of responsibility, and to that end, the PSF Board Directors help out quite a bit. 

If you want to be a part of the board, you can. There's an election coming up right around the corner and you gotta get your nomination in by May 31. You can also join the PSF if you want to vote for who gets to be part of the board. 

But what does it really mean to be on the Board, and what are some of the things the PSF does? 

To help answer those questions, I've got Ewa Jodlowska, the PSF Executive Director, and Christopher Neugebauer, a current board member, on the show today. I've also got some great links in the show notes if we don't answer your questions and you want to find out more.

Special Guests: Christopher Neugebauer and Ewa Jodlowska.

Links:

• Latest PSF Board Elections Discussion
• Python Software Foundation
• Overview of Elections
• Duties and Responsibilities of Directors
• Life as a Python Software Foundation Director - YouTube

Technical debt has to be dealt with on a regular basis to have a healthy product and development team.

The impacts of technical debt include emotional drain on engineers and slowing down development and can adversely affect your hiring ability and retention.

But really, what is technical debt? Can we measure it? How do we reduce it, and when?

James Smith, the CEO of Bugsnag, joins the show to talk about technical debt and all of these questions.

Special Guest: James Smith.

"Code is read much more often than it is written." - Guido van Rossum
 This is true for both production code and test code.

When you are trying to understand why a test is failing, you'll be very grateful to the test author if they've taken the care to make it readable.

David Seddon came up with 6 principles to help us write more readable tests.
 We discuss these, as well as more benefits of readable tests.

David's 6 Principles of Readable Tests:

1. Profit from the work of others
2. Put naming to work
3. Show only what matters
4. Don’t repeat yourself
5. Arrange, act, assert
6. Aim high

Special Guest: David Seddon.

Links:

• How to write readable tests (presentation) · David Seddon
• How to write readable tests (slides)
• pytest
• WebTest 
• factory_boy
• django-webtest

In both unittest and pytest, when a test function hits a failing assert, the test stops and is marked as a failed test. 

What if you want to keep going, and check more things?

 There are a few ways. One of them is subtests.

Python's unittest introduced subtests in Python 3.4.

 pytest introduced support for subtests with changes in pytest 4.4 and a plugin, called pytest-subtests.
 Subtests are still not really used that much.

But really, what are they? When could you use them?

 And more importantly, what should you watch out for if you decide to use them?

That's what Paul Ganssle and I will be talking about today.

Special Guest: Paul Ganssle.

Links:

• Subtests in Python — Paul's article on subtests
• subtests in unittest - Python docs 
• pytest-subtests: plugin to support subtests in pytest
• pytest-check: A pytest plugin that allows multiple failures per test.

Django supports testing out of the box with some cool extensions to unittest. However, many people are using pytest for their Django testing, mostly using the pytest-django plugin.

Adam Parkin, who is known online as CodependentCodr, joins us to talk about migrating an existing Django project from unittest to pytest. Adam tells us just how easy this is.

Special Guest: Adam Parkin.

Links:

• Django Tutorial, on testing
• The Django docs on testing & the test database
• The Django docs on the TestCase class
• pytest-django docs

Financial services have their own unique testing development challenges. But they also have lots of the same challenges as many other software projects. 

Eric Bergemann joins Brian Okken to discuss:

• Specific testing challenges in the financial services domain
• CI/CD : Continuous Integration, Continuous Deployment
• TDD : Test Driven Development
• Confidence from testable applications
• Testing strategies to add coverage to legacy systems
• Testing the data and test cases themselves
• DevOps
• Continuous testing
• Manual testing procedures
• BDD & Gherkin
• Hiring in vs training industry knowledge

Special Guest: Eric Bergemann.

Links:

• Paragon
• The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win