Test & Code

50: Flaky Tests and How to Deal with Them


Listen Later

Anthony Shaw joins Brian to discuss flaky tests and flaky test suites.

  • What are flaky tests?
  • Is it the same as fragile tests?
  • Why are they bad?
  • How do we deal with them?
  • What causes flakiness?
  • How can we fix them?
  • How can we avoid them?
  • Proactively rooting out flakiness
  • Test design
  • GUI tests
  • Sharing solutions
  • Special Guest: Anthony Shaw.

    Links:

    • Dropbox article on flaky tests
    • Microsoft article on flaky tests
    • pytest-rerunfailures: a py.test plugin that re-runs failed tests up to -n times to eliminate flakey failures
    • pytest-randomly: Pytest plugin to randomly order tests and control random.seed
    • pytest-random-order: pytest plugin to randomise the order of tests with some control over the randomness
    • math.isclose()
    • numpy.isclose()
    • pytest.approx() — approx
    • Anthony's testing article on RealPython
    • Ghost Inspector
    • wily: A Python application for tracking, reporting on timing and complexity in tests
    ...more
    View all episodesView all episodes
    Download on the App Store

    Test & CodeBy Brian Okken

    • 4.7
    • 4.7
    • 4.7
    • 4.7
    • 4.7

    4.7

    70 ratings


    More shows like Test & Code

    View all
    Radiolab by WNYC Studios

    Radiolab

    44,019 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    624 Listeners

    Heavy Networking by Packet Pushers

    Heavy Networking

    328 Listeners

    Talk Python To Me by Michael Kennedy

    Talk Python To Me

    588 Listeners

    Python Bytes by Michael Kennedy and Brian Okken

    Python Bytes

    214 Listeners

    Darknet Diaries by Jack Rhysider

    Darknet Diaries

    7,999 Listeners

    The Real Python Podcast by Real Python

    The Real Python Podcast

    141 Listeners

    Unexplainable by Vox

    Unexplainable

    2,278 Listeners

    Network Automation Nerds by Packet Pushers

    Network Automation Nerds

    3 Listeners