Agile Thoughts

​At this point, I hope you got the ideas of why overdoing the upper floors of the pyramid will cause it to tumble. In the previous episode I mentioned you can get a test pyramid worksheet. This worksheet can be used to plan out how to build your project's mighty pyramid of test.

The post 007 Size Matters first appeared on Agile Thoughts.

Visit Agile Thoughts and register to receive free development, analysis, or leadership and management materials and learn to excel at developing software. I’ll also send information on my low cost email courses you can take via the internet.

At this point, I hope you got the ideas of why overdoing the upper floors of the pyramid will cause it to tumble. In the previous episode I mentioned you can get a test pyramid worksheet. This worksheet can be used to plan out how to build your project’s mighty pyramid of test.

To help make this more quantifiable, and give an example of how you can do this, let me talk through some numbers using the worksheet. So if you have it, get it out of your email. If you’ve been delaying to register your email, consider doing that now and a worksheet will show up in your email box within a minute or so. Otherwise, just give me a listen. Here are some typical numbers I see with well oiled Agile teams. Your team’s results may match, or be more or less. The important part is the ratio between the floors of the pyramid.

For example, a team works for a 2 week iteration and finishes 4 user stories. Those stories required 2 UI test, 8 subcutaneous tests, 20 micro tests and their code coverage > 90% for newly created code.

Products that achieve this can live forever as their design is kept up to date and can quickly respond to market changes.b If you’d like help in learning how to develop code using TDD, ATDD, or BDD, and interested in receiving free articles and videos, such as the test pyramid worksheet type http://AgileNoir.biz/AgileThoughts (or http://agilenoir.biz/敏捷理念) into your browser and fill out the contact form. When you do that, I’ll send you the test pyramid worksheet today. Later you’ll get more free videos and articles about test automation and offers of low cost email courses such as how to build the test automation to make your pyramid of test a reality.

​

The post 006 Riddles of the Sphinx and Answers within the Pyramid first appeared on Agile Thoughts.

Visit Agile Thoughts and register to receive free development, analysis, or leadership and management materials and learn to excel at developing software. I’ll also send information on my low cost email courses you can take via the internet.

Sphinx, a mythological creature with the head of a man and the body of a lion, loves to ask questions and demands answers. If you don’t answer the questions correctly, the Sphinx eats you. In IT, the closest to a Sphinx are QA or Release managers, who although may not eat you, will ask questions in order to decide if your product is in good shape.

Thankfully, the automated tests within our unbreakable pyramid exist to answer questions about the quality of the product. Knowing these questions will organize your understanding so when a riddle about software quality is poised, you know how to use the Pyramid to get an answer.

Naturally, if you’ve only a shanty of automated tests to evaluate a mountain of product code, the Sphinx is going to knock over your cardboard house with a swat of its paw. But keep at it! For every story, automate it’s acceptance criteria with ATDD or BDD. For every code change in a class or function, write the unit test first using TDD. And in a few months to a year, you’ll have a pretty decent pyramid that can stand up to the Sphinx.

These are the important, big ideas of test automation. If you’d like help in either learning how to develop code using TDD, ATDD, or BDD, and interested in receiving free articles and videos and offers of low cost email courses you can take, visit AgileNoir.biz/AgileThoughts (or http://agilenoir.biz/敏捷理念/) and receive a free test pyramid worksheet today!

Teams have found that just because acceptance criteria COULD be made into hard to maintain and unreliable UI test, many of them actually don't need to be executed via the UI but instead in some sub layer beneath.

The post 005 Subcutaneous Tests first appeared on Agile Thoughts.

Visit Agile Thoughts and register to receive free development, analysis, or leadership and management materials and learn to excel at developing software. I’ll also send information on my low cost email courses you can take via the internet.

Teams have found that just because acceptance criteria COULD be made into hard to maintain and unreliable UI test, many of them actually don’t need to be executed via the UI but instead in some sub layer beneath.

A flight booking service adding a new carrier, for example, doesn’t need a dozen UI tests to check if their website displays the flights from the new Shanghai Airlines feed. Instead they add one UI test into the penthouse that checks the Shanghai airlines logo comes up and the rest of the acceptance tests talk directly against the feed API itself or test the handler of the feed (we put the test near to the code that has the functionality).  A lot of macro testing can be done without the UI or even without many upstream system/service dependencies. This secret middle floor is windowless as it doesn’t use the UI yet frequently requires deploying and launching a new process.

This middle floor is important. Without it, we’ll have too many UI tests in the penthouse, economics cause the pyramid to topple.  The signs that your pyramid is toppling is is when more macro tests are failing than passing and people start disabling tests in order to cheat the build onto the pipeline. Subcutaneous tests are a key innovation!  Because they are practically impossible to do with manual test strategies, many organizations with manual Test/QA habits have a giant blind spot in this area.

There are many tools for building subcutaneous tests: SoapUI, Rest Assured, Restito, WireMock.  Other, non API example are making database connections directly to the stored procedure that contains the new functionality, and writing tests that link directly to the GUI controller.  All in all, the strategy is to get the test automation as close as possible to the code that supports the new functionality.

Next episode we’ll take another holistic look at the pyramid and talk about what questions these tests are designed to answer.

Images of the test automation pyramid are available online and you'll see good agreement that macro tests are at the top and micro tests in the bottom. But there's a lot of mystery about what goes into the middle floor. I've put on my leather hat and brought my bullwhip and explored the automation pyramids at many other companies.

The post 004 The Pyramid’s Secret Floor first appeared on Agile Thoughts.

Learn development, analysis, or leadership and management skills to do what’s covered in the podcast by registering your email address with Lance(r) Kind. Every few weeks he’ll send you free study materials and occasionally (like monthly, weekly at the most) offer low cost trainings you can take via email. You’re the boss and can unsubscribe at any time. Don’t worry, I’ll not share your contact info. Not ready for that and want to listen to more Agile Thoughts? No problem. I’m the same way when giving up my email address. A new agile thought is produced every two weeks so you’ll find it convenient to subscribe via iTunes link, Google Play link, or RSS feed so each episode automatically arrives on your phone or computer. Still wondering what Agile Thoughts is all about? Browse the podcast archive to see if you discover something valuable.

Images of the test automation pyramid are available online and you’ll see good agreement that macro tests are at the top and micro tests in the bottom. But there’s a lot of mystery about what goes into the middle floor. I’ve put on my leather hat and brought my bullwhip and explored the automation pyramids at many other companies. I’ve learned that some put their service API tests here, others put DB dependent tests or any tests that cross boundaries such as process and network. But everybody’s agreed that they can maintain thousands of simple microtests at the bottom floor but only a few UI and full stack tests in the glass penthouse at the top (so we can keep an eye on this expensive, LV bag toting lot) and this middle, secret floor, is where we put as many macro tests as we can that don’t require a UI. So the secret floor is inhabited with a diverse lot of “subcutaneous” tests.

While the bottom of the test pyramid is biggest enough to house a few soccer fields and a monster truck rally, the top floor of the pyramid only has room for a long long table, a big screen tv, and a minibar fridge. Oh yeah, and macro tests.

The post 003 Macrotests at the Top of the Pyramid first appeared on Agile Thoughts.

Learn development, analysis, or leadership and management skills to do what’s covered in the podcast by registering your email address with Lancer Kind. Every few weeks he’ll send you free study materials and occasionally offer low cost email or trainings. You’re the boss so can unsubscribe at any time. Don’t worry, I’ll not share your contact info.

While the bottom of the test pyramid is big enough to house a few soccer fields and a monster truck rally, the top floor of the pyramid only has room for a  pool table, a big screen tv, and a minibar fridge. Oh yeah, and macro tests.

These high maintence tests require at least part of the application to be deployed, network and DB connections and operate slowly, but they can confirm the functionality works as the customer expected.

All organizations are familiar with how to manually execute such tests using people operating the UI or via API testing tools such as SoapUI and SQL consoles, so these are the first tests that come to mind when organizations think about building test automation. But these tests have the highest cost to implement and maintain, and have the least value due to taking hours to days to report pass or fail. This has resulted in companies with a test automation pyramid flipped on its top because they’ve over invested in macro tests and have barely invested in micro tests.  This is a costly mistake. Buying Macro tests are like shopping with high interest-rate credit cards because they cost a lot to maintain. To avoid going broke, only build them for situations that can’t be checked with micro tests.

Economics is what  gives us a test pyramid, were the bottom (the part largest in area) are unit tests, the top (smallest in area) are macro tests that depend on the UI, DB, and other system dependencies.  Upside down pyramids are very expense and likely to tumble.

A good strategy for building macro tests is to automate acceptance criteria for User Stories. We’ll cover User Stories and acceptance tests in a future episode. Before that, prepare yourself for the next episode, where we reveal the secret middle floor of the pyramid–it’s a secret floor and the hardest to describe. The only question is, will it be filled with treasure or heart ache?