I was pondering the testing practices for software, particularly automated testing. The documentation and literature often reference Mike Cohn’s testing pyramid and Brian Marick’s testing matrix. In an effort to highlight the speed and cost of testing—factors that rarely get mentioned—I found myself sketching out this quadrant that I’ve dubbed the Software Factory Testing Quadrant.

The dimensions

The dimensions that make it up are:

• Cost (expensive / cheap): This refers to how costly tests are to write, as well as their maintenance. We’re talking about effort here, but there’s a strong correlation with actual economic cost.
• Execution speed (slow / fast): This dimension looks at how quickly a single test runs.

The quadrants

Let’s break down who occupies each quadrant.

In the first quadrant (expensive/fast), I couldn’t find any test types that fit, as I don’t come across any testing techniques that are both costly and quick.

In the second quadrant (cheap/fast), I included unit tests and integration tests. They can take anywhere from tenths of a second to a few seconds each, are written alongside development, and are the most economical to create and manage.

In the third quadrant (cheap/slow), we have the broad category of load tests. With modern tools (like k6), they are affordable to write and developer-friendly, but they do require a portion of the system to be released, and their execution times can stretch to several minutes for stress tests.

Finally, in the fourth quadrant (slow/expensive), you find manual tests. They can’t be repeated, require the entire system to be fully operational, and people generally dread performing them. The added value is nearly zero, and their applicability in “Lean” contexts is practically nonexistent.

Conclusion

In conclusion, navigating the landscape of software testing can feel like a maze, but understanding these quadrants helps illuminate the path. By identifying where different testing methods fall, you can make more informed decisions about how to allocate resources, balance speed with cost, and ultimately enhance your software quality. So whether you’re diving into unit tests, embracing modern load testing tools, or grappling with the challenges of manual testing, keep this Software Factory Testing Quadrant in mind. It’s all about finding the right mix for your team and project.

Happy testing!