During the last several years, the testing process has evolved considerably, and the testing principles that govern them have evolved as well. Software testing is a complex activity, and a tester should understand these testing principles to have a broader understanding of the testing process.
In this article, we will focus on The Seven Software Testing Principles.
What are the Software Testing Principles?
Software testing strives to achieve efficiency, and consistently evolve to make the test process effective while reducing the time and cost. We have seven basic principles in software testing. These testing principles have evolved over a period of time and widely accepted as the common guideline for all testing.
Testing shows the presence of defects, not their absence:
The purpose of software testing is to detect software failures. Software testing can prove the presence of defects, but no amount of testing can prove that the software is free of defects. Even multiple test phases and several rounds of testing cannot guarantee that the software is 100% bug-free. Efficient Testing techniques reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.
For example, if we see many swans swimming in a pond and observe that most of them are white. We cannot claim ‘All swans are white’ but as soon as we see one black swan, we can say ‘Not all swans are white’. In the same way, we may execute many tests without finding a bug, but we cannot claim that ‘there are no bugs’. As soon as we find a bug, we can show ‘This code is not bug-free’.
Exhaustive testing is impossible:
Exhaustive testing is the process of testing the functionality of the software where all possible inputs and their combinations are run along with different preconditions.
Consider an example where you are testing a flight booking application. The source and destination fields have got three dropdowns each. The first dropdown has ’10’ countries. On the selection of a country, the second dropdown will populate ’10’ states. On the Selection of a state, the third dropdown will display ’10’ cities. Imagine the number of permutations that these combinations will generate (10 ^ 6).
Instead of attempting to test exhaustively, Risk Analysis and Business Prioritization should be used to minimize the test efforts which further helps in saving cost, time, resources, etc. Risk assessment helps in deciding how much testing is enough. It takes into consideration: the level of risk, including technical and business risks related to the product and project constraints such as budget and time.
Business Prioritization helps in focusing on critical areas, and test efforts can be distributed based on this prioritization
Early testing saves time and money:
To find the defect early in the software life-cycle, you must start static and dynamic test activities as early as possible as it is much cheaper to change the product at the beginning of the development life cycle than in the final stages of the project. The developer needs less time and effort to fix the defects detected in the early phases since a small part of the module needs to be modified. Once the code is written, programmers and testers often run a set of tests so that they can identify and correct defects in the software.
Defects cluster together:
Defect clustering in software testing means that a small module or functionality contains most of the errors or has the most operational failures. A phenomenon that many testers have observed is that defects tend to cluster. This can happen because one area of the code is particularly tricky and complicated, or because the change of software and other products tends to cause regression defects. Testers use this information when doing a risk assessment for the planning of the tests, and will focus on these clusters. According to the Pareto Principle (Rule 80-20), “80% of the defects come from 20% of the modules and the remaining 20% defects are distributed across the rest 80% of the modules”. So, we should emphasize the testing in 20% of the modules where we face 80% of the errors.
Defects cluster change over time, as the functionality becomes more stable. The testing team should always be on the lookout to find new clusters.
Beware of the Pesticide paradox:
The Pesticide Paradox in software testing is the process of repeating the same test cases over and again, eventually, these test cases will no longer find new defects. If we are using the same pesticides, again and again, it will no longer kill the insects. Then, to overcome this Pesticide Paradox, it is necessary to review the test cases regularly and add new test data/inputs or update them to find more defects. As the cluster for the bugs is cleaned up, we must shift our focus to another place, to the next set of risks. Over time, our focus may change from finding coding errors to looking at the requirements and design documents for defects and then improve the processes to prevent defects in the product.
Testing is context-dependent:
It means the test approach depends on the context of the software. For instance, the way you test a POS (Point of Sale) system at a brick and mortar store will be different from the way you are going to test an ATM machine. Similarly, testing a mobile application requires a different test approach than testing a desktop web application.
The risk profile of every software differs and hence the test techniques and test efforts also vary. A flight control software would have a very low appetite for open defects as compared to an e-commerce application. The flight control software cannot afford undetected defects, as it can prove to be life-threatening, and hence additional testing efforts might be required.
Absence-of-errors is a fallacy:
According to the first and second testing principle, it is impossible to run all possible tests and find all possible defects. Further, it is a mistaken belief that finding and fixing a large number of defects is key to the success of a system. For example, sometimes 99% of the bug-free software can remain unusable if the wrong requirements are incorporated into the software. The software we develop must not only have minimal defects, but it must also meet the needs of the business, otherwise, it becomes unusable.
The software test is a vital element in the Software Development Life Cycle (SDLC) and can provide excellent results if done correctly and effectively. Unfortunately, software testing is often less formal because we lack to follow best practices, methodologies, principles, standards for optimal software testing. To accomplish testing goals, the maximum implementation of testing principles in real-world software development is necessary. This can be achieved only when everyone involved in the project must be familiar with the basic principles of software testing.
