Software Testing : Why it’s important and the 7 Software testing principles

Since we are living in a digital era, most of us are aware of what testing is? In this section, I am going to introduce what is testing and why testing is important. We will try and understand the difference between the error, defect, and failure.

What is Testing?

Testing helps us to measure the quality of the software. One role of testing is to ensure key functional and non-functional requirements are examined before the system runs into the service.

Let’s understand the definition for testing according to the ISTQB.

Testing is a process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation, and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.

Let’s break the definition for more understanding. According to the definition, testing is a process with some objectives of a test process:

Process: It means that testing is a process which involves multiple set of activities, not a single activity.
All life cycle activities: Testing as a process itself is a part of the software development life cycle (SDLC) and testing itself has a lifecycle known as software testing life cycle.
Both static and dynamic: Testing where the software code is executed and tests are run according to the requirements are dynamic testing whereas static testing involves testing and finding defects without executing the code.
Planning: Since testing is a process, we need to manage everything prior to the start of testing, and after the closure of testing activities.
Preparation: In this part, we will be designing test cases and select test conditions.
Evaluation: As the name says we must check software under test as well as the exit criteria and decide whether it is meeting the requirements.
Software products and related work products: We are not just testing the product code but we are also testing the requirement and design specifications. Static and dynamic testing both are needed to cover the range of products we need to test.
Determine that they (software products) satisfy specified requirements: The testing we do is mainly focused on meeting the requirements. If the product meets the specification, we can involve the stakeholders to judge the quality and decide whether it is ready to use.
Demonstrate that they (software products) are fit for purpose: ‘Fit for purpose’ is different from ‘satisfying the requirements’, how? Satisfying the specified requirements is meeting the specified requirements but fitting the purpose means whether it’s ready for the users to use, whether the product is serving the need of the users and if it is meeting the expectations of the users or not.
Detect defects: We often think that testing means detecting defects. Finding and fixing defects not only improves the quality of the software but also helps us to understand the risk associated with the defect.

So, now we know why the common perception is just incomplete that testing is to only run the tests and find defects. Moreover, testing is a set of processes that assesses the quality of the product and risks involved in that product.

Now, that we know what testing is, let’s understand why it is important?

Why Testing is important?

When we are testing something, we are checking whether it is OK. Testing is necessary because we all make mistakes, we are fallible. Some of those mistakes can be ignored but some can be expensive or dangerous. We need to check each and everything prior to its usage by the end-users.

If someone makes an error in using the software, the software will not behave as expected. People design and develop the software and they can make mistakes. These defects mean that there are flaws in the software. These are called defects (bug or fault). When the software is built and executed, these defects may cause the system to fail, deviating it from its expected delivery, causing a failure.

Basically, summing up error, defect, and failure, an error leads to a defect, and a defect leads to the failure.

Till now, we have understood that testing is necessary to minimize the risk of failure. Now let’s understand why testing is necessary to ensure the quality of the software.

As per ISTQB,

Quality is the degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.

Testing helps us to improve the quality of the software by finding defects, running tests for both functional and non-functional attributes of the software. We as developers and testers may see the quality in finding bugs, and meeting the requirement specification, but that’s not enough for a quality product handed over to end-users. Moreover, if our customers won’t perform their desired tasks, then our definition of quality will not be appropriate here.

Understanding the root causes of the defects is an important aspect of quality assurance activities and testing contributes by helping us to identify the defects in early stages. Detecting defects in the early stages leads to lesser defects in the software system, which particularly leads to the improvement in the software systems.

Now, that we understood the quality of the software, let’s understand a very common misunderstanding that testing and debugging are the same.

Debugging is a process of fixing a bug found in the software after testing.

The activity begins after the software fails to execute properly and successfully testing the product.
It is basically performed by either developers or programmers.
It can’t be done without the proper design knowledge.
Debugging is always manual. It can’t be automated.

Whereas,

Testing is a process of verification and validation of the software.

Testing is done by software testers in order to ensure that changes and corrections are checked for their effect on other parts of the components or systems.
It can be done without the proper design knowledge.
Testing can be manual or automated.

Now that we have understood what is testing and why testing is necessary, let’s learn that how much testing is enough?

How much testing is necessary?

Now, that we know that what software testing is and why it is important, We need to understand how much testing is necessary to make sure that the product is good to go.

If we are asked to test the software, what comes to our mind first is to test everything. At that moment, we have certain choices to make whether to test everything, test nothing or to test certain features of the software system.

For example, If we want to test for a one-digit numeric field, what will be the complete set of tests to run? So, there are 10 possible numeric fields as valid tests to run and invalid tests include 26 uppercase letters of alphabets and 26 lowercase and at least 6 special characters, in total, 68 tests to run, which will be a bit tedious work to do. Moreover, if you are going to consider some of the realistic situations like a page consisting of 15 entries, all having 5 possible tests to run, then the total tests will be 5¹⁵(30517578125)tests in total, which is never going to meet any of the deadlines in the project.

Let’s look at the below points to conclude how much testing is necessary :

We cannot test everything.
There is a level of quality that is acceptable for a given system.
The most important aspect of achieving the acceptable result of testing is Prioritization.
Most important testing should include functional and non-functional testing.
Deciding how much testing is enough depends upon the business, technical and product risks.
How much testing needs to be done should not be dependent on the size of the team.

Summarizing, Testing everything is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts

Exhaustive testing (complete testing) A test approach in which the last suite comprises all combinations of input values and preconditions.

Going forward, let’s understand the testing principles.

The 7 Testing Principles

There are seven principles of software testing:

Testing shows presence of defects
Exhaustive testing is possible
Early testing
Defect clustering
Pesticide paradox
Testing is context-dependent
Absence-of-errors fallacy

Now, let’s learn each of the testing principles one by one and understand why these software testing principles are important to know before you start testing any software/system.

Principle 1: Testing shows presence of defects

Testing can show that defects are present, but cannot prove that there are no defects.
Testing reduces the Probability of defects, even if there are no defects found, It doesn’t mean that the software is defect-free
Main purpose of the testing is to find bugs.
Tests should be designed in order to find maximum defects.

Principle 2: Exhaustive testing is impossible

Testing all data sets are impossible.
Testing everything would not be feasible except for trivial cases.
Exhaustive testing would cost more time and money.
Testing everything would consume more resources and time and may not meet the project’s dealine.
The focus should be more on prioritizing the tests according to the risk involved.

Principle 3: Early Testing

Testing should start as early as possible.
Testing should be done parallelly with development so that the testers don’t have to wait until the software is available to test.
Defects found in the early stages will save more time and money in later stages.

Principle 4: Defect Clustering

It is also known as “Pareto principle” and it says that approximately 80% of defects are found in about 20% of the module.
It depends on system/software complexity, volatile code, effects of change upon change, development staff experience, development staff inexperience.

Principle 5: Pesticide Paradox

If the same set of tests are repeated over and over again, there will be no new defects found.
To overcome this problem, test cases need to reviewed and revised regularly.
Revising and reviewing test cases will help in finding new defects.

Principle 6: Testing is context-dependent

Testing is done differently in different circumstances/contexts.
Risk can be a larger factor in determining the type of testing involved to test in different scenarios.
The higher the possibility of losses, the more we need to invest in testing the software before it is implemented
E-commerce and online book websites are entirely different from each other, so is testing.

Principle 7: Absence of errors fallacy

Finding and fixing defects will not help if the system doesn’t meet the user requirements.
Software with no known errors is not necessarily ready to be implemented.

I hope with the above discussion about testing principles, we have now understood why testing principles are important and that they tell us more on how and why testing is important for any project.

So far, we have understood that what is software testing, why is software testing is important and how much testing is necessary? We have learned about the quality of the product, and the differences between an error, defect, and failure. We talked about the various testing principles.

Now we will move on to the next section where I’ll be focusing on various development models along with Software testing life cycle.