NatureofTesting

A software product can be described as a collection of features that meets some requirement. A bug is a part of the hardware or software that causes the product to not to behave in the required manner. A test is a procedure intended to verify or refute the existence of a bug. Testing is the evaluation a product by executing a collection of test to demonstrate it is bug-free.

Given the same conditions a good test always produces the same result; either pass or fail. A test may fail because of a bug in the product, but it can also fail due to a bug in the test itself; a false negative. Equally, a test may pass due to a bug in the test rather than because the product has behaved correctly; a false positive. Good tests avoid false negatives and should never yield false positives.

We say that a product is bug-free when we believe the likelihood of a bug being encountered is low enough to warrant its use. Testing is the accepted way of reaching such a conclusion. Unfortunately, we cannot prove that a product has no bugs unless we write tests for every conceivable bug; an impractical course of action given the limitations of our current technology. At most, we can write procedures intended to verify or refute the existence of a part of the hardware or software that causes the product to not to behave in the manner stated in its specification; that is to say we test against specifications.

Seldom do we have the time (or energy) to write a test for each part of the specification, and as the specification itself is only a partial statement of requirements, we must accept that we cannot test everything. We can only attempt to test what seems important. Our understanding of what is important improves as we gain experience in developing the product, and this will inevitably lead to more and more tests being performed.

Increased testing may improve our confidence that the product is bug-free, but it also means we will need to apply more time and resources to this activity. This is particularly true when there is a significant time delay between failing a test, fixing the bug, and then being able to re-run the test. We need to decide the types of tests that will yield the greatest value and apply them in an efficient way, if we are to avoid bringing our project to its knees.

Extracted from an article by BillStott