The What, Why and How of Testing

This article is for developers and those who are interested in testing. It covers the need for testing, as well as the basic concepts and the various techniques used in the process. It also explores some open source tools for testing.

Here’s a quick quiz to see how many of us understand what testing is all about and why we do it—mentally note ‘Yes/True’ or ‘No/False’ regarding each of these statements: Testing is the same as quality control. We do testing to ensure that the product is 100 per cent defect-free. Testing is very easy, but a boring job. Testing does not involve any creativity. Needless to say, most would respond with a ‘Yes’ to all the above questions. However, that would be incorrect; and I will explain why, soon. Many (in fact, most) of us tend to feel testing is a low-grade and boring task. But it is not so—it is perhaps the most challenging task anybody can take up. No development activity will teach you as much as testing will, provided it's done correctly, and with the right attitude. This is my personal opinion based on my experience so far; other people may have different opinions, though.

We know that a defect is an error in a program that prevents it from producing the expected results. Is this dHfiniWiRn FRPpOHWH? :H wiOO OHDrn PRrH DERuW WhDW Ds wH proceed. For now, here is a preliminary understanding: Testing is done to show that there are no errors in the program; and that the program is performing its intended functions correctly—i.e., doing what it was designed to do.

However, while that understanding is correct, it is not complete. What about when a program does something it is nRW suppRsHd WR dR? SR, WhH pRinW is, dR nRW WHsW WR shRw that your program contains no defects. Instead, assume that your program contains defects, and then start testing to dig them out.

How do you determine whether a test was successful or nRW? YRu wRuOd prREDEOy sDy WhDW iI nR dHIHFWs DrH IRund, then the test is successful—and if some defects are found, then the test is unsuccessful. This is completely contrary to the idea behind testing!

To understand it better, let us take a simple analogy. SuppRsH D pDWiHnW is YHry siFN Dnd is suIIHring IrRP D high fever. He visits the doctor for a diagnosis. The doctor recommends some tests, but they don't detect any prREOHP. SR, wH sDy WhDW WhH WHsWs wHrH unsuFFHssIuO in detecting the problem. However, if the tests were able to detect the problem, the doctor can begin treatment of the patient—that's a successful test. The same is true for software testing. Assume that your program is like a sick patient. Now, it is up to you to design the tests to detect WhH prREOHPs. SR, RnH FDn dHIinH D suFFHssIuO WHsW Ds RnH that is able to detect defects, and an unsuccessful test as one that can’t find any.

As should be evident by now, it is all about having the right attitude to testing. As is rightly said, “You will see what you want to see.” If your intention is to show that your program is defect-free, for sure, you are unlikely to find any defects.

Test cases

:hDW is D WHsW FDsH, Dnd whDW shRuOd iW FRnWDin? ,n siPpOH words, a test case is a set of variables or conditions that are used to test a program. A typical test case is expected to contain the following information: Test case ID Description of the test case Expected input Expected output AFWuDO rHsuOW (PASS / )A,/) CRnfigurDWiRn (hDrdwDrH/sRIWwDrH) SWHps WR run WhH WHsW FDsH Author Date Recalling what we've discussed earlier, a good test case is RnH WhDW hDs D high prREDEiOiWy RI finding dHIHFWs.

SR hRw dR yRu wriWH WHsW FDsHs? ThDW dHpHnds Rn hRw large and complex the program or functionality to be tested is. If the program/functionality is too small (say, a program with 8-10 lines of code), it might make sense to write test cases manually. However, if the program is big, use scripting languages to write the test cases. For example, the open source CPython, one of the most widely used implementations of the Python language, can be used to write test scripts. Other popular scripting languages include -DYDSFripW, PHrO, PHP Dnd RuEy.

Now, once you have written the test cases, how would yRu HnsurH WhDW yRu hDYH HnRugh? HRw PDny WHsW FDsHs wRuOd yRu nHHd WR wriWH WR HnsurH WhDW WhHrH DrH nR dHIHFWs? It has been often observed (and my own experience FRrrRERrDWHs Whis finding), WhDW pHRpOH wriWH WHsWs RnOy Ds ORng Ds WhHy FDn find dHIHFWs. 0RsW RI WhHP gHW WirHd Rr bored if they keep testing and no defects are found. In such FDsHs, dHYHORpHrs DssuPH WhDW WhHir FRdH wRrNs finH, Dnd keep on adding new code without any tests. I am sure most project managers would agree with this observation.

If you aren’t able to produce any effective results in testing, you probably need to re-check your testing process and approach. Often, the testing involves adding a large number of tests, but the tests are written randomly, and the testing tends to become aimless.

While testing, it is extremely important to clearly classify the areas to be tested. You need to check not only the positive patterns but the negative patterns as well. (I will share more details on this in my next article.) It is also very important to check boundary conditions.

As I mentioned earlier, you must not only check for whether the program is working correctly, but also check whether it is doing something it ought not to—there might be some hidden behaviour that you might not be aware of, and which might only get uncovered after the product has been shipped to the customer. This reminds me of an inWHrHsWing DrWiFOH Ey S G GDnHsh in WhH LFY April 2012 issuH: “A Bug Rr D )HDWurH?”. As hH righWOy puWs iW, “... the customers don’t know that it is a bug, but think it is a feature, and start using it. Now, even when you realise that it is D Eug, yRu FDnnRW fix iW.” SR, iI yRu ship WhH prRduFW wiWh a bug, and the customer gets used to it as a feature, you will not be able to undo it, and will have to support the bug in future releases as well.

Most of the time, developers test in and around their code, keeping the design in mind. However, rather than the design, it is important to keep the customer or the end-user in mind, while testing. After all, they are the ones who will be using the product, not you!

In my next article, I will explain the various kinds of testing techniques that are available. Till then, here are some suggestions for further reading.