Arunkumar Khannur's Software Testing Knowledge Center

14.3. Reporting Test Results

Test Process is being initiated in any project right from its earlier phases of software development. Testing team, after studying and analyzing different work products that include documents, code, etc, identifies relevant test cases that would ensure adequate testing. Also, testing team for a specific level of testing uses appropriate testing techniques falling into requirement-based or code-based types of testing to identify, write, review, and finalize test cases. It is always recommended to ensure that test cases are being reviewed for their appropriateness, effectiveness, and adequacy to meet testing objectives. Test case review is the most ignored activity however this activity, if done, adds a lot of value in testing.

Co-developer writes Unit Level Test Cases, designer and module/project leader reviews them, and co-developer executes the test cases and reports the errors. Detailed design, program Specifications and Code are the inputs for unit testing. Effective unit testing shall address the adequacy criteria of code based testing techniques. In addition to this static code review and technical review of the code, by using code walkthroughs and technical reviews shall be incorporated to check conformity of code with coding standards and to identify as many internal logic errors. Thus findings of Unit Testing are reported in: Review Forms; and Unit Test related error reports

Integration testing involves integrating independent software units or components to form a sizeable build and then testing the assembly. The objective of Integration test is to prove that all areas of the system interface with each other correctly and that there are no gaps in the data flow. Module leaders by involving a set of developers plan, write integration test cases, and execute integration testing. The best reviewers of Integration Test cases are designers and architects. Initial design, functional specification, COTS, and Software Units or Components that have been unit tested are the inputs for integration testing. During integration testing errors related to coupling and cohesiveness are verified. During integration test execution, parameter passing and interfacing different layers of the product become area of focus. Findings of Integration Test Execution are reported in: Integration Test related error reports

System Testing involves testing the system as a whole that include all the software components, all the hardware components and any interfaces. The whole computer based system is checked not only for validity but also for extent to which objectives are met. The system test is intended to verify program performance in accordance with the system specification and also, requirement specifications. System Test Execution involves: Functional Tests; Interface Validation Tests; Regression tests; Load / Stress Tests; Performance Test; and Security Test.

Functional Tests include Functional Operability Testing (FOT) and Functional Stress Testing (FST). FOT tests the functional requirements of the SRS and FST tests the stress requirements. FOT and FST can be combined within a single set of procedures.

Interface Validation Tests include Interface Message Tests, Interface Recovery Tests and Interface Stress Tests. These three components test all interface messages, software recovery from interface protocol errors, and software response to interface stress, respectively. All Interface Validation Tests can be run with simulators.

Regression tests are run to verify that program changes implemented after the beginning of System Testing have not introduced program regression.

These tests are designed to satisfy the load and stress requirements for critical computer programs

Performance Tests are conducted to evaluate the performance of a system or component with specified performance requirements. The Performance requirements can be obtained from Software requirements Specifications. These tests are concerned with knowledge about validating the performance requirements of a system. These tests includes knowledge about techniques to instrument performance measures like logging, event counts, event duration, and sampling. It also includes knowledge about methods for tuning a system for optimum saturation, load, and throughput threshold.

In a networked environment the security test is performed to test the safety of the application and the system. This test includes checking the unauthorized activities on the application and the system such as hacking, illegal use of the application, etc.

System Test Execution Reports can thus be reported in different forms: Functional Test related error reports; Reports on Performance, Load/Stress Test; Benchmarking Reports on System Performance; List of open bugs

Thus by executing Unit, Integration, and System Test Cases we shall be in a position to: validate the system against requirements; verify code for correctness and conformance to standards; build the system by having good interfaces among constituents. Also, we shall be in a position to understand limitations of the system, compare system performance, identify list of open bugs and in addition shall have sufficient grounds to decide on releasing the product.
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
Skype: arun.isqt