Arunkumar Khannur's Software Testing Knowledge Center

9.7. Comparison Testing

In some applications the reliability is critical. There is growing need to verify performance of the software and its different variants under different hardware and software configurations. This is achieved by comparison testing. The two variants of comparison testing are as follows:
  • Use separate teams to develop independent versions of the software. Test each version with same test data to ensure all provide identical output.
  • Run all versions in parallel with a real-time comparison of results.

When outputs of versions differ, each is investigated to determine if there is a defect. Thus comparison testing is not to catch errors in the specification but to understand how reliable the software is under various test environments.

In second variant where we want to run all versions in parallel with a real-time comparison of results, we have to test under three possible combinations: Identical, Similar, and heterogeneous combinations of hardware and software platforms. Such tests will bring out possible associated problems like effect of platforms on software, effect of different resolutions and size of screen on appearance, impact of varieties of browsers in web applications, and so on. By doing these tests we can benchmark performance of software and also, one can recommend suitable hardware and software platform for the given software.

Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
Skype: arun.isqt