Arunkumar Khannur's Software Testing Knowledge Center

4.6. System Testing

System Testing involves testing the system as a whole. System Testing commences after completion of unit, functional, and integration testing. The emphasis is on verifying end-to-end workflows and scenarios. All the software components, all the hardware components, all external interfaces, and any interfaces should be tested. . The whole system is checked not only for validity but also for met objectives. The system test is intended to verify program performance in accordance with the System Specification and those requirement specifications referenced from the RS document.

System Testing requires System Test Environment that comprises of deployment like environment from hardware and software requirements. In addition, it may require test automation tools and analysis tools. In addition, we may need test data. As days are progressing, part of system testing is happening in the live environment as well that may include performance testing and external interface testing including interoperability testing.

4.6.1. System Test Expectations
Following are the expectations from System Testing:
  • The primary emphasis is verification of the system as a whole. Typically this means verifying all hardware and software requirements from a business workflow perspective. There should be a minimal number of errors and no critical errors in this test phase. If there are, it may indicate a lack of readiness to proceed.
  • This serves as the final verification of requirements and design (for those items that can be tested at this time; some requirements are monitored throughout the life of the contract).
  • Correct operation of external interfaces must be verified. This may require coordination with other departments, agencies or companies.
  • Some performance tests may be conducted and used to model or extrapolate behavior.
  • All affected documentation should be updated to reflect fixes and changes, including in-line code comments and unit/module/function headers, design documents, user manuals, help desk procedures or bulletins, and help files. Training materials should be finalized during this phase.
  • If there were any non-testable requirements, they should be verified at this time. This may include difficult code paths, or such things as the help desk, training materials, etc.

4.6.2. System Test Considerations
System testing focuses on:
Perform Functional Tests
Functional Tests include Functional Operability Testing (FOT) and Functional Stress Testing (FST). FOT tests the functional requirements of the SRS and FST tests the stress requirements. FOT and FST can be combined within a single set of procedures.

Perform Interface Validation Tests
Interface Validation Tests include Interface Message Tests, Interface Recovery Tests and Interface Stress Tests. These three components test all interface messages, software recovery from interface protocol errors, and software response to interface stress, respectively. All Interface Validation Tests can be run with simulators.

Perform Regression tests
Regression tests are run to verify that program changes implemented after the beginning of System Testing have not introduced program regression.

Perform Load / Stress Tests
These tests are designed to satisfy the load and stress requirements for critical computer programs. Three periods of maximum stress are distributed throughout at least a 25-hour period.

Perform Performance Test
Performance Tests are conducted to evaluate the performance of a system or component with specified performance requirements. The Performance requirements can be obtained from Software requirements Specifications. These tests are concerned with knowledge about validating the performance requirements of a system. These tests include knowledge about techniques to instrument performance measures like logging, event counts, event duration, and sampling. It also includes knowledge about methods for tuning a system for optimum saturation, load, and throughput threshold.

Perform Security Test
In a networked environment the security test is performed to test the safety of the application and the system. This test includes checking the unauthorized activities on the application and the system such as hacking, illegal use of the application, etc.

4.6.3. Tasks in System Testing
Following are the tasks to be carried out to complete the System testing
  • Arrive at detailed System Test Plan
  • Perform System Tests
  • Prepare System Testing Report

Arrive at Detailed System Test Plan
A detailed System Test Plan includes a detailed plan for the various tasks that are to be performed as mentioned in the task column of Roles and Responsibility table. This plan includes the time schedule, resource and the responsibilities for the activities to be performed. The detailed System Test Plan is made in accordance with the Requirements Specifications and Requirements for the system testing.

Perform System Tests
  • Build the system
  • Perform system testing by executing all the identified test cases or test scripts.
  • Log issues arising during the testing

Prepare System Test Report
System test report is a result of the system test activity. After testing, a detailed test report is generated to analyze the test results and take corrective actions. The Test report contains results of the testing, indicating the compliance or non-compliance of the specific requirement
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
Skype: arun.isqt