Arunkumar Khannur's Software Testing Knowledge Center

2.4. Test Environment

The test environment contains the infrastructure for testing. When the test environment is designed, the real relationships between the objects in the environment and the users of the environment are established. The way the system is planned and arrangement of the objects in the system must reflect how the system would be going to be used. All objects in the test environment should be under the control of the configuration management system. The test environment has an owner, who is responsible for its maintenance and integrity.
Test environment ideally consists of
  • Test Approach
  • Test Framework
  • Test standards
  • Test Repository

We discuss on these test environment related elements in the following sections.

Test Approach
Test approach provides the ways and means of dealing with test related activities so as to carryout efficient and effective testing. This chapter discusses the approach of doing testing level-wise. This approach allows us to address testing at basic unit level and moving towards higher constructs involving module/sub-system and system testing. The steps described in each level of testing are similar but intention, objectives, and deliverables differ.

Deciding on test approach is very crucial step in succeeding in testing. It actually speaks about organization’s attitude towards testing. Main objectives of Test Approach are:
  • Providing ways and means of doing testing
  • Ensuring framework of covering tests at different levels
  • Ensuring better ground work so that adequate testing is being done.

Good test approach reflects following practices in place.
  • Specified requirements are used as the basis for testing. Specified requirements shall be analyzed and also, reviewed by relevant stake holders including testers.
  • System testing begins much earlier in development cycle. This allows more calendar time for testing the integrated system and enables the testers to identify operational deficiencies earlier in the project.
  • The developers get more time to resolve incomplete requirements and to respond to the changing requirements that are inevitable in our business.
  • The software is placed under configuration control near the completion of build 1. System testing begins right after the start of configuration control. Because of this it is possible to reduce the overhead of configuration control by checking-in tested code.
  • Reviews, peer reviews and inspections are carried out through out. These activities go a long way in improving the reliability and quality of the product by locating and removing defects in the earlier phases of life cycle.

Test Approach contains development documents of interest to testing and details the infrastructure of the tools and resources the test system uses (test environment). Test Approach is the way that is adapted to testing a system. Every organization should have a standard test approach as part of its standards documents. The job of the test approach is two fold:

First, the test approach must describe in detail how to build test models, test objects and test scripts. It must describe how to combine test scripts into coherent test suites that you run against the designated technical components. Second, the test approach must explicitly specify the optimization approach characteristics of all these objects: the vision, mission and operational objectives.

As part of the optimization approach for the test plan, the following tasks must be accomplished in the document.
  • Enumerate a set of acceptable and measurable testing goals that sufficiently address the quality risks.
  • Produce a checklist of measurable testing goals for the entire team, including the customers and the users, agree upon and recognize the goals of this testing effort.
  • Prepare a vision and mission statement. Propose a set of objectives, which have to be reviewed, negotiated and revised with customers, until Sign off from customer, is achieved. This iterative process ultimately results in a formal acceptance of the set of objectives.

Vision is a method of testing that can be reused in many projects to achieve an acceptable level of risk with high degree of productivity. Mission is providing an optimized testing approach that can be reused in many projects to get an acceptable level of risk and a high level of productivity in testing a software system.

The objectives (objective, approach and measurement) serve as criteria for the success of the test model coverage, risk reduction and failure rate.

The job of the test approach is to make these objectives explicit, in writing, in a standard document.

Test Framework
Test framework manages the interface among a developer, the test code and the product code. It provides a consistent interface to the tester. It also serves as the Test Driver and Test manager. The test frameworks can be designed as simple test drivers or as highly automated tools.

Test Standards
Test standards are meters applied to a broad category of test objects. The hierarchical method suggests standards for the following test objects:
  • Directory structure
  • Naming conventions
  • Test objects associated with different test approaches
  • Test plan sections
  • Test approach guidelines
  • Entry and coverage objectives for conditional models
  • Entry and coverage objectives for module models
  • Entry and coverage objectives for integration models
  • Entry and coverage objectives for system models
  • Baseline exit criteria

Test Repository
Test Repository is a storage system to organize and keep track of the test objects such as Requirement documents, Test Plans, Test Suites, Test scripts, Test execution reports etc. This is a common place where the testers and Test managers look into to get any details on the Test Design, Test Execution part and Test Evaluation. Ideally, Test Repository must include Test design documentation, low level database specification and Database creation scripts such as SQL scripts.
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
Skype: arun.isqt