Arunkumar Khannur's Software Testing Knowledge Center
   
 

4.11. Pilot or Field Testing Considerations

The purpose of field (pilot) testing is to verify the system works in the actual user environment (or a representative set of locations). The emphasis is on verifying business processes, interfaces, connectivity, co-residency with other applications, and performance on the actual user hardware.

Pilot or field testing (as it is sometimes called) is an optional phase, but recommended whenever significant changes have been made or when a new system is deployed.

Pilot or field testing involves release of working software as often as possible in order to realize value it contains or in order to stabilize product or are warranted with an explicit, stated reason. Such a need is defined by the business, the problem domain, and the distribution channel's ability to absorb releases. The pilot or field testing commences after completion of successful completion of system and/or acceptance testing and also, after addressing all high priority errors. An updated version of the code after these activities shall be delivered to the Configuration Manager.

4.11.1. Pilot or Field Test Expectations
Following are major expectations from Pilot or Field Testing:
  • Pilot or field testing (as it is sometimes called) is an optional phase, but recommended whenever significant changes have been made or when a new system is deployed
  • Need for pilot or field test (or several pilot/field tests) are warranted with an explicit, stated reason for conducting a pilot and a specific goal
  • There are many types of Pilot or Field Testing, such as full or main product releases and earlier life cycle versions such as alphas, betas, as well as defect patches and minor upgrades, often referred to as point releases
  • The primary emphasis is verification the system works in the actual user environment under real "live" business conditions.
  • Pilot test should be performed with the final product.
  • Pilot testing is often discussed in legislation, the RFP/ITP and/or contract.
  • Pilot testing may occur before, after or as part of acceptance testing, depending on the proposal or contract
  • Usually the existing or legacy system would be operating in parallel during the pilot as a contingency in the event of a problem.
  • At first sight need for pilot or field testing seems to be small, may often demand more time and effort with no extra revenue
  • It is good to evaluate the outcomes of such pilot or field testing requirements by addressing questions like what happens if the pilot fails? what happens if it is successful? what constitutes "success" for the pilot? Etc.
  • Pilot or field testing helps in making Go/No-Go Decision on the system ready for production after addressing following questions:
    • Were there significant errors or problems found?
    • Did the business processes work as expected? Do they need to be adjusted?
    • Were there any problems with the interfaces or co-resident applications?
    • Did the system cause any problems for other interfaces or applications at the user environment (performance, data, etc.)?
    • Were there any system performance problems at this location?
    • Should the existing/legacy system be converted or shutdown?

4.11.2. Pilot or Field Testing Considerations
Pilot or field testing focuses on:
  • Developing an understanding and give attention to the clients requirements such as establishing type of user environment, training end-users,
  • Volume of workload, types of work processed
  • Location and impact to day-to-day operations, choice of a pilot location.
 
 
 
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
         
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
STEP-AUTO Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
URL: www.isqtinternational.com
Email: khannur@isqtinternational.com
Skype: arun.isqt