Arunkumar Khannur's Software Testing Knowledge Center
   
 

4.10. Acceptance Testing

Acceptance testing is the activity performed by the client at his site, possibly involving the representative of the development team. It is black box testing at client site with test cases written by the client based on UAT test cases. This document briefly explains the various activities performed as part of Acceptance Testing.

The scope of acceptance testing will not be clear and would result in extended acceptance testing period and warranty with no extra revenue. It is nice to address this issue related to acceptance testing by arriving at Test Success/Completion Criteria and also decide upon testing needs so as to include the business processes, help desk functions (including knowledge base and procedures), backup and recovery, disaster recovery features, system administration tools, specialized hardware, maintenance and operation procedures, year-end and quarterly reports, and other user documentation.

In some cases, it may be difficult or impractical to test a given requirement such as verifying peak time performance in banking application, system reliability during festival season sales on business to customer site. A method of verifying such requirements should be established. These un-testable requirements should be included in a test procedure/script(s) and verified during or just prior to commencing Acceptance Test. The method of verification and appropriate witnesses and supporting documentation should be documented. Typical verification methods include code inspection, simulation using test tools, or, as a last resort, a release note and its content if the client demands shall be carefully arrived at by giving greater attention to claim for any damages resulting from failure of the requirement.

The criterion for acceptance is a critical decision that must be documented. Although not all criteria may be identified during the Planning phase, the majority should be documented as part of the RFP and/or contract. Acceptance criteria typically include (but are not limited to) satisfaction of all requirements (as stated in the RFP/ contract and any associated change orders), approval of all deliverables, and satisfaction of all performance requirements. Some projects have required the system to be in production for a set period of time (to test system stability and its ability to satisfy the user's business needs) prior to conferring acceptance.

The purpose of acceptance testing is for the users to verify the system or changes meet their original needs. The emphasis is on evaluating the system via normal business circumstances, but in a controlled testing environment.

Formal testing conducted to enable a user, customer, or other authorized entity to determine whether to accept a product or product component.

Acceptance testing shall commence after:
  • Successful completion of system and regression and all critical errors have been addressed
  • An updated version of the code has been delivered to the Configuration Manager, installed under configuration control, and a full backup of the system has been performed
  • All test data has been delivered to the Configuration Manager, placed under configuration control and loaded to the system
  • The requirements and design documents are in Final format
  • The Sponsor, Users and any other participation stakeholders have been briefed on their roles and responsibilities during this test phase.
  • An overview of the testing procedures and methods should be presented to ensure all parties are aware of the expectations
  • A formal Go/No-Go Decision should be made to enter into Acceptance Testing. This may be part of the prior test phase's exit decision (from System, Performance, or Regression), or may be a standalone decision
  • All participants have been briefed on the any special expectations/criteria specified in the proposal and/or contract.
  • Acceptance criteria is documented, typically in the proposal and/or contract. Also, these criteria are discussed and any interpretations or clarifications are made before beginning of testing. It is recommended to document all these agreements on interpretation. Also, while documenting it is better to check whether such criteria are testable and objective. Also, where possible, specific test cases or procedures should be identified to ensure traceability and proof of satisfaction.

4.10.1. Acceptance Test Expectations
Major expectations from acceptance test are:
  • The primary emphasis is verification from the user's perspective where in it is verified to see whether the original requirement/change request/defect is addressed correctly or not.
  • Performance testing (including stress, load and response time) should be conducted again, if there are any concerns or if this is part of the acceptance criteria. There should be particular emphasis on response time from the user's perspective.
  • Try to allow for extra time after formal testing allowing the users to "play" with the system. This will allow them to try out other unusual scenarios and to become more familiar with the system.
  • Provide the user manual to the testers and allow them to use it (and the on-line help) during testing. This will help test the usability of the documentation as well as the completeness and the effectiveness of the project's configuration management procedures.
  • There may also be non-testable requirements which should be verified at this time. This may include difficult-to-execute code paths, or such things as the help desk (and help desk escalation), M&O procedures (such as backup and recovery or database mirroring), revised business processes, manual processes, etc.
  • At the completion of testing, review with the Sponsor and User their findings and the incidents found. Determine the criticality of the incidents and any comments. For each incident, indicate if there is a workaround procedure(s) available and what the impact to the user is.
  • If the decision is made to accept the system, then the project team, Sponsor and User should review the plans for the Implementation/Go-Live date. (If the date was not already set by the contract or project plan, negotiate the date.) Review roles and responsibilities for the implementation, and critical next steps.

4.10.2. Tasks in Acceptance Testing
Following are the tasks to be carried out to complete the System testing
  • Establish a User Acceptance Test Environment
  • Software Installation at Client’s Site
  • Training for using the software or maintaining the software
  • Perform Testing and Bug fixing
  • Support during Testing
  • Post Test Support

Establish a User Acceptance Test Environment
At client’s place, the developing organization may have to help in establishing a User Acceptance Test Environment. This step involves:
  • Acquire and commission the facilities and equipment required for user acceptance testing.
  • Notification that the user acceptance test environment is ready for use.

Software Installation at Client’s Site
The software delivered by the development team has to be installed at the client site to facilitate the client to perform the acceptance testing. This is done, based on agreement, by Support Team from Software Development Organization.

Providing training for using the software or maintaining the software
Training shall be provided to the acceptance test team / client for using and maintaining the installed software. This is done, based on agreement, by Support Team from Software Development Organization.

Perform Testing
Performing Acceptance Test includes the following tasks which are taken care by Client:
  • Prepare detailed Acceptance Test Plan: The client shall prepare a detailed acceptance test plan through collaborative approach with the development team, based on the acceptance criteria and acceptance test plan.
  • Implement Acceptance Test Plan: Implementing the Acceptance test plans include the following tasks
    • Pre-test Checklists
    • Identifying Acceptance test norms
  • Run a series of tests defined for the functionality in release candidate build.
  • Bug Reporting: Reporting the bugs include the following tasks: Identify the bugs as per the Acceptance test cases; Reporting the bugs through bug reports

Support during Testing
Support Team from Software Development Organization provides The problems reported by the client during the testing shall be recorded and addressed.
Bug fixing includes: Debugging the system; Regression testing; and Bug status report

Post Test Support
Post acceptance support could consist of the following activities as stated in the contract or mutually agreed upon:
  • Fixing the field problems reported
  • Helping in data transfer or conversion
  • Helping in implementation at multiple sites
  • Providing operation assistance
  • Handing over of the software to the client.
 
 
 
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
         
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
STEP-AUTO Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
URL: www.isqtinternational.com
Email: khannur@isqtinternational.com
Skype: arun.isqt