Arunkumar Khannur's Software Testing Knowledge Center
  1.4 A Framework for Usability Assessment

Assessment is an essential step in any development activity in order that the managers of the process can:

  • Predict whether usability targets will be met
  • Obtain feedback which can be used to improve design
  • Assess whether the targets have been met

Early in design the emphasis will be on obtaining feedback (typically consisting of a list of usability defects) which can be used to guide re-design, while later when a realistic prototype is available it will be possible to measure whether user and organizational objectives have been achieved.

In the early stages of the development and design process, changes are relatively inexpensive. The longer the process has progressed and the more fully the system is defined, the more expensive the introduction of changes will be. It is therefore important to start assessment as early as possible. Assessment cannot take place, however, in a vacuum.

It must take place considering who the end users are or will be and the situations the end users will find themselves in. The analysis of Context of Use is therefore an essential first step in all assessment activities. The more integrated these activities are within the overall design process, of course, the more effective they will be.

The process of user-based assessment yields information that will enable design decisions to be made by:

  • Eliciting feedback and further or revised requirements from the users;
  • Selecting the design option which best fits the functional and usability criteria.

When assessment is set against specific usability targets or objectives, it is referred to as evaluation. Particular evaluation goals will reflect some of the objectives suggested below:

  • To diagnose potential usability problems and identify needs for improvements in the interface, the supporting material, the workstation environment or the training proposals
  • To determine whether usability targets have been met.

Assessment techniques will vary in their degree of formality, rigor and user involvement depending on the environment in which the evaluation is conducted. The choice will be determined by financial and time constraints, the stage of the development lifecycle and the nature of the system under development.

The four main approaches to usability assessment are:

  • Heuristic (sometimes known as ‘rule-based’) evaluation, usually carried out by human factors specialists, and is possibly supported by task experts. It can be fast and economical and is good for identifying major problems but is not sufficient to guarantee a successful interactive system. Experts are not considered to be able to predict all the problems that end users will experience.

  • User-based evaluation can be used to provide feedback at any stage of design. In the early stages, users may be involved in the evaluation of scenarios, simple paper mock-ups or partial/rapid prototypes. As design solutions become more developed, evaluations involving users will be based on progressively more complete and concrete versions of the system.

  • Evaluation against design criteria is an established technique, which can contribute to usability. These criteria are contained in design guides, collections of ergonomic guidelines and standards.

  • Model-based assessment takes place against a theoretical model of human abilities, and the specification of how the human-computer interface is going to operate. The nature of the models used is usually quite generic and does not guarantee that the end users will react in the same way to the product. Model-based evaluation is only gradually being introduced to industry.

Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
Skype: arun.isqt