Arunkumar Khannur's Software Testing Knowledge Center
   
 

7.6. Dynamic Testing

Dynamic Testing involves execution of code in order to check code coverage which is commonly referred to as Control Flow Testing (Basic Path Testing)

Dynamic Testing in code based testing is adequate if “significant” parts of a module’s structure have been exercised. Various notions of “significant” lead to various coverage measures. Kinds of coverage measures (or coverage criteria) are:

Statement coverage
• Was every statement executed at least once?
Edge coverage
• Was every edge in the control flow graph executed?
Condition coverage
• As edge + every Boolean (sub) expression.
Path coverage
• Was every path in the control flow graph executed?

7.6.1. Statement Coverage
The first set of control flow elements to be exercised is statements. The statement coverage criterion requires each elementary statement in the program to be executed at least once, reflecting the idea that a fault in a statement cannot be revealed without executing the faulty statement. Here we have to focus on minimizing number of test cases by identifying few inputs that are capable of executing many statements.

By taking into consideration the specification, statements are identified and the corresponding test cases are defined. Depending on the required coverage degree, either all or only a certain number of statements are to be used for the test case definition.
The statement coverage of test suite in a program is the fraction of statements of program executed by at least one test case in test suite.

Statement Coverage, C- Statement = Number of executed statements / Total number of statements

Test suite satisfies the statement adequacy criterion if Statement Coverage= 1. The ratio of visited control flow graph nodes to total nodes may be different from the ratio of executed statements to all statements, depending on the granularity of the control flow graph representation. Nodes in a control flow graph often represent basic blocks rather than individual statements, and so some standards refer to basic block coverage, thus indicating node coverage for a particular granularity of control flow graph.

Example:
read(x); read(y);
if x > 0 then
write ("ONE");
else
write ("TWO");
end;
if y > 2 then
write ("THREE");
else
write ("FOUR");
end;

To test above mentioned, programming elements we may have following test cases:
(x=-5, y=-8)
(x=15, y=25)
(x=0, y=2)
(x=1, y=3)
(x=-1, y=1)
(x=-9999, y=-9999)
(x=9999, y=9999)

However, we can arrive at following minimal test set that fulfills adequacy criteria
(x =2 , y = 3)
(x =-5 , y =0 )

Weakness of Statement Coverage criterion is that it sometimes fail to ensure execution of each statement at least once. Though at times we feel “complete” coverage, in practice we fail to address certain important cases.

For example, consider
if x < 0 then
x := -x;
end;
z := log(x);

7.6.2. Edge Coverage (or Branch Coverage)
In edge coverage, also referred to as branch coverage, a test suite can achieve complete statement coverage without executing all the possible branches in a program. In the branch coverage, we focus to identify the test cases that would allow us executing each branch at least once in the program to be assessed.

By taking into consideration the specification, adequate number of test cases must be designed by means of an analysis so that both the THEN and the ELSE branch are executed at least once for each decision, i.e., the exit for the fulfilled condition and the exit for the unfulfilled must be utilized and each entry must be addressed at least once. For multiple decisions there exists the additional requirement to test each possible exit at least once and to address each entry at least once.

̺ Every edge of the control flow graph of a program is executed at least once
̺ Requires formalizing the concept of the control flow graph
̺ How to construct the graph from a program text
̺ Edges represent statements (of various kinds, nested)
̺ Nodes represent entry into the statement and exit

The branch coverage of test suite in program is calculated as:
Edge Coverage = Number of executed branches / Total number of branches

Test suite satisfies the edge adequacy criterion if Edge Coverage = 1. Intuitively, since traversing all edges of a graph causes all nodes to be visited, test suites that satisfy the edge adequacy criterion for a program satisfy also the statement adequacy criterion for the same program.

7.6.3. Condition Coverage
Condition coverage is combination of edge coverage and more detailed conditions. Here test adequacy criteria require that every edge of the control flow graph is executed at least once, and also, all values of conditions are exercised at least once.

Branch coverage is useful for exercising faults in the way a computation has been decomposed into cases. Condition coverage considers this decomposition in more detail, forcing exploration not only of both possible results of a Boolean expression controlling a branch, but also of different combinations of the individual conditions in a compound Boolean expression.

The simplest condition adequacy criterion, called basic (or elementary) condition coverage requires each elementary condition to be covered, i.e., each elementary condition shall have a True and a False outcome at least once during the execution of the test set. The basic conditions, sometimes also called elementary conditions, are comparisons, references to Boolean variables, and other Boolean-valued expressions whose component sub-expressions are not Boolean values.

The basic condition coverage of test suite can be calculated as:
Basic Condition Coverage, = Number of executed conditions / Total Number of conditions

Test suite satisfies the condition adequacy criterion if Basic Condition Coverage = 1. Notice that the total number of truth-values of all basic conditions is twice the number of basic conditions, since each basic condition can assume value true or false.

An obvious extension that includes both the basic condition and the branch adequacy criteria is called branch and condition adequacy criterion, with the obvious definition: A test suite satisfies the branch and condition adequacy criterion if it satisfies both the branch adequacy criterion and the condition adequacy criterion.

A more complete extension that includes both the basic condition and the branch adequacy criteria is the compound condition adequacy criterion, which requires a test for each possible combination of basic conditions. It is most natural to visualize compound condition adequacy as covering a truth table, with one column for each basic condition and one row for each combination of truth-values that might be encountered in evaluating the compound condition.

7.6.4. Path Coverage
Loops are important part of structured programs. The course of action and direction in which control of program proceeds, and number of times the execution of statement takes place before exit criteria depends on initiation and termination condition of the loop. Ensuring adequate testing of such loops is a challenging task. Decision and condition adequacy criteria force consideration of individual program decisions. Sometimes, though, a fault is revealed only through exercise of some sequence of decisions, i.e., a particular path through the program.

Every path through the control flow graph is executed at least once. Finer than all previous criteria, this is actually an ideal criterion. However, number of paths exponential in decisions. Worse, consider arbitrary loops, infinite paths.

Rule of thumb for loops
  • Zero iterations (if possible) so loop is skipped
  • One iteration (if possible) so body once
  • Two iterations (if possible) and “infer” it’s okay

Test suite satisfies the loop boundary adequacy criterion if for each loop:
  • In at least one execution, control reaches the loop and then the loop control condition evaluates to False the first time it is evaluated.
  • In at least one execution, control reaches the loop and then the body of the loop is executed exactly once before control leaves the loop.
  • In at least one execution, the body of the loop is repeated more than once.
 

 
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
         
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
STEP-AUTO Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
URL: www.isqtinternational.com
Email: khannur@isqtinternational.com
Skype: arun.isqt