Arunkumar Khannur's Software Testing Knowledge Center
 

8.2. Dynamic Testing Vs. Static Testing

Dynamic Testing detects errors that can be in the program. Here, we write test cases to execute some conditions and parts of the program. Success depends on number of test cases and test coverage. However there is a possibility that all possible paths can not be executed and hence, many paths in program are uncovered and also, errors are undetected.

On the other hand, Static Testing is an analysis based and there is no need to execute the program. Here one needs to analyze the code or representation of program in some form like dependence graph or syntax tree. In static testing, there is no need or notion of writing test cases but analysis of code by review or inspection or walkthrough or audit.

Following are the different types of static testing:
  • Code Inspection
  • Code Walkthrough
  • Code Review
  • Code Audit

Following sections discuss on each of them.

8.2.1. Code Inspection
Code inspection is a visual examination of a code to detect and identify code related anomalies including errors and deviations from standards and specifications. Inspections are conducted by peers led by impartial facilitators. Inspectors are trained in Inspection techniques. Main objective of Software inspection is determination of remedial or investigative action for an anomaly. However code inspection does not attempt to discover the solution for the fault and this is not part of the inspection meeting.

Code inspections help in detecting and fixing defects at coding phase and hence cost of fixing will be less. Code inspections give management an insight into the development process – through metrics. Other benefit of inspection is that inspectors learn from the inspection process. It also allows easy transfer of ownership, should staff leave or change responsibility. In addition, code inspections build team strength at emotional level.

Code inspection team composition consists of Author, Reader, Moderator, Inspector, and Recorder. Inspection team can have only 3 to 6 participants maximum. Author shall not act as Inspection leader, reader or recorder. Management member shall not participate in the inspection. Reader is responsible for leading the inspection team through the program written interpreting sections of work line by line. Inspector(s) relate the code back to higher level work products like Design, Requirements.

To make code inspection successful, adequate preparation time must be provided to participants. The inspection time must be limited to 2-hours sessions, with a maximum of 2 sessions a day. The inspection meeting must be focused only on identifying anomalies, not on the resolution of the anomalies. The author must be dissociated from his work. The management must not participate in the inspections. Selecting the right participants for the inspection is very crucial.

During code inspection, inspection team checks whether code meets its objectives. In case of anomalies, the same are reported. After analysis, anomalies are classified and placed into any of the type: Missing; Superfluous (additional); Ambiguous; Inconsistent; Improvement desirable; Non-conformance to standards; Risk-prone (safer alternative methods are available); factually incorrect; Non-implementable (due to system or time constraints). Following classification, recommendations regarding each anomaly are arrived at. Following this, list of actions, due dates and responsible people are arrived at. Recommendations, if any, to the QA group to improve the process are provided.

Refer to Table 8.1 Checklist for Code Inspection for details on what to be covered during Code Inspection.

Table 8.1 Checklist for Code Inspection
SL. No. Checkpoints Observations
(Y/N)
1 Coding Standards
Have coding standards been followed consistently?
Is the code conforming to Naming Conventions?
Is the code conforming to Formatting Conventions?
Is the code well documented?
All required Documents Available?
Is code related to traceability matrix?
2 Code Functionality
Does the code match with solution specification?
Is the algorithm correctly implemented?
Is the code written with performance requirements- speed, memory usage, and disk space usage in mind?
Are the functions simple enough?
3 Data Reference Defects
Is there a referenced variable whose value is unset or is not initialized?
For all array references, is each subscript value within the defined bounds of the corresponding dimensions?
For all array references, does each subscript have an integer value? This may not be a defect, but it is a dangerous practice.
For all references through pointer variables, is the referenced storage currently allocated?
Does a storage area have alias names with different pointer variables? This is not a defect, but it is a dangerous practice.
Does the value of a variable have a type or attribute other than that expected? This is a common problem for storage referenced through pointers.
Are there any explicit or implicit addressing problems? Examples are
The physical storage is smaller than the storage addressed in the program, and
The address is defined as byte address but used as bit address.
Does the index of a string exceed its boundary?
Are there any off-by-one defects in indexing operations or in subscript references to arrays?
4 Data Declaration Defects
Have all variables been explicitly declared? If a variable is not declared, is it understood that the variable is a global variable?
If a variable is initialized in the declarative statement is it properly initialized?
Is each variable assigned the correct length, type, and storage class?
Are there any variables that have similar names (e.g., VOLT and VOLTS)? This is not a defect, but it is a sign of confusion.
5 Computation Defects
Are there any computations using variables having inconsistent data types (e.g., a Boolean variable in an arithmetic expression)?
Are there any mixed-mode (such as integer and floating-point) computations?
Are there any computations using variable having the same type but different length?
Is an overflow or underflow exception possible during the computation of an expression?
Is it possible for the denominator in a division operation to be zero?
Is it possible that a variable goes outside its meaningful range?
For expressions with more than one operator, is the order of computation and precedence of operators correct?
Are there any invalid uses of integer arithmetic, particularly divisions? Note that [2 x (1/2)] may not be equal to 1.
Are there any computations on non-arithmetic variable?
6 Comparison Defects
Are there any comparisons between variable having incompatible data types?
Are there any mixed-mode comparisons or comparisons between variable of different length?
Are the comparison operators correct?
Does the Boolean expression state what it is supposed to state? (Programmers often make mistakes when writing Boolean expressions involving and/or.)
Is the precedence or evaluation order of the Boolean expressions correct?
Do the operands of a Boolean expression have logical values (0 or 1)?
Are there any comparisons of equality between two floating-point numbers? Note that [10.0 x 0.1] is seldom equal to 1.0.
7 Control Flow Defects
Does every loop eventually terminate? Devise an informal proof or arguments showing that each loop will terminate.
Does the program have any GOTO statements? If yes, can you eliminate them?
Is it possible that a loop will never be executed because the entry condition is always false?
Are there any off-by-one defects (i.e., more than one or fewer by one iteration)?
For each switch statement, does it have a default branch?
8 Interface Defects
Is the number of formal parameters (in a called routine) equal to the number of actual parameters (in the calling routine)? Are their orders correct?
Do the attributes (e.g., type and length) of each formal parameter match the attributes of the corresponding actual parameters?
Does the unit system of each formal parameter match that of the corresponding actual parameters?
Does a function modify the value of a parameter, which is intended to be an input value?
Do global variables have the same definitions and attributes in all functions referencing them?
9 Input/output Defects
Have all files been opened before use?
Are end-of-file (EOF) conditions detected and handled correctly?
Are end-of-line conditions detected and handled correctly?
Do the format specifications match the information in the input/output (I/O) statements? For example, does the program expect an integer while the input is a character?
Does the program check the validity of its input?
10 Missing Code
The last and most common and serious defect in the program is missing code, i.e., when programs do not check a certain condition(s)or do not implement a certain function(s).

8.2.2. Code Walkthrough
Code Walkthrough is a static analysis technique in which a designer or programmer leads members of the development team and other interested parties through a software program. Participants ask questions on the program and make comments about possible errors, violation of coding standards, programming style etc. Code walkthroughs, find anomalies with a focus of improve software program and recommend alternative implementation if required (not done in inspections).

Code walkthrough team comprises of: Author, Walkthrough Leader, Recorder, and Team member. All members are trained on conducting code walkthrough. The process of code walkthrough is similar to code inspection.

8.2.3. Comparison between Code Inspection and Code Walkthrough
Code inspection is formal compared to code walkthrough. See Table 8.2 Comparison of Code Inspection with Code Walkthrough to understand the differences between Code Inspection with Code Walkthrough.
Table 8.2 Comparison of Code Inspection with Code Walkthrough
Code Inspection Code Walkthrough
A group of relevant persons from different departments participate in the inspection. Usually team members of the same project take participation in the walkthrough. Author himself acts the walkthrough leader.
Checklist is used to find faults No checklist used in walkthroughs
Inspection processes includes overview, preparation, inspection, and rework and follow up. Walkthrough process includes overview, little or no preparation, examination (actual walkthrough meeting), and rework &follow up.
Formalized procedure in each step No formalized procedure in the steps
Inspection takes longer time as the list of items in the checklist is tracked to completion. Shorter time is spent on walkthroughs as there is not formal checklist used to evaluate the program.

8.2.4. Area-wise Checklist for Code Inspection and Code Walkthrough
Following is an area wise checklist for code inspection and walkthrough with programming in focus.

Data Declaration
  • All variables declared?
  • Default attributes understood?
  • Arrays and strings initialized properly?
  • Correct lengths, types, and storage classes assigned?
  • Initialization consistent with storage class?
  • Any variables with similar names?

Data Reference
  • Unset variables used?
  • Subscripts within bound?
  • Non integer subscripts?
  • Dangling references?
  • Correct attributes when aliasing?
  • Record and structure attributes match?
  • Computing addresses of bit strings? Passing bit-string arguments?
  • Based storage attributes correct?
  • Structure definitions match across procedures?
  • String limits exceeded?
  • Off-by-one errors in indexing or subscripting operations?

Computation
  • Computations on non-arithmetic variables?
  • Mixed mode computations?
  • Computations on variables of different lengths?
  • Target size less than size of assigned value?
  • Intermediate result overflow or underflow?
  • Division by zero?
  • Base-2 inaccuracies?
  • Variables value outside of meaningful range?
  • Operator precedence understood?
  • Integer divisions correct?

Comparison
  • Comparisons between inconsistent variables?
  • Mixed-mode comparisons?
  • Comparison relationships correct?
  • Boolean expressions correct?
  • Comparison and Boolean expressions mixed?
  • Comparisons of base-2 fractional values?
  • Operator precedence and associativity understood?
  • Compiler evaluation of Boolean expressions understood?

Control Flow
  • Multi-way branches exceeded?
  • Will each loop terminate?
  • Will program terminate?
  • Any loop bypasses because of entry conditions?
  • Are possible loop fall through correct?
  • Off-by-one iteration errors?
  • DO/END statements match?
  • Any non-exhaustive decisions?
  • Handling of the exceptions, including the default exception handler?

Interfaces
  • Number of input parameters equal to number of arguments?
  • Parameter and argument attributes match?
  • Parameter and argument units system match?
  • Number of arguments transmitted to called modules equal to attributes of parameters?
  • Attributes of arguments transmitted to called modules equal to attributes of parameters?
  • Units system of arguments transmitted to called modules equal to units system of parameters?
  • Number, attributes, and order of arguments to built in functions correct?
  • Any references to parameters not associated with current point of entry?
  • Input only arguments altered?
  • Global variable definitions consistent across modules?
  • Constants passed as arguments?
  • Limitations of the function called, taken into account?

Input/Output
  • File attributes correct?
  • OPEN Statements correct?
  • Format specification matches i/o statement?
  • Buffer size matches record size?
  • Files opened before use?
  • End of file conditions handled?
  • I/O errors handled?
  • Any textual errors in output information

Other checks
  • Any unreferenced variables in cross reference listing?
  • Attribute list what was expected?
  • Any warning or informational messages?
  • Input checked for validity?
  • Missing functions?
  • Suitability of the algorithm and data structures
  • Adherence to language specific coding convention (like data naming, indentation, etc.)
  • Readability and maintainability
  • Efficiency
  • Provision for testing, debugging and maintenance
 
 
Khannur's Book
Arunkumar Khannur, Software Testing - Techniques and Applications, Published by Pearson Publications, 2011 (ISBN:978-81-317-5836-6; Pages:341 + xxii)
Follow Khannur
         
Khannur's Company
ISQT Process & Consulting Services Pvt. Ltd., Bangalore, INDIA
Khannur's Software Testing Forum
STEP-AUTO Forum
 Contact Khannur
ISQT Process & Consulting Services Pvt. Ltd.
#732, 1st Floor, 12th Main,
3rd Block, Rajajinagar,
Bangalore - 560010, INDIA
Phone: +91 80 23012511
URL: www.isqtinternational.com
Email: khannur@isqtinternational.com
Skype: arun.isqt