Unit 4 ST&M
Unit 4 ST&M
Software Engineering
Dr. MAHAVISHNU.V.C
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Syllabus
Dr. Mahavishnu.V.C
Software Testing
Dr. Mahavishnu.V.C
What Testing Shows
errors
requirements conformance
performance
an indication
of quality
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Strategic Approach
Dr. Mahavishnu.V.C
V&V
Dr. Mahavishnu.V.C
Who Tests the Software?
Dr. Mahavishnu.V.C
Testing Strategy
System engineering
Analysis modeling
Design modeling
Integration test
Validation test
System test
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Testing Strategy
For OO software
– our focus when “testing in the small” changes
from an individual module (the conventional
view) to an OO class that encompasses
attributes and operations and implies
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Strategic Issues
Dr. Mahavishnu.V.C
Unit Testing
module
to be
tested
results
software
engineer
test cases
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Unit Testing
module
to be
tested
interface
local data structures
boundary conditions
independent paths
error handling paths
test cases
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Unit Test Environment
driver
interface
local data structures
stub stub
test cases
RESULTS
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Integration Testing Strategies
Options:
• the “big bang” approach
• an incremental construction strategy
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Top Down Integration
A
top module is tested with
stubs
B F G
Dr. Mahavishnu.V.C
Bottom-Up Integration
B F G
Dr. Mahavishnu.V.C
Sandwich Testing
A
Top modules are
tested with stubs
B F G
cluster
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Regression Testing
Dr. Mahavishnu.V.C
Smoke Testing
Dr. Mahavishnu.V.C
Object-Oriented Testing
Dr. Mahavishnu.V.C
Validation Testing
Dr. Mahavishnu.V.C
Validation Test criteria
Dr. Mahavishnu.V.C
Configuration Review
Dr. Mahavishnu.V.C
Alpha and Beta Testing
Dr. Mahavishnu.V.C
Alpha testing
Dr. Mahavishnu.V.C
Beta testing
Dr. Mahavishnu.V.C
System Testing
Dr. Mahavishnu.V.C
Recovery Testing
Dr. Mahavishnu.V.C
Security Testing
Dr. Mahavishnu.V.C
Stress Testing
Dr. Mahavishnu.V.C
Performance Testing
Dr. Mahavishnu.V.C
Acceptance Testing
Dr. Mahavishnu.V.C
Acceptance Testing
Dr. Mahavishnu.V.C
Acceptance Testing
Dr. Mahavishnu.V.C
Testing
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
THE ART OF DEBUGGING
Dr. Mahavishnu.V.C
Debugging Process
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Debugging Process
Results are examined and a lack of correspondence
between expected and actual performance is
encountered ( due to cause of error).
Debugging process attempts to match symptom with
cause, thereby leading to error correction.
One of two outcomes always comes from debugging
process:
– The cause will be found and corrected,
– The cause will not be found.
The person performing debugging may suspect a cause,
design a test case to help validate that doubt, and work
toward error correction in an iterative fashion.
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Why is debugging so difficult?
Dr. Mahavishnu.V.C
6. The symptom may be intermittent (connection
irregular or broken). This is particularly common in
embedded systems that couple hardware and software
7. The symptom may be due to causes that are
distributed across a number of tasks running on
different processors
Dr. Mahavishnu.V.C
Debugging Approaches or strategies
Debugging has one overriding objective: to find and correct the
cause of a software error.
Three categories for debugging approaches
– Brute force
– Backtracking
– Cause elimination
Brute Force:
probably the most common and least efficient method for
isolating the cause of a software error.
Apply brute force debugging methods when all else fails.
Using a "let the computer find the error" philosophy,
memory dumps are taken, run-time traces are invoked,
and the program is loaded with WRITE or PRINT
statements
It more frequently leads to wasted effort and time.
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Backtracking:
common debugging approach that can be used
successfully in small programs.
Beginning at the site where a symptom has been open,
the source code is traced backward (manually) until the
site of the cause is found.
Cause elimination
Is cleared by induction or deduction and introduces the
concept of binary partitioning (i.e. valid and invalid).
A list of all possible causes is developed and tests are
conducted to eliminate each.
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Correcting the error
Dr. Mahavishnu.V.C
Testability
Dr. Mahavishnu.V.C
A good test has a high probability of
finding an error
A good test is not redundant.
A good test should be “best of breed”
A good test should be neither too simple
nor too complex
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Internal and External Views
Dr. Mahavishnu.V.C
"Bugs lurk in corners
and congregate at
boundaries ..."
Boris Beizer
Exhaustive Testing
Dr. Mahavishnu.V.C
loop < 20 X
14
There are 10 possible paths! If we execute one
test per millisecond, it would take 3,170 years to
test this program!!
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Selective Testing
Selected path
loop < 20 X
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Software Testing
white-box black-box
methods methods
Methods
Strategies
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
White-Box Testing
Dr. Mahavishnu.V.C
Why Cover?
Dr. Mahavishnu.V.C
Basis Path Testing
or
Dr. Mahavishnu.V.C
Cyclomatic Complexity
modules
V(G)
Dr. Mahavishnu.V.C
Basis Path Testing
1
Next, we derive the
independent paths:
2 Since V(G) = 4,
there are four paths
3 Path 1: 1,2,3,6,7,8
4
5 6
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive test
7 cases to exercise these
paths.
8
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Basis Path Testing
Dr. Mahavishnu.V.C
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Deriving Test Cases
Summarizing:
– Using the design or code as a
foundation, draw a corresponding flow
graph.
– Determine the cyclomatic complexity of
the resultant flow graph.
– Determine a basis set of linearly
independent paths.
– Prepare test cases that will force
execution of each path in the basis set.
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Graph Matrices
Dr. Mahavishnu.V.C
Control Structure Testing
Dr. Mahavishnu.V.C
Loop Testing
Simple
loop
Nested
Loops
Concatenated
Loops Unstructured
Loops
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Loop Testing: Simple Loops
Minimum conditions—Simple Loops
Dr. Mahavishnu.V.C
Loop Testing: Nested Loops
Nested Loops
Start at the innermost loop. Set all outer loops to their
minimum iteration parameter values.
Test the min+1, typical, max-1 and max for the
innermost loop, while holding the outer loops at their
minimum values.
Move out one loop and set it up as in step 2, holding all
other loops at typical values. Continue this step until
the outermost loop has been tested.
Concatenated Loops
If the loops are independent of one another
then treat each as a simple loop
else* treat as nested loops
endif*
for example, the final loop counter value of loop 1 is
used to initialize loop 2.
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Black-Box Testing
requirements
output
input events
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Black-Box Testing
How is functional validity tested?
How is system behavior and performance tested?
What classes of input will make good test cases?
Is the system particularly sensitive to certain input
values?
How are the boundaries of a data class isolated?
What data rates and data volume can the system
tolerate?
What effect will specific combinations of data have on
system operation?
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Graph-Based Methods
To understand
the objects that object
#1
Directed link object
#2
(link weight)
are modeled in
software and Undirected link
Node weight
(value
the Parallel links
)
relationships object
#
that connect 3
context. It contains
background color: white
encompasses data document
text color: default color
tex
objects, traditional t or preferences
components (b)
(modules), and
object-oriented
elements of
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Equivalence Partitioning
user output FK
queries formats input
mouse
picks data
prompts
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Sample Equivalence Classes
Valid data
user supplied commands
responses to system prompts
file names
computational data
physical parameters
bounding values
initiation values
output data formatting
responses to error messages
graphical data (e.g., mouse picks)
Invalid data
data outside bounds of the program
physically impossible data
proper value supplied in wrong place
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Boundary Value Analysis
user output FK
queries formats input
mouse
picks data
prompts
output
input domain domain
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Comparison Testing
Used only in situations in which the
reliability of software is absolutely critical
(e.g., human-rated systems)
– Separate software engineering teams develop
independent versions of an application using
the same specification
– Each version can be tested with the same
test data to ensure that all provide identical
output
– Then all versions are executed in parallel with
real-time comparison of results to ensure
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Orthogonal Array Testing
Y Y
X X
One input item at a time L9 orthogonal array
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Model-Based Testing
Dr. Mahavishnu.V.C
Software Testing Patterns
Dr. Mahavishnu.V.C
Program analysis
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Symbolic execution
PSG Institute of Technology and Applied Research
Dr. Mahavishnu.V.C
Model Checking
THANK
YOU