Software quality

• software quality measures

– how well software is designed (quality of design),
– and how well the software conforms to that
design (quality of conformance)
 Software product quality
• Reliability
• Scalability
• Correctness
• Completeness
• Absence of bugs
• Fault-tolerance
– Extensibility
– Maintainability
• Documentation
 Source code quality

• Readability
• Ease of maintenance, testing, debugging,
fixing, modification and portability
• Low complexity
• Low resource consumption: memory, CPU
• Number of compilation or lint warnings
• Robust input validation and error handling,
established by software fault injection
 Methods to improve the quality:

• Refactoring
• Code Inspection or software review
• Documenting code
 Good Testing Practices
• A good test case is one that has a high
probability of detecting an undiscovered
defect, not one that shows that the program
works correctly
• It is impossible to test your own program
• A necessary part of every test case is a
description of the expected result
 Good Testing Practices (cont’d)
• Avoid nonreproducible or on-the-fly testing
• Write test cases for valid as well as invalid
input conditions.
• Thoroughly inspect the results of each test
• As the number of detected defects in a piece
of software increases, the probability of the
existence of more undetected defects also
increases
 Unit Testing
• Algorithms and logic
• Data structures (global and local)
• Interfaces
• Independent paths
• Boundary conditions
• Error handling
 Software Testing Fundamentals

• Software testing is an investigation conducted

to provide stakeholders with information
about the quality of the product or service
under test.
 Testing Method

• The box approach

– Software testing methods are traditionally divided
into white- and black-box testing.
– These two approaches are used to describe the
point of view that a test engineer takes when
designing test cases.
 White box testing
• White box testing is when the tester has
access to the internal data structures and
algorithms including the code that implement
these.
 White-Box Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...
 Types of white box testing
• API testing (application programming interface)
- testing of the application using public and
private APIs
• Code coverage - creating tests to satisfy some
criteria of code coverage (e.g., the test designer
can create tests to cause all statements in the
program to be executed at least once)
• Fault injection methods - improving the coverage
of a test by introducing faults to test code paths
• Mutation testing methods
• Static testing - White box testing includes all static
testing
 Basis Path Testing

• White-box testing technique proposed by Tom McCabe

• Enables the test case designer to derive a logical complexity measure
of a procedural design
• Uses this measure as a guide for defining a basis set of execution
paths
• Test cases derived to exercise the basis set are guaranteed to execute
every statement in the program at least one time during testing

13
 Flow Graph Notation
• A circle in a graph represents a node, which stands for a sequence of one
or more procedural statements
• A node containing a simple conditional expression is referred to as a
predicate node
– Each compound condition in a conditional expression containing one or
more Boolean operators (e.g., and, or) is represented by a separate
predicate node
– A predicate node has two edges leading out from it (True and False)
• An edge, or a link, is a an arrow representing flow of control in a specific
direction
– An edge must start and terminate at a node
– An edge does not intersect or cross over another edge
• Areas bounded by a set of edges and nodes are called regions
• When counting regions, include the area outside the graph as a region,
too

14
 Flow Graph Example
FLOW CHART FLOW GRAPH
0 0
R4
1 1

2 2

3 R3
3

6 4 6 4
R2
7 8 5
7 R1 8 5
9
9
11 10 11 10 15
 Independent Program Paths
• Defined as a path through the program from the start node until the
end node that introduces at least one new set of processing
statements or a new condition (i.e., new nodes)
• Must move along at least one edge that has not been traversed
before by a previous path
• Basis set for flow graph on previous slide
– Path 1: 0-1-11
– Path 2: 0-1-2-3-4-5-10-1-11
– Path 3: 0-1-2-3-6-8-9-10-1-11
– Path 4: 0-1-2-3-6-7-9-10-1-11
• The number of paths in the basis set is determined by the cyclomatic
complexity

16
 Mutation testing methods

• which involves modifying programs' source

code or byte code in small ways.
• In short, any tests which pass after code has
been mutated are considered defective
 Static testing
• It is generally not detailed testing, but checks
mainly for the sanity of the code, algorithm, or
document.
• It is primarily syntax checking of the code and/or
manually reviewing the code or document to find
errors.
• This type of testing can be used by the developer
who wrote the code, in isolation.
• Code reviews, inspections and walkthroughs are
also used.
 LEVELS APPLICABLE TO

• White Box Testing method is applicable to the

following levels of software testing:
– Unit Testing: For testing paths within a unit.
– Integration Testing: For testing paths between
units.
– System Testing: For testing paths between
subsystems.
• However, it is mainly applied to Unit Testing.
 Example
• A tester, usually a developer as well, studies the
implementation code of a certain field on a
webpage, determines all legal (valid and invalid)
AND illegal inputs and verifies the outputs against
the expected outcomes, which is also determined
by studying the implementation code.
• White Box Testing is like the work of a mechanic
who examines the engine to see why the car is
not moving.
 Advantages of White Box testing
• Forces test developer to reason carefully
about implementation
• Reveals errors in "hidden" code
• As the knowledge of internal coding structure
is prerequisite, it becomes very easy to find
out which type of input/data can help in
testing the application effectively.
 Disadvantages
• As knowledge of code and internal structure is
a prerequisite, a skilled tester is needed to
carry out this type of testing, which increases
the cost.
 Black box testing
• Black box testing treats the software as a "black
box"—without any knowledge of internal
implementation.
• This method attempts to find errors in the
following categories:
– Incorrect or missing functions
– Interface errors
– Errors in data structures or external database access
– Behavior or performance errors
– Initialization and termination errors
 Black-Box Testing
requirements

output

input events
 Example
• A tester, without knowledge of the internal
structures of a website, tests the web pages
by using a browser; providing inputs (clicks,
keystrokes) and verifying the outputs against
the expected outcome.
 LEVELS APPLICABLE TO

• Black Box Testing method is applicable to the

following levels of software testing:
– Integration Testing
– System Testing
– Acceptance Testing
• The higher the level, and hence the bigger and
more complex the box, the more black box
testing method comes into use.
 Types of Black Box Testing
• Equivalence partitioning:
– It is a software test design technique that involves dividing
input values into valid and invalid partitions and selecting
representative values from each partition as test data.
• Boundary Value Analysis:
– It is a software test design technique that involves
determination of boundaries for input values and selecting
values that are at the boundaries and just inside/ outside
of the boundaries as test data.
• Cause Effect Graphing:
– It is a software test design technique that involves
identifying the cases (input conditions) and effects (output
conditions), producing a Cause-Effect Graph, and
generating test cases accordingly.
 Equivalence Partitioning
• A black-box testing method that divides the input domain
of a program into classes of data from which test cases are
derived
• An ideal test case single-handedly uncovers a complete
class of errors, thereby reducing the total number of test
cases that must be developed
• Test case design is based on an evaluation of equivalence
classes for an input condition
• An equivalence class represents a set of valid or invalid
states for input conditions
• From each equivalence class, test cases are selected so
that the largest number of attributes of an equivalence
class are exercise at once
28
Guidelines for Defining Equivalence Classes
• If an input condition specifies a range, one valid and two invalid
equivalence classes are defined
– Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x >
10}
• If an input condition requires a specific value, one valid and two
invalid equivalence classes are defined
– Input value: 250 Eq classes: {250}, {x < 250}, {x >
250}
• If an input condition specifies a member of a set, one valid and one
invalid equivalence class are defined
– Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any
other x}
• If an input condition is a Boolean value, one valid and one invalid
class are define
– Input: {true condition} Eq classes: {true condition}, {false
condition} 29
 Boundary Value Analysis
• A greater number of errors occur at the boundaries of the input domain
rather than in the "center"
• Boundary value analysis is a test case design method that complements
equivalence partitioning
– It selects test cases at the edges of a class
– It derives test cases from both the input domain and output domain

30
 Advantages
• The black box tester has no "bonds" with the
code, and a tester's perception is very simple: a
code must have bugs.
• Using the principle, "Ask and you shall receive,"
black box testers find bugs where programmers
do not.
• Tests are done from a user’s point of view and
will help in exposing discrepancies in the
specifications.
• Tests can be conducted by a body independent
from the developers, allowing for an objective
perspective and the avoidance of developer-bias.
 Disadvantages
• Only a small number of possible inputs can be
tested and many program paths will be left
untested.
• Without clear specifications, which is the
situation in many projects, test cases will be
difficult to design.
 Seat Work
• What is difference between white box and
black box testing. Give an example for each.
• Explain each type of black box testing