ISYS6338– Testing and System

Implementation

Session 5:
Static Test Techniques
 Learning Outcomes

• LO-2 : Explain the test process and types for a software

project
 References

• Dorothy Graham, Rex Black, Erik van Veenendaal. (2019).

Foundations of Software Testing: ISTQB
Certification, 4th Edition. Cengage Learning.
Hampshire, UK. ISBN: 9781473764798.
• Black, Rex. (2009). Managing the testing process :
practical tools and techniques for managing hardware
and software testing. 03. Wiley. Indianapolis. ISBN:
9780470404157.
 Sub Topics

• Statis techniques and test process

• Benefits of static testing
• Work product review process
• Roles and responsibilities in a formal review
• Applying review techniques
• Success factors for reviews
Static Test Techniques in Overview
 Overview

• Static test techniques provide a powerful way to

improve the quality and productivity of software
development.
• This chapter describes static test techniques,
including reviews, and provides an overview of how
they are conducted.
• The fundamental objective of static testing is to
improve the quality of software work products by
assisting engineers to recognize and fix their own
defects early in the software development process.
• While static testing techniques will not solve all the
problems, they are enormously effective.
• Static techniques can improve both quality and
productivity by impressive factors.
 Overview
(cont..)
• In the previous session, we saw that testing is defined
as all life cycle activities, both static and dynamic, to
do with planning, preparing and evaluating software
and related work products.
• Static testing is
– Testing a work product without code being executed.
• Dynamic testing is
– Testing that involves the execution of the software of
a component or system
• The difference between those two above: considering
the code we want to evaluate; dynamic testing
actually
executes that code; static testing (including static
analysis) does NOT execute the code we are
evaluating.
Static & Dynamic Testing
 Work Products That Can
Be Examined by Static
• Static
Testing
analysis is most often used to evaluate code
against various criteria, such as adherence to coding
standards, thresholds for complexity, spelling and
grammar correctness and reading difficulty
• However, static testing is broader than automated
evaluations; reviews can be applied to any type of work
product, including: • Any type of specification: business requirements,
1 functional requirements, security requirements.

• Epics, user stories and acceptance criteria.

2

• Code
3

• Testware
4

• User guides
5

• Models such as activity diagrams

6
 Benefits of Static Testing

• The use of static testing, such as reviews on software

work products, has two major advantages:
– Since static testing can start early in the life cycle,
early feedback on quality issues can be established
• For example an early validation of user requirements
rather than late in the life cycle during acceptance
testing. Feedback during design review or backlog
refinement is more useful than after a feature has been
built.
– By detecting defects at an early stage, rework
costs are most often relatively low, and thus
relatively cheap improvements to the quality of
software products can be achieved, as many of the
follow-on costs of late updates are avoided,
• For example additional regression tests, confirmation
tests, etc
 Static Testing
(cont..)
• In conclusion, static testing is a very suitable method
for improving the quality of software work products.
This applies primarily to the assessed work products
themselves.
• It is also important that the quality improvement
is not achieved just once, but has a more
permanent nature.
• The feedback from the static testing process to the
development process allows for process improvement,
which supports the avoidance of similar errors
being made in the future. This is
particularly useful for sprint or project retrospectives.
 Static Testing vs Dynamic
Testing
• Static and dynamic testing have the same objectives:
to assess the quality of work products and identify
defects as early as possible.
• But static and dynamic testing are not the same. They
find different types of defect, so they are
complementary and are best used together.
• It does not make sense to ask if one is better than the
other; both are useful and needed.
• With dynamic testing methods, software is executed
using a set of input values and its output is then
examined and compared to what is expected.
• During static testing, software work products are
examined manually, or with a set of tools, but not
executed.
 Static Testing vs Dynamic
Testing (cont..)
• Compared to dynamic testing, static testing finds def
ects rather than failures.
• In the early chapter we made a distinction between
errors, defects and failures.
– An error is a mistake made by a human being (for example
in writing code),
– A defect is something that is wrong (for example in the
code itself) and
– A failure is when the system or component does not
perform as it should (for example returns the wrong
balance).
• Static testing does not cause the system or
component to do anything, so it cannot find failures;
only dynamic testing can do that.
• However, static testing does find defects directly and
dynamic testing has to investigate the failure to find
Review Process
 Overview

• In this section, we will focus on reviews as a distinct –

and distinctly useful – form of static testing.
• We’ll discuss the process for carrying out reviews.
We’ll talk about who does what in a review meeting and
as part of the review process.
• We’ll cover types of reviews that you can use. We’ll
look at different variations for reviews based
on different ways to prepare for and perform
reviews, and finally we will look at success factors
to enable the most effective and
efficient reviews possible.
• One reason why reviews are so useful is that
having a different person look at a work product
is a way to overcome cognitive bias, the
tendency to see what we intended rather than
what we actually wrote.
 Review is?

• Review is
– A type of static testing during which a work product
or process is evaluated by one or more individuals to
detect issues and to provide improvements.
• Reviews vary from very informal to formal (that is,
well-structured and regulated).
• Although inspection is perhaps the most documented
and formal review technique, it is certainly not the
only one.
• The formality of a review process is related by factors
such as the maturity of the development process, any
legal or regulatory requirements or the need for an audit
trail.
 Informal Review vs
Formal Review
• Informal Review is
– A type of review without a formal (documented)
procedure
• Formal Review is
– A form of review that follows a defined process with a
formally documented output.
• In practice, the informal review is perhaps the most
common type of review. Informal reviews are applied at
various times during the early stages in the life cycle of a
work product
• Formal reviews generally have team participation,
documented results of the review and specified
procedures to follow in carrying out the review.
Work Product Review Process
 Review Process Activities

• In contrast to informal reviews, formal reviews

follow a formal process. Informal reviews may
perform at least some of the same activities to some
extent.
• The review process consists of the following main
activities:

Fixing &
Reporting
Issue
Communication
& Analysis
Individual
Review

Initiate Review

Planning
 Planning

• The Foundation Syllabus specifies the following

elements of the planning activity:
• Defining the scope of the review: the
1 purpose of the review

• Estimating effort and the timeframe for

2 the review.

• Identifying review characteristics such as

the type of review with roles, activities and
3 checklists.
• Selecting the people to participate in the
review and allocating roles to each
4 reviewer

• Defining the entry and exit criteria for

5 more formal review types

• Checking that entry criteria are met before

6 the review starts
 Planning
(cont..)
• For more formal reviews, for example inspections, the
facilitator performs an entry check and defines at this
stage formal exit criteria.
• The entry check is carried out to ensure that the
reviewers’ time is not wasted on a work product that is
not ready for review. A work product containing too
many obvious mistakes is clearly not ready to enter
a formal review process, and it could
even be very harmful to the review process.
• If the work product passes the entry check, the review
leader and author decide which part(s) of it to review.
Because the human mind can comprehend a limited
set of pages at one time, the number should not be
too high
– The team normally consists of four to six
participants, including facilitator (moderator) and
 Initiate Review

• The Foundation Syllabus specifies the following

activities for initiating a review:

• Distributing the work

products and any other
relevant material such as
1 logging forms, checklist, etc

• Explaining the scope,

objectives, process, roles and
work products to the
2 participants

• Answering any questions that

participants may have about
3 the review.
 Initiate Review
(cont..)
• The goal of this set of activities in the previous slide is
to get everybody on the same wavelength regarding
the work product under review and to commit to the
time that will be spent on checking (that is, individual
reviewing).
• The result of the entry check and defined exit criteria
are discussed in case of a more formal review. This
stage of the review process is important to increase
the motivation of reviewers and thus the effectiveness of
the review process.
• The review initiation may be done remotely, or it may
involve a meeting (in person or using
video conferencing). If a meeting is held,
the reviewers receive a short introduction to the
objectives of the review and the work products.
 Individual Review

• The Foundation Syllabus specifies the following

activities for the individual review:
• Reviewing all or part
of the work
1 documents(s)

• Noting potential
defects,
recommendations and
2 questions

• In the individual review, the participants work alone

on the work product under review using the related
work products, procedures, rules and checklists
provided. The individual participants identify defects,
recommendations, questions and comments,
according to their understanding of the work
product and the particular role they have been given
 Individual Review
(cont..)
• All issues are recorded, preferably using a logging
form. Spelling mistakes are recorded on the work
product under review, but not mentioned during the
meeting.
• A critical success factor for a thorough
preparation is the number of pages individually
reviewed per hour. This is called the checking rate.
The optimum checking rate is the result of a mix of
factors, including the type of work product, its
complexity, the number of related work products and the
experience of the reviewer.
 Issue Communication &
Analysis
• The Foundation Syllabus specifies the following activities
for issue communication and analysis:
• Communicating identified potential
defects, for example in a review meeting.
1

• Analyzing potential defects, assigning

ownership and status to them.
2

• Evaluating and documenting quality

characteristics
3
• Evaluating the review findings against
the exit criteria to make a review
4 decision
 Issue Communication &
Analysis (cont..)
• In a formal review, the things found by the individual
reviewers are communicated to the author of the work
product (or the person who will fix defects), and may also
be communicated to the other reviewers.
• We may refer to these as issues at this point because we
do not yet know if they are defects or not. The issues
may be communicated electronically, or in a review
meeting, which may consist of the following activities
(partly depending on the review type):
– Logging,
– Discussion,
– Decision making
 Logging in a Review
Meeting
• A separate person to do the logging (a scribe) is
especially useful for formal review types such as an
inspection.
– To ensure progress and efficiency, no real discussion is
allowed during logging. If an issue needs discussion, the
item is noted as a discussion item and then handled in the
discussion part of the meeting.
– A detailed discussion on whether or not an issue is a
defect is not very meaningful, as it is much more efficient
to simply log it and proceed to the next one.
• The participant who identifies the defect may propose
the severity, or the facilitator may assign a severity.
If reviewers assign severity, it is important that the
meaning of each category is understood by all
reviewers in the same way.
 Severity

• Severity classes could be:

– Critical: defects will cause downstream damage; the
scope and impact of the defect is beyond the work
product under inspection.
– Major: defects could cause a downstream effect, for
example a fault in a design can result in an error in
the implementation.
– Minor: defects are not likely to cause downstream
damage for example non-compliance with the
standards and templates
 Discussion Part of a
Review Meeting
• For a more formal review, the issues classified as
discussion items will be handled during a discussion
part of the review meeting, which occurs after the
logging has been completed. Less formal reviews
will often not have separate logging and discussion
parts and will start immediately with logging mixed with
discussion.
• Participants can take part in the discussion by
bringing forward their comments and reasoning.
• The facilitator also paces this part of the meeting and
ensures that all discussed items either have an
outcome by the end of the meeting or are noted as
an action point if the issue cannot be solved
during the meeting.
• The outcome of discussions is documented for future
reference.
 Decision-Making Part of a
Review Meeting
• At the end of the meeting, a decision on the work
product under review has to be made
by the participants, sometimes
based on formal exit criteria.
• The most important exit criterion may be the average
number of critical and/or major defects found per
page
– If the number of defects found per page exceeds a
certain level, the work product may need to be reviewed
again, after it has been reworked. If the work product
complies with the exit criteria, it will be checked later by
the review leader, facilitator or one or more participants.
– Subsequently, the work product can leave the review
process.
 Fixing and Reporting

• The Foundation Syllabus specifies the following

activities for fixing and reporting:
• Creating defect reports for those findings that
1 require changes.

• Fixing defects found (typically done by the author) in the

2 work product reviewed

• Communicating defects to the appropriate person or

3 team

• Recording updated status of defects

4

• Gathering metrics (for more formal review types), for

5 example of defects fixed, deferred, etc

• Checking that exit criteria are met

6

• Accepting the work product when the exit criteria

7 are reached

• Defect reports may have been recorded in a general

defect logging tool (for example also used by testers) or
may have been recorded in a review log
 Roles and
Responsibilities in a
• The
Formal Review
participants in any type of formal review should
have adequate knowledge of the review process.
• In the case of an inspection or technical review,
participants should have been properly trained, as both
types of review have proven to be far less successful
without trained participants. This indeed is a critical
success factor.
• The best formal reviews come from well-organized
teams, guided by trained facilitators (moderators).
• Within a review team, six types of roles can be
distinguished:
– author,
– management,
– facilitator (or moderator),
– review leader,
 Roles and
Responsibilities in a
•
Formal Review (cont..)
The author has two main responsibilities:
– Creating the work product under review.
– Fixing defects in the work product (if necessary).
• Management has five main responsibilities:
– Ensuring that reviews are planned.
– Deciding on the execution of reviews.
– Assigning staff, budget and time.
– Monitoring ongoing cost effectiveness.
– Executing control decisions in the event of inadequate
outcomes.
 Roles and
Responsibilities in a
• Facilitator
Formal
or
Review
moderator has
(cont..)
three main
responsibilities:
– Ensuring the effective running of review meetings (when
held).
– Mediating, if necessary, between the various points of view.
– Being the person upon whom the success of the review
often depends.
• Leader has two main responsibilities:
– Taking overall responsibility for the review.
– Deciding who will be involved.
• Reviewers has three main responsibilities:
– Being subject matter experts, persons working on the
project, stakeholders with an interest in the work product,
and/or individuals with specific technical or business
backgrounds.
– Identifying potential defects in the work product under
 Roles and
Responsibilities in a
•
Formal Review (cont..)
Scribe or recorder has two main responsibilities:
– Collating potential defects found during the individual
review activity.
– Recording new potential defects, open points and
decisions from the review meeting (when held).
Type of Review
 Informal Review

• An informal review is characterized by the following

attributes:
– Main purpose/objective: detecting potential
defects.
– Possible additional purposes: generating new
ideas or solutions, quickly solving minor problems.
– Not based on a formal (documented) review
process.
– May not involve a review meeting.
– May be performed by a colleague of the author
(buddy check) or by more people.
– Results may be documented (but often are not).
– Varies in usefulness depending on the reviewer(s).
– Use of checklists is optional.
– Very commonly used in Agile development.
 Walkthrough

• Walkthrough is
– A type of review in which an author leads members
of the review through a work product and the
members ask questions and make comments about
possible issue
• A walkthrough is characterized by the following
attributes:
– Main purposes: find defects, improve the software product,
consider alternative implementations, evaluate conformance
to standards and specifications.
– Possible additional purposes: exchanging ideas about
techniques or style variations, training of participants,
achieving consensus.
– Individual preparation before the review meeting is
optional.
– Review meeting is typically led by the author of the work
 Walkthrough (cont..)

– Use of a scribe is mandatory.

– Use of checklists is optional.
– May take the form of scenarios, dry runs or
simulations.
– Potential defect logs and review reports may be
produced.
– May vary in practice from quite informal to very
formal.
 Technical Review

• Technical review is
– A formal review type by a team of technically-
qualified personnel that examines the suitability of
a work product for its intended use and identifies
discrepancies from specifications and standards.
• A technical review is characterized by the following
attributes:
– Main purposes: gaining consensus, detecting potential
defects.
– Possible further purposes: evaluating quality and
building confidence in the work product, generating
new ideas, motivating and enabling authors to
improve future work products, considering alternative
implementations.
 Technical Review
(cont..)
– Reviewers should be technical peers of the author,
and technical experts in relevant disciplines.
– Individual preparation before the review meeting is
required.
– Review meeting is optional, mideally led by a trained
facilitator (typically not the author).
– Scribe is mandatory, ideally not the author.
– Use of checklists is optional.
– Potential defect logs and review reports are typically
produced.
 Inspection

• Inspection is
– A type of formal review to identify issues in a work
product, which provides measurement to improve the
review process and the software development
process.
• An inspection is characterized by the following attributes:
●Main purposes: detecting potential defects, evaluating
quality and building confidence in the work product,
preventing future similar defects through author learning
and root cause analysis. ●Possible further purposes:
motivating and enabling authors to improve future work
products and the software development process,
achieving consensus. ●A defined process is followed,
with formal documented outputs, based on rules and
checklists. ●There are clearly defined roles, such as
those specified in Section 3.2.2 which are mandatory and
 Inspection
(cont..)
• ●Reviewers are either peers of the author or experts in
other disciplines that are relevant to the work product.
●Specified entry and exit criteria are used. ●A scribe is
mandatory. ●The review meeting is led by a trained
facilitator/moderator (not the author). ●The author
cannot act as the review leader, facilitator, reader or
scribe. ●Potential defect logs and review reports are
produced. ●Metrics are collected and used to improve
the entire software development process, including the
inspection process
Applying Review Techniques
 Applying Review Techniques

• Ad hoc reviewing A review technique carried out by

independent reviewers informally, without a structured
process.
• Checklist-based reviewing A review technique guided by
a list of questions or required attributes.
• Scenario-based reviewing A review technique where the
review is guided by determining the ability of the work
product to address specific scenarios.
• Role-based reviewing A review technique where
reviewers evaluate a work product from the perspective
of different stakeholder roles.
• Perspective-based reading (Perspective-based reviewing)
A review technique whereby reviewers evaluate the work
product from different viewpoints
Success Factors for Reviews
 Organizational success
factors for reviews

• Have clear objectives

1

• Pick the right review type and technique

2

• Review materials need to be kept up to

3 date

• Limit the scope of the review

4

• It takes time
5
 People-related success
factors for reviews

• Limit the scope of the review and pick things that really
count
• Defects found should be welcomed
• Review meetings are well managed
• Trust is critical
• How you communicate is important
• Follow the rules but keep it simple
• Train participants
• Continuously improve process and tools
• Just do it!
Thank