Defect Tracking

PES Testing Competency Team

© 2017 Wipro wipro.com confidential 1

Introduction
 The main role of testing lies in the detection and reporting of a system's
deviations from its requirements

 To correct a deviation, development requires detailed information about

symptoms, reproduction, and consequences of abnormal system behaviour
he Test process should place focus on accurate documentation and tracking
of deviations

© 2017 Wipro wipro.com confidential 2

Terminology - Defect
 A Defect is any variance between actual and expected results
 An error or defect in software or hardware that causes a software to
malfunction
 Testing is a process of identifying defects, where defects can be caused by
a flaw in the application software or by a flaw in the application specification
 For example, unexpected (incorrect) results can be from errors made during
construction phase, or from the algorithm incorrectly defined in the
specification

© 2017 Wipro wipro.com confidential 3

Error, Defect, Failure
 An error is an erroneous act of a person or group causing a defect in the
system

 A defect is a fault in the system that, with a certain probability, will lead to a
failure

 A failure is the visible deviation of a system's operational behaviour from

the requirement specifications

© 2017 Wipro wipro.com confidential 4

Error
 Refers to the difference between actual output of a software and the correct
output

 Also used to refer the human action that results in software containing a
fault

© 2017 Wipro wipro.com confidential 5

Fault
 Fault is a condition that causes the system to fail in performing its required
function

 Synonymous with the commonly used term “BUG”

© 2017 Wipro wipro.com confidential 6

Failure
 Failure is the inability of a system or component to perform a required
function according to its specifications

 Human error results in a fault being injected into the software and failure
happens only when the program or part of it containing the fault is executed.

 Failure happens as and when faulty parts are exercised and it could happen
anytime in the software life cycle

© 2017 Wipro wipro.com confidential 7

Incident Reports
 Incident reports are mostly based on differences between the specifications
of a particular test case's expected result and the actual behaviour of the
tested system
 Reasons can lie in abnormal system behaviour and in inaccurate test case
specification
 The correct answer becomes clear only during resolution Many projects use
common management to deal with incident reports written against the tested
system and incident reports written against the testware. ie., test plan, test
specifications and test automation

© 2017 Wipro wipro.com confidential 8

Documenting Incidents
 Communication of Incident reports can be done by properly documenting
and communicated via email, spread sheet, or a specialized deviation
management system
 The description of the incident report must allow simple and accurate
reproduction of the failure
 The test run and required inputs are to be succinctly described so the test
run can be reproduced easily
 The incident report refers directly to a test case specification and/or an
automated test procedure based on which development can reproduce the
failure

Cont…
© 2017 Wipro wipro.com confidential 9
Documenting Incidents
 The report shall address only one single, clearly defined problem and not a
combination of several anomalies

 The environment in which the test was executed must be clearly

documented – Ex: the version of the test object, platform, reference data
etc.,

 It must be possible to prioritize a number of anomalies relative to each other

to resolve them according to their assigned priority

Cont…
© 2017 Wipro wipro.com confidential 10
Documenting Incidents
 Many people are involved in resolution of an incident, effective control must
be supported. Ex: assigning unique status to the report, people responsible
for debugging, defect resolution and retesting

 Date and time of the incident was detected and documented

© 2017 Wipro wipro.com confidential 11

Sample Reporting Template (1 of 3)
Identification
Number Incremental, unique number of report
Identifier of the test object (system or impacted
Test object
component or components
Exact version identifier (tested build) of the test
Version
object
Identification of the hardware/software platform or
Platform
test environment where the problem occurred
Unique identification of the person who detected
Reporting person ID
the incident

Responsible corrector Unique identification of the person responsible to

ID (developer) correcting the defect

Date of occurrence Date and time the incident was first observed
© 2017 Wipro wipro.com confidential 12
Sample Reporting Template (2 of 3)
Classification
1

Current processing status of the report , possibly

2 historical
Status information allowing tracking of status changes with
corresponding change dates
Severity (e.g., crash, malfunction, deficiency, nice to have,
Class
change request)

Priority Urgency of defect removal

Requirement( Reference(s) to the requirement(s) violated or not fulfilled by

s) the incident
Phase of assumed defect injection: coding, documentation,
Source
design, etc.

© 2017 Wipro wipro.com confidential 13

Sample Reporting Template (3 of 3)

Problem description

Test case reference (identification) or direct

Test case description of the steps necessary for the
reproduction of the failure
Accurate description of the incident stating the
Problem/symptom
differences between target and actual behavior
Comments by involved staff on the report; possibly
Comments
historical information

Correction Details concerning correction measures

References to other, related incident reports; if

References required, additional references to background
information, etc.

© 2017 Wipro wipro.com confidential 14

Incident Handling
 Incident Life Cycle - Anomaly Detected -> Documented -> Analyzed ->
Corrected
 The Incident Report may also get discarded or rejected if it is
unsubstantiated
 Different people with different roles will participate in the correction process
 For effective coordination among the roles and to ensure orderly
communication, its mandatory to define a deviation management process

© 2017 Wipro wipro.com confidential 15

Roles in Deviation Management
 The Tester reports incidents, documenting them with input data for analysis.
He performs retesting after defect correction

 The Test Manager leads the test team and organizes the deviation
management, tracking the incidence occurrence and the correction
measures and use this as a base for planning and control of his test
activities. The developer analyses the incident reports and resolves defects.

Cont…
© 2017 Wipro wipro.com confidential 16
Roles in Deviation Management
 The development manager is in charge of development project and leads
the development team

 The development manager is responsible for the success of the defect

resolution process.

 The development manager is responsible for estimating and accounting the

effort required to perform the resolution activities

Cont…
© 2017 Wipro wipro.com confidential 17
Roles in Deviation Management
 The product manager is responsible for product developed in the project. He
approves change requests and is responsible for the prioritization of the correction
activities

 The customer may be closely involved in the development process , evaluating

new incidents. He may even report incidents himself in many cases

 The Change Control Board (CCB) regularly tracks the progress of deviation
detection and resolution and decides on prioritization of defect resolution and
change requests

© 2017 Wipro wipro.com confidential 18

Deviation Management Process

Report checked for New report received

completeness (open) New (new) person
done by: test reporting: tester
manager

Problem not Reporting rejected

reproducible Observatio Open Rejection (rejection) rejected
(observation) n by: test manager
reported by:
developer
Report is analyzed
(analysis) done by: Analysi
developer s

Defect correction Correction process

released for retest
Correctio (correction) done by:
(test) done by:
n project manager
developer

Defect correction Correction

failed (failed) Failed Test Closed successfully
reported by: tested completed (closed)
done by: tester

© 2017 Wipro wipro.com confidential 19

Sample Incident Report (1 of 3)
Identification

Number VSR-Systest 03347

Test object VSR Component Dream Car

Version Build 02.013 – 04.27.2006

Platform Windows XP – Standard-Client-Configuration

Reporting person G. Myers

Responsible
K. Beck
corrector (Developer)
Date of occurrence/
04.30.2006 10:17
reporting

© 2017 Wipro wipro.com confidential 20

Sample Incident Report (2 of 3)
Classification

Status Date Set by Comment

closed 05.14.2006 G. Myers successful re-test
test 05.12.2006 K. Beck Recorrected check
failure 05.06.2006 G. Myers Discount = 100% still possible
Status test 05.04.2006 K. Beck Checking function corrected
correction 05.03.2006 K. Beck Correction in callback function
analysis 05.02.2006 K. Beck source code review
analysis 04.30.2006 CCB class 3->2
New 04.03.2006 G. Myers

Class 2 (critical malfunction)

Priority 2 (correction prior to release)

Requirements CC-Sys-CalcPrice-01037 rebates

Source Validation procedures for rebates entry

© 2017 Wipro wipro.com confidential 21

Sample Incident Report (3 of 3)

Problem description

Test case TC-Price Calculation/0313 rebated purchase price

Problem / Irrespective of the selected model, special edition, and

accessories, it is possible to enter a discount of > 100%.
symptom As a result, a negative purchase price is displayed.

CCB, 04.30.2006: high classification as wrong calculation

Comments will leave an extremely bad impression with the customer

© 2017 Wipro wipro.com confidential 22

Kinds of Incidents
 Incidents from Specifications
 Product build varies from the product specified

 Incidents in capturing User Requirements

 Variance is something that user wanted is not in the build
product that was not specified in the product

© 2017 Wipro wipro.com confidential 23

Incident Categories
 Wrong : Incorrect Implementation

 Missing : User Requirements are not built into the product

 Extra : Unwanted Requirements built into the product

© 2017 Wipro wipro.com confidential 24

Incident Tracking Systems
 A Incident Tracking System helps manage software development projects
by tracking software bugs, action items, and change requests with problem
reports.

© 2017 Wipro wipro.com confidential 25

Features of Incident Tracking Systems
 These software testing tools helps manage developing projects more
efficiently by tracking bugs.

 This will bring you better response to the user because it identifies how
many bugs are in process, and how many bugs are closed.

 Besides it records bug report in database and it allows simultaneous

access by different users.

 And it offers bug classification

Cont…
© 2017 Wipro wipro.com confidential 26
Features of Incident Tracking Systems
 Defect Tracking system allows to keep track of defect information during
software testing.

 It provides general information such as

 Origin of defect
 Status
 Symptoms
 Repair priority
 Severity etc.,

Cont…
© 2017 Wipro wipro.com confidential 27
Incident Information
 General- Allows to specify the Basic information of the Defect, priority,
Severity, number of times the problem occurred and symptoms.

 Priority - Specifies Repair priority of defect.

 The default priorities settings are :

 1 - Resolve immediately
 2 - Give High Attention
 3 - Normal Queue
 4 - Low Priority

Cont…
© 2017 Wipro wipro.com confidential 28
Incident Information
 Severity can be customized using the admin option.
 Occurrences - Specifies how many times the defect has occurred during testing.
 Symptoms - Allows to specify the symptoms for the defect.

 We need two parameters to identify the nature of the defect. Ie.,, Severity
and Priority
 Severity tells us how bad the defect is
 Priority tells us how soon it is desired to fix the problem

Cont…
© 2017 Wipro wipro.com confidential 29
Incident Information
 Some of default symptom settings are:
1. Cosmetic Flaw
2. Data Corruption
3. Data loss
4. Documentation Issue
5. Incorrect Operation
6. Installation Problem
7. Missing Feature
8. Slow Performance
9. System Crash
10. Unexpected Behaviour etc.,

Cont…
© 2017 Wipro wipro.com confidential 30
General flow of Overall testing and defect
tracking sequence

Automated Yes Generate Defect

test case Test Log Test from Using
Test Procedure Viewer Failure? Test Log Viewer
No

Exit
Viewer Review status
in the
Defect Form

Manually Enter
Manual Testing Defect into the
Defect management tool

© 2017 Wipro wipro.com confidential 31

Incident Metrics
 Defect data provides wealth of information for analysis.
Some of the frequently used metrics by the Test Manager are
 Number of defects
 Defect density (Defect / size – KLOC,FP,COSMIC FFP)
 Defects per test level
 Defects per unit / module
 Defects per cause
 Defect per status
 Defect per priority

© 2017 Wipro wipro.com confidential 32

Sample Template – Incident Report

Defect Defect Tested Tested Fixed Fixed

S.No Severity Priority Status Comments
ID Description By Date By Date

© 2017 Wipro wipro.com confidential 33

Standardized Classification for software anomalies –
IEEE 1044/1044.1 Standard
 The [IEEE 1044] standard describes the classification of anomalies, the
documentation of the attributes of an anomaly, and the associated process.

 Overview of the classification Process

 It defines a sequence of 4 steps:

 Recognition
 Investigation
 Action
 Disposition

© 2017 Wipro wipro.com confidential 34

Overview of the Classification Process
 Each of the four steps consists of the execution of parallel activities:
 Recording
 Classifying
 Identifying impacts
 Applied on each step, the 3 activities classify and document the features of
the incident from different perspectives
 The focus of this work flow is on report classification

Cont…
© 2017 Wipro wipro.com confidential 35
Overview of the Classification Process
 The standard does not describe a complete management process and does
not give details on how to deal with rejected reports or ineffective
corrections

 It is important to classify in each step; i.e., to classify not only the severity
and impact of an anomaly defined during analysis but also the
circumstances that have led to its detection, the necessary resolution
actions and the report's disposition after closure

© 2017 Wipro wipro.com confidential 36

Classification according to IEEE 1044

(C) IDENTIFY 1.RECOGNITION (D) CLASSIFY

IMPACT CLASSIFY
RECOGNITION
A RECORD RECOGNITION

2. INVESTIGATION
CLASSIFY
INVESTIGATION
A RECORD RESULTS
INVESTIGATION RESULTS

3. ACTION
CLASSIFY ACTON
A RECORD ACTION

4. DISPOSITION
CLASSIFY
RECOGNITION
A RECORD DISPOSITION

© 2017 Wipro wipro.com confidential 37

Classification Steps - Recognition
 Sample Recognition Classification Scheme – Project Activity
RR=recognition step; IV=investigation step; Three-digit number for further
AC=action step; DP=disposition step; refinement of the classification
IM=impact

Category Compliance required Code Classification

Mandatory RR110 Analysis
Mandatory RR120 Review
Mandatory RR130 Audit
Mandatory RR140 Inspection

Project
Mandatory RR150 Code/Compile/Assemble
activity RR100
Mandatory RR160 Testing
Mandatory RR170 Validation/qualification Testing
Mandatory RR180 Support/Operational
Mandatory RR190 Walk-through

Mandatory element to meet Description of the activity

the standard’s requirements
Cont…
© 2017 Wipro wipro.com confidential 38
Classification Steps - Recognition
 Each Project member can report anomalies. The following activities of the
recognition step are performed as
soon as the anomaly is encountered:
 Documentation by collecting recognition supporting data items. These items
comprise of data related to the environment in which the anomaly was detected
 Example: Data such as h/w and s/w environment and test supporting s/w, contact
ID of the reporting person, time of occurrence etc.,

Cont…
© 2017 Wipro wipro.com confidential 39
Classification Steps - Recognition
 Classification via selection of values out of the recognition classification
scheme. It consists of project phase, symptom, system status etc., that help
providing accurate reporting

 Identifying the Impact by means of the impact classification scheme and the
impact supporting data items

© 2017 Wipro wipro.com confidential 40

Classification Steps - Investigation
 Example : Investigation supporting data item

© 2017 Wipro wipro.com confidential 41

Classification Steps - Analysis
 Each reported Anomaly must be investigated. The aim of investigation is to
be able to evaluate the anomaly. This is carried out mainly by the
developers, but testers or other project members too provide useful
information

 Documentation: The investigation supporting data items are collected to

confirm the existence and reproducibility of the anomaly and to identify
possible workarounds and correction measures

Cont…
© 2017 Wipro wipro.com confidential 42
Classification Steps - Analysis
 Classification: Selection of suitable data out of the investigation
classification scheme to describe the actual cause, the impacted documents
and the nature of the anomaly

 Identification of the Impact: The assumptions made on the impact of the

anomaly at the first step are checked and corrected

© 2017 Wipro wipro.com confidential 43

Classification Steps in Detail - Resolution
 Based on the analysis results, resolution actions are planned. Development
is primarily involved, also the product manager and quality assurance
department may also get involved.
 Documentation: Action supporting data items are collected; Ex: planned
resolution date and product delivery status, description of the resolution
activities, and the names or functions of people responsible for correction or
retesting.
 Classification: This is done using action classification scheme, describing
the type of action (eg., code or document changes) and their priority
(immediate -> no resolution)

Cont…
© 2017 Wipro wipro.com confidential 44
Classification Steps in Detail - Resolution
 Identifying the Impact: Impact categories documented in the previous steps
are reviewed and if necessary updated

© 2017 Wipro wipro.com confidential 45

Classification Steps in Detail - Disposition
 After all resolution activities have been completed the removal of the
anomaly is to be documented

 The main actors involved in this step are testers, support staff and product
manager.

 Documentation: The disposition supporting data items, for instance,

document that the customer has been
informed about the resolution. They are used to document verification
results

Cont…
© 2017 Wipro wipro.com confidential 46
Classification Steps - Disposition
 Classification: The disposition classification scheme documents the final
version of the anomaly report (e.g., “resolution completed” or “duplicate
problem”

 Identifying the Impact: A concluding consideration is made and previously

documented impacts are reviewed and updated

© 2017 Wipro wipro.com confidential 47

Thank You
PES Testing Competency Team

© 2017 Wipro wipro.com confidential 48