METRICS

• A measure is a quantitative indication of the

extent, amount, dimension, capacity, or size of
some attribute of a product or process.
• Example:
• 500 lines of code (LOC)
• 20 test cases
• 4 defects found in testing
• It is the raw data collected during software
development.
• A metric is a quantitative measure of the
degree to which a system, component, or
process possesses a given attribute.
• In other words:
A metric is derived from one or more
measures and is used to assess, compare, or
predict software quality or performance.
• Example:
• Defect Density = Total Defects / KLOC
• Code Coverage = (Tested LOC / Total LOC) ×
100%
Measurements and Metrics
A measurement is an indication of the size, quantity, amount or
dimension of a particular attribute of a product or process. For
example the number of errors in a system is a measurement.

A Metric is a measurement of the degree that any attribute belongs

to a system, product or process. For example the number of errors
per person hours would be a metric.

Thus, software measurement gives rise to software metrics.

Measurement is done by metrics. Three parameters are measured:

process measurement through process metrics,
product measurement through product metrics, and
project measurement through project metrics.
• Project Metrics
• These metrics help track and manage the project
itself—its progress, efficiency, cost, and schedule.
• Examples:
• Effort (person-hours/days)
• Cost variance (planned vs. actual)
• Schedule variance
• Team productivity
• Defect density over time
• Milestone completion rate
• Purpose:
To ensure the project is on track regarding time,
budget, and resources.
Process Metrics
• These focus on the efficiency and effectiveness of the
development process used to build the software.
• Examples:
• Defect removal efficiency (DRE)
• Process cycle time.
• Rework percentage.
• Review efficiency.
• Change request rate.
• Purpose:
To analyze and improve development and
maintenance processes for better outcomes in the
future.
• Grady [Gra92] suggests a “software metrics etiquette” that is
appropriate for both managers and practitioners as they institute a
process metrics program:
• Use common sense and organizational sensitivity when interpreting
metrics data.
• Provide regular feedback to the individuals and teams who collect
measures and metrics.
• Don’t use metrics to appraise individuals.
• Work with practitioners and teams to set clear goals and metrics that
will be used to achieve them.
• Never use metrics to threaten individuals or teams.
• Metrics data that indicate a problem area should not be considered
“negative.” These data are merely an indicator for process
improvement.
• Product Metrics
• These measure the characteristics of the software
product being developed—its quality, functionality,
and performance.
• Examples:
• Lines of Code (LOC)
• Cyclomatic complexity
• Defect density (defects per KLOC)
• Code coverage from testing
• Mean Time to Failure (MTTF)
• Response time / Throughput
• Purpose:
To evaluate the quality and reliability of the software
product.
• Various metrics formulated for products in the
development process are listed below.
• Metrics for analysis model: These address various
aspects of the analysis model such as system
functionality, system size, and so on.
• Metrics for design model: These allow software
engineers to assess the quality of design and include
architectural design metrics, component-level design
metrics, and so on.
• Metrics for source code: These assess source code
complexity, maintainability, and other characteristics.
• Metrics for testing: These help to design efficient and
effective test cases and also evaluate the effectiveness
of testing.
• Metrics for maintenance: These assess the stability of
the software product.
 Metrics for the Analysis Model
• There are only a few metrics that have been proposed
for the analysis model. However, it is possible to use
metrics for project estimation in the context of the
analysis model.
• These metrics are used to examine the analysis model
with the objective of predicting the size of the
resultant system.
• Size acts as an indicator of increased coding,
integration, and testing effort; sometimes it also acts
as an indicator of complexity involved in the software
design.
• Function point and lines of code are the commonly
used methods for size estimation.
• Function Point (FP) Metric
• The function point metric, which was proposed by
A.J Albrecht, is used to measure the functionality
delivered by the system, also estimate the effort
and cost of software system.
• Function point is derived by using a relationship
between the complexity of software and the
information domain value.
• Information domain values used in function point
include the number of external inputs, external
outputs, external inquires, internal logical files,
and the number of external interface files.
• Step 2: Calculate Unadjusted Function Points
(UFP)
• UFP = ∑ (number of items × weight for each
item)
• Example:
• 4 EIs (average) → 4 × 4 = 16
• 3 ILFs (high) → 3 × 15 = 45
→ Total UFP = 16 + ... + 45
• Step 3:Compute Value Adjustment Factor (VAF)
• There are 14 general system characteristics, such
as:
• Performance
• Security
• Distributed functions
• End-user efficiency
• Each is rated from 0 (no influence) to 5 (strong
influence). The total is used to calculate:
• VAF = 0.65 + (0.01 × ∑ ratings)
• Step 4: Final FP Calculation
• FP = UFP × VAF
• This gives the total function points, which can
be used to estimate:
• Effort (in person-hours)
• Cost
• Project complexity
Unadjusted Function Points (UFP):
UFP=4+6+12+5+7+4+15+10=63
• Calculate VAF:
• VAF=0.65+(0.01×∑Fi)= 0.65+0.35=1.00
• Step 3: Final Function Point (FP)
• FP=UFP×VAF=63×1.00=63
• Estimate effort: If 1 FP ≈ 5 person-hours → 63
FP ≈ 315 hours
• Estimate cost: If hourly rate = $50 → Cost ≈
$15,750
• Lines of Code (LOC)

• Lines of code (LOC) is one of the most widely used methods

for size estimation. LOC can be defined as the number of
delivered lines of code, excluding comments and blank lines.
• It is highly dependent on the programming language used as
code writing varies from one programming language to
another. For example, lines of code written (for a large
program) in assembly language are more than lines of code
written in C++.
• From LOC, simple size-oriented metrics can be derived such as
errors per KLOC (thousand lines of code), defects per KLOC,
cost per KLOC, and so on.
• LOC has also been used to predict program complexity,
development effort, programmer performance, and so on.
• For example, Halstead proposed a number of metrics, which
are used to calculate program length, program volume,
program difficulty, and development effort.
• Metrics for Coding

• Halstead proposed the first analytic laws for computer science by

using a set of primitive measures, which can be derived once the
design phase is complete and code is generated. These measures
are listed below.
• nl = number of distinct operators in a program
• n2 = number of distinct operands in a program
• N1 = total number(occurences) of operators
• N2= total number of operands.
• By using these measures, Halstead developed an expression for
overall program length, program volume, program difficulty,
development effort, and so on.
• Program length (N) can be calculated by using the following
equation.
• N = n1log2nl + n2 log2n2.
• Program volume (V) can be calculated by using the following
equation.
• V = N log2 (n1+n2).
• Note that program volume depends on the
programming language used and represents the
volume of information (in bits) required to specify a
program. Volume ratio (L)can be calculated by using
the following equation.
L =Volume of the most compact form of
program
• Volume of the actual program
• Where, value of L must be less than 1. Volume ratio
can also be calculated by using the following equation.
• L = (2/n1)* (n2/N2).
• Program difficulty level (D) and effort (E)can be
calculated by using the following equations.
• D = (n1/2)*(N2/n2).
• E = D * V.
int sum(int a, int b)
{
int result = a + b;
return result;
}
Step 1: Count Operators and Operands
Operators (n1 distinct, N1 total):
int
=
+
return
(), {} — can also be considered operators depending on definition
Let’s say:
Distinct operators (n1): 4 → int, =, +, return
Total operator occurrences (N1): 5
int, =, +, return, int again (for result)
• Operands (n2 distinct, N2 total):
• sum, a, b, result
• Usage counts: a, b, result used multiple times
• Let’s say:
• Distinct operands (n2): 4 → sum, a, b, result
• Total operand occurrences (N2): 6
– sum (1), a (2), b (2), result (1)
• Step 2: Apply Halstead Formulas
• Vocabulary:
• n=n1+n2=4+4=8
• Program Length:
• N=N1+N2=5+6=11
• Volume (V):
• V=N⋅log2n=11⋅log28=11⋅3=33
• Difficulty (D):
• D = (n1/2)*(N2/n2).
• (4/2)*(6/4)=3
• Effort (E):
• E=D⋅V=3⋅33=99
int sum(int a, int b)
{
int result = a + b;
return result;
}
Step 1: Count Operators and Operands
Operators (n1 distinct, N1 total):
int
=
+
return
(), {} — can also be considered operators depending on definition
Let’s say:
Distinct operators (n1): 4 → int, =, +, return
Total operator occurrences (N1): 5
int, =, +, return, int again (for result)
 Architectural Design Metrics

• These metrics focus on the features of the

program architecture with stress on
architectural structure and effectiveness of
components (or modules) within the
architecture.
• In architectural design metrics, three software
design complexity measures are defined,
namely, structural complexity, data
complexity, and system complexity.
• In hierarchical architectures (call and return
architecture), say module ‘j’, structural complexity is
calculated by the following equation.
• S(j) =f2 out(j)
• Where
• f out(j) = fan-out of module ‘j’ [Here, fan-out means
number of modules that are subordinating module j].
• Complexity in the internal interface for a module ‘j’ is
indicated with the help of data complexity, which is
calculated by the following equation.
• D(j) = V(j) / [fout(j)+1]
• Where
• V(j) = number of input and output variables passed to
and from module ‘j’.
• System complexity is the sum of structural
complexity and data complexity and is
calculated by the following equation.
• C(j) = S(j) + D(j)
• The complexity of a system increases with
increase in structural complexity, data
complexity, and system complexity, which in
turn increases the integration and testing
effort in the later stages.
• 1. Structural Complexity:
• Formula:
• S(j)=fout(j)
• Where:
• fout(j)= fan-out of module j (i.e., number of modules
that depend on or are called by module j)
• 2. Data Complexity:
• Formula:
• D(j)=V(j)/fout(j)+1
• Where:
• V(j) = number of input/output variables passed to and
from module j
• fout(j)= fan-out of module j
• Structural Complexity (S)
• Measures control flow, branching, and
connections between components/modules.
Often calculated as
• S(j) = [f_out(j)]²
• Where f_out(j) is the number of outgoing
connections from node/module j.
• Data Complexity (D)
• Measures how many variables/data elements
are processed relative to branching.
• Often calculated as:
• D(j) = V(j) / [f_out(j) + 1]
• Where V(j) is the number of input/output
variables at node/module j.
• Let’s say we have:
• f_out(j) = 3 → the module calls 3 others
• V(j) = 6 → it processes 6 pieces of data
• Then:
• Structural Complexity:
S(j) = 3² = 9
• Data Complexity:
D(j) = 6 / (3 + 1) = 1.5
• System Complexity:
SC(j) = 9 + 1.5 = 10.5
• Metrics for Software Design

• The success of a software project depends largely on

the quality and effectiveness of the software design.
Hence, it is important to develop software metrics
from which meaningful indicators can be derived.
• With the help of these indicators, necessary steps are
taken to design the software according to the user
requirements.
• Various design metrics such as architectural design
metrics, component-level design metrics,
user-interface design metrics, and metrics for
object-oriented design are used to indicate the
complexity, quality, and so on of the software design.
• Quality of software design also plays an important role in determining the
overall quality of the software. Many software quality indicators that are
based on measurable design characteristics of a computer program have
been proposed.
• One of them is Design Structural Quality Index (DSQI), which is derived
from the information obtained from data and architectural design. To
calculate DSQI, a number of steps are followed, which are listed below.
• 1. To calculate DSQI, the following values must be determined.
• Number of components in program architecture (S1)
• Number of components whose correct function is determined by the
Source of input data (S2)
• Number of components whose correct function· depends on previous
processing (S3)
• Number of database items (S4)
• Number of different database items (S5)
• Number of database segments (S6)
• Number of components having single entry and exit (S7).
• 2. Once all the values from S1 to S7 are known, some intermediate values
are calculated, which are listed below.
• Program structure (D1): If discrete methods are used for
developing architectural design then D1= 1, else D1 = 0
• Module independence (D2): D2 = 1-(S2/S1)
• Modules not dependent on prior processing (D3): D3 =
1-(S3/S1)
• Database size (D4): D4 = 1-(S5/S4)
• Database compartmentalization (D5):D5 = 1-(S6/S4)
• Module entrance/exit characteristic (D6): D6 = 1-(S7/S1)
3. Once all the intermediate values are calculated, DSQI is
calculated by the following equation.
• DSQI = ∑WiDi
• Where i = 1 to 6
• ∑Wi = 1 (Wi is the weighting of the importance of
intermediate values).
• Library Management System (LMS)
• Assume the system handles:
• Book check-in/check-out
• Member registration
• Search and catalog
• Database of books, members, and
transactions
• S1 – Number of components in program architecture
• Definition: Total number of logical components/modules
in the system.
• Example:
• Login module
• Book Search module
• Borrow Book module
• Return Book module
• Member Management module
• Report Generation module
• S1 = 6
• S2 – Number of components dependent on input
data source
• Definition: Modules that rely heavily on raw input
data (e.g., user inputs, sensor data, files).
• Example:
• Borrow Book (needs input: book ID, user ID)
• Return Book
• Login (needs credentials)
• Search (uses input keywords)
• S2 = 4
• S3 – Number of components dependent on previous
processing
• Definition: Modules that rely on the output/result of
other modules.
• Example:
• Report Generation (needs borrowing history)
• Member Management (updates records after login)
• Book Search → used before borrow
• S3 = 3
• S4 – Number of database items used
• Definition: How many fields or data elements
are accessed from the DB.
• Example:
• Book ID, Title, Author, Genre
• Member ID, Name, Expiry
• Issue Date, Due Date, Return Status
• S4 = 10
• S5 – Number of different database items
available
• Definition: Total number of unique fields/tables in
the database.
• Example:
• Books Table: 6 fields
• Members Table: 5 fields
• Transactions Table: 5 fields
• S5 = 16 (total fields)
• S6 – Number of database segments
• Definition: Logical groupings of data (tables or
DB schemas).
• Example:
• Books Segment
• Members Segment
• Transactions Segment
• S6 = 3
• S7 – Number of components with single entry/exit
• Definition: Structured modules that have only one way in
and one way out (clean logic).
• Example:
• Borrow Book (yes)
• Return Book (yes)
• Search (yes)
• Login (yes)
• Member Management (has multiple error states → maybe
not)
• Report Generation (yes)
• S7 = 5
• In conventional software, the focus of component –
level design metrics is on the internal characteristics of
the software components;
• The software engineer can judge the quality of the
component-level design by measuring module
cohesion, coupling and complexity;
• Component-level design metrics are applied after
procedural design is final. Various metrics developed
for component-level design are listed below.
• Cohesion metrics: Cohesiveness of a module can be
indicated by the definitions of the following five
concepts and measures.
• Data slice: Defined as a backward walk through a
module, which looks for values of data that affect the
state of the module as the walk starts
• Data tokens: Defined as a set of variables defined for a
module
• Glue tokens: Defined as a set of data tokens, which lies
on one or more data slice
• Superglue tokens: Defined as tokens, which are
present in every data slice in the module
• Stickiness: Defined as the stickiness of the glue token,
which depends on the number of data slices that it
binds.
• Coupling Metrics: This metric indicates the degree to
which a module is connected to other modules, global
data and the outside environment. A metric for
module coupling has been proposed, which includes
data and control flow coupling, global coupling, and
environmental coupling.
 – Measures defined for data and control flow coupling are listed
below.
• di = total number of input data parameters
• ci = total number of input control parameters
• do= total number of output data parameters
• co= total number of output control parameters
• $1§ Measures defined for global coupling are listed
below.
– gd= number of global variables utilized as data
– gc = number of global variables utilized as control
• $1§ Measures defined for environmental coupling are
listed below.
– w = number of modules called
– r = number of modules calling the modules under consideration
• By using the above mentioned measures, module-coupling
indicator (mc) is calculated by using the following equation.
• mc = K/M
• Where
• K = proportionality constant
• M = di + (a*ci) + do+ (b*co)+ gd+ (c*gc) + w + r.
• Note that K, a, b, and c are empirically derived. The values
of mc and overall module coupling are inversely
proportional to each other. In other words, as the value of
mc increases, the overall module coupling decreases.

• Complexity Metrics: Different types of software metrics can

be calculated to ascertain the complexity of program control
flow. One of the most widely used complexity metrics for
ascertaining the complexity of the program is cyclomatic
complexity.
• By using the above mentioned measures, module-coupling
indicator (mc) is calculated by using the following equation.
• mc = K/M
• Where
• K = proportionality constant
• M = di + (a*ci) + do+ (b*co)+ gd+ (c*gc) + w + r.
• Note that K, a, b, and c are empirically derived. The values
of mc and overall module coupling are inversely
proportional to each other. In other words, as the value of
mc increases, the overall module coupling decreases.

• Complexity Metrics: Different types of software metrics can

be calculated to ascertain the complexity of program control
flow. One of the most widely used complexity metrics for
ascertaining the complexity of the program is cyclomatic
complexity.
• Many metrics have been proposed for user interface
design. However, layout appropriateness metric and
cohesion metric for user interface design are the
commonly used metrics.
• Layout Appropriateness (LA) metric is an important
metric for user interface design. A typical Graphical
User Interface (GUI) uses many layout entities such as
icons, text, menus, windows, and so on.
• These layout entities help the users in completing their
tasks easily. In to complete a given task with the help
of GUI, the user moves from one layout entity to
another.
• Appropriateness of the interface can be shown by
absolute and relative positions of each layout entities,
frequency with which layout entity is used, and the
cost of changeover from one layout entity to another.
• Cohesion metric for user interface measures the
connection among the onscreen contents. Cohesion
for user interface becomes high when content
presented on the screen is from a single major data
object (defined in the analysis model). On the other
hand, if content presented on the screen is from
different data objects, then cohesion for user interface
is low.
• In addition to these metrics, the direct measure of
user interface interaction focuses on activities like
measurement of time required in completing specific
activity, time required in recovering from an error
condition, counts of specific operation, text density,
and text size. Once all these measures are collected,
they are organized to form meaningful user interface
metrics, which can help in improving the quality of the
user interface.
• In order to determine the specificity of requirements, a metric based
on the consistency of the reviewer’s understanding of each
requirement has been proposed. This metric is represented by the
following equation.
• Q1 = nui/nr
• Where
• nui = number of requirements for which reviewers have same
understanding
• Q1 = specificity.
• Ambiguity of the specification depends on the value of Q. If the value
of Q is close to 1 then the probability of having any ambiguity is less.
• Completeness of the functional requirements can be calculated by the
following equation.
• Q2 = nu / [nj*ns]
• Where
• nu = number of unique function requirements
• ni = number of inputs defined by the specification
• ns = number of specified state.
• Metrics for Software Testing
• Majority of the metrics used for testing focus on
testing process rather than the technical
characteristics of test. Generally, testers use
metrics for analysis, design, and coding to guide
them in design and execution of test cases.
• Function point can be effectively used to estimate
testing effort. Various characteristics like errors
discovered, number of test cases needed, testing
effort, and so on can be determined by estimating
the number of function points in the current
project and comparing them with any previous
project.
• Metrics used for architectural design can be used to indicate how
integration testing can be carried out. In addition, cyclomatic
complexity can be used effectively as a metric in the basis-path
testing to determine the number of test cases needed.
• Halstead measures can be used to derive metrics for testing effort.
By using program volume (V) and program level (PL),Halstead
effort (e)can be calculated by the following equations.
• e = V/ PL
• Where
• PL = 1/ [(n1/2) * (N2/n2)] … (1)
• For a particular module (z), the percentage of overall testing effort
allocated can be calculated by the following equation.
• Percentage of testing effort (z) = e(z)/∑e(i)
• Where, e(z) is calculated for module z with the help of equation
(1). Summation in the denominator is the sum of Halstead effort
(e) in all the modules of the system.
• Effort reflects the amount of work a developer would need to understand,
develop, or test a program.
• Higher Effort (E) means the module is harder to understand/test, so it may
require more time and resources.
• Lower Effort (E) suggests the module is easier to work with.

• Module A:
n1=10(unique operators)
n2=15 (unique operands)
N2=100(total operands)
V=300 (program volume)
Module B:
n1=5
n2=10
N2=80
V=250
• Metrics for Software Maintenance
• For the maintenance activities, metrics have been designed
explicitly. IEEE have proposed Software Maturity Index (SMI),
which provides indications relating to the stability of software
product. For calculating SMI, following parameters are considered.
• Number of modules in current release (MT)
• Number of modules that have been changed in the current release
(Fe)
• Number of modules that have been added in the current release
(Fa)
• Number of modules that have been deleted from the current
release (Fd)
• Once all the parameters are known, SMI can be calculated by using
the following equation.
• SMI = [MT– (Fa+ Fe + Fd)]/MT.
• Note that a product begins to stabilize as SMI reaches 1.0. SMI can
also be used as a metric for planning software maintenance
activities by developing empirical models in order to know the
effort required for maintenance.