20 54-1-pb

International Journal of Advanced Research in Computer Engineering & Technology
Volume 1, Issue 1, March 2012

An Approach for Calculating the Effort Needed
on Testing Projects
Priya Chaudhary and C.S. Yadav

comprehensive one.
Abstract— Software testing effort estimation has always Use case based methods [9, 10, 12, 13] are also derived
been an on-going challenge to software engineers, as testing from software requirements and considers actor ranking,
is one of the critical activities of SDLC. Accurate effort use case points, normal and exceptional scenarios and
estimation is the state of art of software engineering, as it is
the preliminary phase between the client and the business various technical and environmental factors in order to
enterprise. The credibility of the client to the business compute test effort based on a constant conversion factor.
enterprise increases with the accurate estimation. The The limitation with these methods is that, it depends on
earlier test estimation is done, the more benefits will be the prudence of analyst.
achieved in the testing life cycle. However on the other hand, code based test effort
This paper proposes an approach for estimating the size estimation methods [4, 7, 8, 11] first considers the code,
and efforts required in the testing projects using test case
generates the test cases, execution points, productivity
point. The proposed model outlines all major factors that
affect testing projects. Covering all major factors helps to do factors and later computes the test effort. These
a fair estimation using the proposed approach. methodologies are computation intensive and amount of
This proposal is technology independent and supports the rework also gets increased.
need for estimating, project efforts management and Cognitive complexity based test effort estimation
forecasting quality deliveries. This approach would be used measures [6, 17] computes the cognitive complexity of
by the individuals who would want to have good testing the software and co-relates the cognitive complexity with
effort estimation for any given project under test.
test effort estimation. The measure uses line of the code
The computation of proposed test effort estimation
involves least overhead as compared to others. and their subsequent basic control structure. The measure
uses a tedious calculation to finally arrive at test effort.
Index Terms—Test Case Point, Test Cycles, Test However, the proposed measure considers
Effort Estimation, Testing Efforts Calculation. requirements written in standard IEEE 830:1998 [1]
format in order to obtain effective and precise result.
Hence the strength of the measure lies in computation of
I. INTRODUCTION
requirement based test cases from requirement
Software testing is an indispensable aspect, directly engineering document prior to estimation of software test
influencing the quality of the software. In order to carry effort. This will lead to early defect detection which has
out a systematic testing, it is absolutely necessary to been shown to be much less expensive than finding
predict the effort required to test the software. Hence this defects during integration testing or later.
paper proposes an approach for calculating the efforts
needed on testing projects using test point analysis. A II. RELATED WORK
number of the methodologies that have been proposed in
During the last few decades, various models, methods
the past are code based, but when we have the code, it is
and techniques have been developed for the estimation of
too late. Therefore, the effort requirement for the software
software test effort. This section presents a survey of
can be minimized, if the computation of test effort can be
some leading papers describing the work carried out so
done in early phases of software development lifecycle
far, for the estimation of software testing effort. The work
(SDLC).
presented in this paper is based on SRS. The Software
In this direction, we have captured the test case points
Engineering Standard Committee of IEEE computer
from test case preparation, execution of the SRS
society [1] presents the guidelines for the documentation
Document of the project being worked upon. [16].
of software requirements using IEEE 830: 1998 format.
As SRS acts as a verifiable document, hence,
Antonio Bertilino [2] discusses software testing roadmap
estimation of test effort based on SRS will be early
for the achievements, challenges and dreams for software
warning, systematic, less complex, faster and
testing. Symon [3] discusses the computation of function
Priya Chaudhary, Department of Computer Science and point for the estimation of size and cost of the software.
Engineering, U. P. Technical University / N.I.E.T., (e-mail: The paper also considers technical and environmental
er.priya06@gmail.com). ,Gr. Noida , India factors. Boehm [5] discusses constructive cost model
C.S. Yadav, Department of Computer Science and Engineering, U. P.
Technical University / N.I.E.T., (csyadav@yahoo.com)Gr. Noida , India (COCOMO) and its various categories for the estimation
of software cost using cost driver attributes. Eduardo
35
All Rights Reserved © 2012 IJARCET


Aranha [4] uses a controlled natural language (CNL) tool III. PROPOSED APPROACH
to convert test specification into natural language and Lot many traditional techniques exist for estimating
estimate test effort. It also computes test size and test efforts on a project for either development or testing. All
execution complexity measure. Zhou, Xiaochun [9] techniques take into account user experiences, past history
presents an experience based approach for the test suit and data to generate estimates, there by estimates differ
size estimation. Aranha and Borba [8] discusses about test because user experiences and past historydiffer from
execution and a test automation effort estimation model individual to individual across the globe. Another
for test selection. The model is based on the test requirement lies where industries have really good
specifications written in a controlled natural language. It estimation techniques for development but no concrete
uses manual coverage and automated test generation model exists for testing, though techniques exist.
technique. Nageshwaran [13] presents a use-case based In development there is a fairly good model which
approach for test effort estimation and considers weight comprises of all the attributes that are sufficient to
and environmental factors with a constant conversion compute function point analysis (FPA) [3] for any
factor for the computation of test effort. Zhu Xiaochunet. software. The function point measure includes five
al [10] presents an empirical study on early test execution parameters i.e. external input, external output, interfaces,
effort estimation based on test case number prediction and file and enquiry. These parameters act as the bases to
test execution complexity. Erika Almeida et. al. [12] measure the size of the software for the estimation of
discusses a method for test effort, based on use cases. It function point. But, black box testing does not contain
uses parameters like: actor ranking, technical and knowledge on software intrinsic qualities like functions,
environment factor related to testing like test tools, input, code, modules etc.
environment, distributed system, interfaces etc. for the Proposed model suggest that:
calculation of test effort. 1. Requirements can be used for estimation on which
Kushwaha and Misra [17] discusses CICM and functions or modules are created in development.
modeling of test effort based on component of the shelf 2. External inputs in function points can be replaced by
(COTS) & component based software engineering test case points (generated on the basis of requirements
(CBSE). and the test cases intended to build around requirements.)
Further the CICM is compared with cyclometric 3. Development attributes like External inputs, external
number. The measure is able to demonstrate that the outputs, external queries etc. Can be replaced by testing
cyclometric number and the test effort increase with attributes such as number of interfaces being touch by the
increase in software complexity. particular test case covering any requirements, number of
Sharma &Kushwaha [15, 16] discusses the improved validations being made.
requirement based complexity (IRBC ) based on SRS of TCP analysis is an approach for doing an accurate
the proposed software and also presented an object based estimation of functional testing projects. This approach
semi-automated model for the tagging and categorization emphasizes on key testing factors that determine the
of software requirements. Contribution of the researchers complexity of the entire testing cycle and gives us a way
towards this issue and a roadmap showing the various of translating test creation efforts to test execution efforts,
approaches used for the test effort estimation is which is very useful for regression testing estimation.
summarised in table 1. TCP analysis generates test efforts for separate testing
activities. This is essential because testing projects fall
Table 1.Commonly Used Measures. under four different models: Test Case Generation,
Automated Script Generation, Manual Test Execution,
S. Researchers Technique and Automated Test Execution. Though the proposed
No. approach focuses on Test Case Generation and Manual
1 Antonio Bertilino Software testing Roadmap Test Execution.
2 Symon Function Point
3 Boehm COCOMO TCP Analysis uses a 6-step process consisting of the
4 Eduardo Aranha Controlled Natural following stages:
Language(CNL) 1. Identify Requirements
5 Zhou, Xiaochun Experienced based Approach 2. Calculate TCP for Test Case Generation
6 Aranha and Borba Test execution & test 3. Calculate TCP for Test Case Execution
automation effort estimation 4. Calculate other related tasks (One time task &
model miscellaneous task)
7 Nageshwaran Use Case based Approach 5. Calculate Productivity Task
8 Erika Almeida Method for test effort based 6. Calculate Total TCP
on Use Case
9 Kushwaha and Components of The Given below is an overview of different phases
Misra Shelf(COTS) & Component To determine the TCP for Test Case Creation, first
Based Software determine the complexity of the Test Cases. Some test
Engineering(CBSE) cases may be more complex due to the inherent
36


functionality being tested. The complexity will be an The TCP estimates above might be affected by certain
indicator of the number of TCPs for the test case. application level parameters. This might increase the
effort required. To factor the impact of such parameters,
we will increase the TCP by an Adjustment Factor. Some
of the Application level parameters that might have an
IV. influence on the TCP are listed in the table below:
A. Calculate TCP for test case generations

Table 2. Calculate test case generation complexity based
on the factors given below. Table 5. The Application level parameters that might have
an influence on the TCP
S.No Test Case Generation Complexity Weights
Factors Sl. No. Factors Adjusted Adjustment Weight
1 The number of steps in the Test case 1 Not present or No affect 0
Assuming that test cases are atomic
and that they test only one condition, 2 Incidental influence 0.05
the number of steps will be an indicator
of the complexity. 3 Moderate influence 0.1
2 Interface with other Test Cases
This usually involves calling other test 4 Average influence 0.25
cases from this use case.
5 Significant influence 0.5
Ex. Address Book lookup from
Compose Mail
The weights in the table are only indicative and will be
Weight = (((Validation x 1+test steps x 2+ Interface x subject to the application for which estimation is being
3)/5) x 3) carried out.
Table 3.Calculating complexity weights. Table 6. Calculating adjustment b weights based on the
test preparation factors.
Test Complexity Factors
Complexity
S.No. Case Test Adjusted
Validation Interface Weight Test Preparation Factors
Category steps Weight
1 Simple 2 3 0 5 Domain Complexity 0
Integration with other devices
2 Medium 3 5 0 8 Multi-lingual Support
3 Complex 4 8 2 16 Re-usable scripts generated for execution
Documentation quality
Calculate Rough Test Case Points for Test Case Detailed report with screenshots
Generation by using the below formulae: Total Adjusted Weight 0
Rough TCP-G = ( Simple Test Cases X its complexity Adjustment Factor = 1 + Total adjustment weight
weight ) + (Medium Test Cases X its complexity weight )
+ ( Complex Test Cases X its complexity weight ) Each of these factors is scored based on their influence on
the system being counted. The resulting score will
Table 4.Calculating the rough TCP for test generation. increase the Unadjusted Test Case Point count. This
calculation provides us with the Test Case Point
S.No. Feature/ Number of test cases Test Generation count.
Screen Case
Simple Medium Complex Point TCP-G =Rough TCP -generation X Adjustment Factor
5 8 16 B. Calculate TCP for Test Case Execution
1 Screen1 2 0 0 10
2 Screen2 0 6 0 48
3 Screen3 2 0 0 10
4 Screen4 3 0 0 15
5 Screen5 1 0 0 5

37


Table 7. Calculate test case execution complexity based Rough TCP-E = (Simple Test Cases X its complexity
on the factors given below. weight) + (Medium Test Cases X its complexity weight)
+ (Complex Test Cases X its complexity weight)
S.No Manual Execution Complexity Weights
Factors Test Cases need to be manually executed in all the
1 Pre-conditions scenarios. To arrive at the additional Test Case Points, the
This usually involves setting up the TCP -E (single scenario) will be added for every scenario
test data. It also includes the steps for which manual execution is required.
needed before starting the execution. The TCP estimates above might be affected by certain
E.g. to test whether the printing application level parameters. This might increase the
facility is working fine, the pre- effort required. To factor the impact of such parameters,
conditions would include opening the we will increase the TCP by an Adjustment Factor. Some
application, inserting a record into the of the Application level parameters that might have an
database, generating a report and then influence on the TCP are listed in the tables above.
checking the printing facility. The weights in the table are only indicative and will be
2 Steps in the Test Case + subject to the application for which estimation is being
If the steps themselves are complex,
carried out.
the manual execution effort will
increase. Please refer to the legend
Adjustment Factor = 1 + Total adjustment weight
below to determine the complexity
factor.
This calculation provides us with the Test Case Point
Generation count.
Weight = (((Test steps x 1+ Validation x 2+ Interface x
3)/5) x 3)
TCP-E =Rough TCP – Execution X Adjustment Factor
Table 8.Calculating complexity weights.
C. Calculate Other Tasks
Tes Complexity Factors C.1. One time tasks (person hrs.)
t Test
Tes Comple • Consider Test Strategy preparation time
S.N cas Case • Consider Environment setup time
t Validat Precond xity
o. e Categ • Consider time, frequency and duration of team
Ste ions ition Weight
na ory meetings
me ps
• No. of days depending on prior knowledge,
TC Simpl times no. of staff.
1 1 e 3 2 0 4 • Consider setup of testing support tool like
TC Medi bugzilla, TD etc.
2 2 um 5 3 1 8 • Regression Testing Efforts
TC Comp
• Browser Compatibility
3 3 lex 8 4 2 13
• Additional Configuration
Table 9.Calculating the rough TCP for test execution.
C.2. Miscellaneous task (person hrs.)
• Consider documents like weekly, progress
Number of test cases with Te reports preparation time
Mo test type and weight st
S. • Consider review time of different arti. facts
dule Feature Si Me Co Highly ca (Typically 10% of Total Preparation effort)
N
nam /Screen mp diu mpl Comple se
o. • Consider rework time for document preparation
e le m ex x po (Typically 5% of Total Preparation effort)
4 8 13 0 int
• Consider retesting time for bug fixes (Typically
Mo 10% of Total Execution effort)
dule Screen • Consider regression testing time
1 1 1 2 0 0 0 8 • Consider test data preparation time
Screen • Consider database creation time
2 2 0 6 0 0 48
Screen D. Calculate Productivity Task
3 3 2 0 0 0 8 Productivity sheet, lists out various domain/application
type. Industry standard or Historical data of Productivity
Calculate Rough Test Case Points for Test Case in hrs. /TCP is collected. These are indicative figures
Execution by using the below formulae: based on the platform of choice. This figure will become
more consistent and reliable as we collect historical data

38


from projects that get completed and feedback 2. Total Project efforts(In Person Hours)= total test case
productivity figures into the system. preparation efforts+ total execution efforts+ onetime
task+ miscellaneous task
Table 10.Table representing PCB productivity standards
for test generation. The total TCP is indicative of the size of the testing
project.
Test Generation
Domain/ Type of
Productivity in V. CONCLUSION
application
TCP/person- hrs. The proposed work computes the requirement based
Oracle Application test effort estimation based on IEEE-830: 1998 standard
Web Application 2 of requirement engineering document, soon after freezing
Client Server Application the requirements. The proposed measure is reliable
Database because it is derived from SRS of the software to be
Mainframes Application developed. This makes the estimation precise and perfect.
Installer Applications 3.00 Since test case point estimates provides an approach for
early estimation, if used properly, may help organization
in better product/projects management and tracking. Early
Table 11. Table representing PCB productivity standards estimates can help organization to gain early knowledge
for test execution. about the schedule, project length and thereby early
planning to reduce their cost and gain profits. It facilitates
Test Execution early detection and process improvements. It provides a
Domain/ Type of better view to decision makers. Better planning and
Productivity in
application estimation can be of great help in effective utilization of
TCP/person-hrs.
Oracle Application resources.
Web Application 5
Client Server Application VI. FUTURE WORK
Database There may be different solutions for a single problem.
Mainframes Application Likewise, in many of the available approaches to test
Installer Applications 6.00 effort estimation, the Test Case Point approach is one. To
make this research a more robust method of estimation
over a period of time, it is required to make use of the
E. Calculate Total TCP
available data from past projects. This will definitely
The Total TCP is computed by summing up the individual contribute to the accuracy of test case point estimates and
TCPs for Test Case Generation and Test Execution. their enhancements. The estimation model is not claimed
to be exact, but the approach offers significant practical
1. Test Preparation Productivity (TCP/person hrs.):
advantages over other estimation techniques currently in
Organisation Level Productivity in TCP/person
use. Further research and experimentation will definitely
hrs.[Refer the Industry Standard Productivity figures
provide more benefits in arriving at a method to validate
]
2. Test Execution Productivity (TCP/person hrs.): the estimates.
Organisation Level Productivity in TCP/person This research paper is based on the analysis of the
hrs.[Refer the Industry Standard Productivity] literature available on the estimations common across
3. One time task (person hrs.) industry. To be more specific for the results obtained, one
4. Miscellaneous task (person hrs.) should have seen similar result on large empirical data.
5. Test case Generation effort = (test case preparation x Further work can be done in this direction by validating
preparation adjustment factor)/ test preparation each estimate on large volume of empirical data so that
productivity. there is no chance of error in effort estimations.
6. Test case execution effort = (test case execution x
execution adjustment factor)/ test execution REFERENCES
productivity.
7. Calculate the system test cycles conducted. [1] Software engineering standard committee of IEEE Computer
Society. IEEE Recommended Practice or Software Requirement
F. Effort Calculation Specifications”, IEEE Inc. NY, USA, 1998.
[2] Antonio Bertilino, Software Testing Research: Achievements,
To translate the Test Case Points into the total person challenges and Dreams, IEEE Future of software Engineering-
months involved based on your prior experience estimate FOSE 2007
the number of test case points per person month. [3] Charles R Symons. Function point: Difficulties and Improvements,
IEEE Transactions on Software Engineering, Vol.14, No.1, Jan.
1. Total execution efforts(In Person Hours)= total test 1988.
case execution efforts+ total system test cycles [4] Eduardo Aranha, Filipe de Almeida, ThiagoDiniz, VitorFontes,
conducted X total efforts Paulo Borba. Automated Test Execution Effort Estimation Based
On Functional Test Specification. Proceedings of Testing:

39


Academic and Industrial Conference Practice and Research
Techniques, MUTATION 2007.
[5] Barry Boehm, Cost models for future software life cycle processes,
Annals of Software Engineering, Special Volume on Software
Process and Product Measurement, Neitherlands, 1985
[6] Thomas J McCabe. A Complexity Measure. IEEE Transactions on
Software Engineering, Vol., SE-2, No. 4, Dec. 1976.
[7] AjithaRajan, Michael W Whalen, Mats P. E. Heimdahl.2007.
Model Validation Using Automatically Generated Requirements-
Based Tests. 10th IEEE- High Assurance Systems Engineering
Symposium, 2007.
[8] Aranha E, Borba P. Test Effort Estimation Model Based On Test
Specifications. Testing : Academic and Industrial Conference-
Practice and Research Techniques,IEEE Computer Society, 2007
[9] ZHU Xiaochun, ZHOU Bo, WANG Fan, QU Yi CHEN Lu.
Estimate Test Execution Effort at an Early Stage: An Empirical
Study”, International Conference on Cyber World , IEEE
Computer Society,2008
[10] Qu Yi Zhou Bo, Zhu Xiaochun. Early Estimate the Size of Test
Suites from Use Cases”, 15th Asia-Pacific Software Engineering
Conference, IEEE Computer Society, 2008
[11] DAnielGuerreiro e Silva, Bruno T. de Abreu, Mario Jino. A
Simple Approach For Estimation of Execution of Function Test
Case. IEEE-International Conference on Software Testing
Verification and Validation, 2009
[12] Erika R. C De Almeida, Bruno T. de Abreu, Regina Moraes. An
Alternative Approach to Test Effort Estimation Based on Use
Case. IEEE-International Conference on Software Testing
Verification and Validation, 2009
[13] Suresh Nageshwaran, Test Effort Estimation Using USE CASE
Points. Quality Week 2001, San Francisco,California USA, 2001
[14] The Standish group research for staggering bugs and effort,
http://standishgroup.com
[15] Sharma Ashish, Kushwaha DS, A Complexity measure based on
requirement engineering document, JCSE UK, May 2010
[16] Sharma Ashish, Kushwaha DS, NLP based component extraction
and its complexity analysis, ACM Sigsoft,, Issue 36, January 2011
[17] Kushwaha DS, Misra AK, Software Test Effort Estimation. ACM
Sigsoft, Software Engineering Notes, Vol. 33 , No. 3, May 2008

40

20 54-1-pb

More Related Content

What's hot (12)

Viewers also liked (20)

Similar to 20 54-1-pb (20)

More from Editor IJARCET (20)

Recently uploaded (20)

20 54-1-pb