Chapter 10 Automation Testing v1.1
Chapter 10 Automation Testing v1.1
Page 1 of 27
Page 2 of 27
Confidentiality Statement
This document should not be carried outside the physical and virtual boundaries of TCS and
its client work locations. Sharing this document with any person other than a TCS associate
would tantamount to violation of confidentiality agreement signed by you while joining
TCS.
Notice
The information given in this course material is merely for reference. Certain third party
terminologies or matter that may be appearing in the course are used only for contextual
identification and explanation, without an intention to infringe.
Certificate in Software Testing Skill TCS Business Domain Academy
Contents
Chapter-10 Automation Testing ....................................................................................... 5
10.1 Introduction to Automation Testing ......................................................................6
10.2 Automation Testing Process and Phases ...............................................................8
10.3 Automation techniques ....................................................................................... 12
10.4 Automation testing tools .....................................................................................20
Summary ........................................................................................................................26
Page 4 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Introduction
Automation Testing means use of specialized tools to control the execution of tests and
relates the actual results against the expected result.
We are automating manual testing to saves time & money, improves accuracy, increase test
coverage etc.
Learning Objective
After reading this chapter, you will be able to understand:
• Test Automation and why it is needed
• When not to automate
• Which test cases to automate
• How to automate and benefit automation testing
• Automation Testing process
• How to choose an automation tools
• Automation scripting technique
• Record and Playback concept
• Automation Testing Myth
• Automation best Practices
• Test Automation Tools
• TCS Assurance CoE Asserts
Page 5 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
10.1.1 Overview
Manual testing is done by a human sitting in front of a computer cautiously executing the
test steps. Automation Testing means use of specialized tools to control the execution of
tests and relates the actual results against the expected result.
Although manual tests may find many defects in a software application, it is a difficult and
time consuming process. Also it may not be effective in finding particular classes of defects.
Test automation is a process of writing computer program to do testing that would need to
be done manually. Once tests have been automated, they can be run rapidly and
repeatedly. This is often the utmost cost-saving method for software products that have a
lengthy maintenance life, because even small patches over the lifetime of the application
can origin features to break which were working at a prior point in time.
Test automation tools can be costly, and are generally employed in combination with
manual testing. It can be made cost saving in the larger term, particularly when used
repeatedly in regression testing. What to automate, when to automate test, or even
whether one truly needs, automation are crucial decisions which the testing (or
development) team must decide. Selecting the right features of the product for automation
mainly determines the success of the automation. Automating unbalanced features or
features that are ongoing changes should be avoided.
Test automation consist of automating a manual process already in place that uses a
formalized testing process. In simple words, Automation Testing is automating the manual
testing process currently in use.
Page 6 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Note: The more creativity needed for the test case execution, the less appropriate it is for
automation
It is difficult to automate all testing, so it is important to identify which test cases should be
automated first.
The advantage of automated testing is associated to how many times a test can be
repeated. Tests that are only performed a limited times are better left for manual testing.
Page 7 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Good test cases for automation are ones that are run regularly and need a large amounts of
data to perform the same action.
You can get the maximum benefit out of your automated testing efforts by automating:
• Tests that tend to cause human error.
• Repetitive tests run for multiple builds.
• Tests that need multiple data sets.
• Tests that are impossible to perform manually.
• Frequently used functionality that presents high risk conditions.
• Tests that take a lot of effort and time when manual testing.
• Tests that run on several different hardware or software platforms and
configurations.
This section will cover How to Automate Testing, its Benefits, Testing Process, Automation
phases\Roles and How to choose an automation Tools.
You can automate your testing process by using a computer language like VB Scripting and
an automated software tools\applications. There are numerous tools available in the market
that one can use to write automation scripts. If you are doubting, how you can achieve
automation testing activities.
Below is a step-by-step procedure that will help you draw up a plan:
• Identify areas in your software testing that want automation: it’s difficult and not
cost saving to automate the whole testing process of software mostly due to the
expensive nature of the testing tools and also may be the unbalanced nature of
certain sections of application code. So it is very crucial to first find out the areas
that need automation.
• Select right tool for test automation: There are various kinds of testing tools
available; however, choosing the right tool keeping in mind the nature of testing
involved, is very important for automation. Whether it’s a code-driven testing
process or GUI based testing, you must select the suitable tool to automate the
testing.
• Write Test Scripts: you need to develop test cases and scripts to cover large area of
software to ensure that they are functioning properly.
Page 8 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
• Develop test suites: test suites are developed to confirm that the automated test
cases run one by one without any manual intervention. This is complete by creating
a test suite that has multiple test cases, a libraries and command line tool that runs
the test suite.
• Execute the scripts: The test scripts can be executed either manually with the
developer choosing which test suites to run or can be done automatically with a
monitoring mechanism in place. This ensures that problems are detected well in
case any issue arises due to code change etc.
• Create Outcome Reports: you need to create report formats so that individual test
logs describing the actions performed during testing are logged, define the type of
test report format going to created, screenshots, messages etc.
• Identify any potential problems or performance related issues: you must be able
to simply identify any problems detected while testing and their causes, identify
events of any failed test operations and set fix the problems to achieve greater
testing efficiency.
In short we can say that manual testing is effective in projects with multiple operating
environments and hardware configurations, automation of testing is absolutely essential
today to successfully deliver a large scale projects that require execution of repetitive and
complex test cases. Automation testing is not only essential but also critical to the
successful development and delivery of a software project.
Page 9 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Test Tool Selection: Test tool selection mostly depends on the technology the application
under test is built on. For example QTP does not support Informatica. So QTP cannot be
used for testing Informatica applications.
Define the scope of Automation: Scope of automation is the area of our Application under
Test which will be automated. Following points help determine scope:
• Feature that are important for the business
• Scenarios which have large amount of data
• Common functionalities across applications
• Technical feasibility
• Extent to which business components are reused
• Complexity of test cases
• Ability to use the same test cases for cross browser testing
Planning, Design and Development: During this phase we create Automation strategy &
plan, which contains following details
• Automation tools selected
• Framework design and its features
• In-Scope and Out-of-scope items of automation
• Automation test bed preparation
• Schedule and Timeline of scripting and execution
• Deliverables of automation testing
Test Execution: Automation Scripts are executed during this phase. The scripts need input
test data before there are set to run. Once executed they provide detailed test reports.
Maintenance: - As new functionalities are added to the System under Test with successive
cycles, Automation Scripts need to be added, reviewed and maintained for each release
cycle. Maintenance becomes necessary to improve effectiveness of Automation Scripts.
Page 10 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Selecting the right tool can be a tricky task. Tool selection is one of biggest challenges to be
tackled before going for automation. First, identify the requirements, explore various tools
and its capabilities, set the expectation from the tool and go for a Proof of Concept.
Selecting an automated testing tool is essential for test automation. There are a lot of
automated testing tools on the market, and it is important to choose the automated testing
tool that best suits your overall requirements.
For selecting the appropriate tools we need to consider below key points
• Support for Various Applications and Platforms
• Support for Various Operating Systems
• Support for Mobile Devices
• Support for Various Software Testing Types
• Creating Automated Tests Without Programming
• Automated Test Scripting
• Recording Automated Software Tests
• Creating Cross-Browser Web Tests
• Creating Cross-Platform Mobile Web Tests
Page 11 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
This section will cover Scripting Techniques, types of scripting vs cost and Frameworks
/Methodologies
Linear Scripting: Linear scripting approach is a simple record and playback used by a test
engineer to automate a test flow/test case of a system. It can contain some redundant
functions etc. that may not be required at times.
Page 12 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Structured Scripting: Structured scripting uses control structures in the scripts. These
control structures enable testers to control the flow of the test script/test case.
The control structures are typically ‘if-else’, ‘switch’, ‘for’, ‘while’ conditions/ statements
that help in implementing decision making in script, in performing some tasks iteratively
and calling other common functions that house commonly needed functionality
Shared scripting: Shared scripting is one of techniques in which the scripts representing an
application behaviour are shared amongst other scripts. This means that the application
under test has a common functionality that is scripted as shared script and this script is then
called by other scripts. This makes the scripts modular in terms of common functionality.
Page 13 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Data driven Scripting: This technique separates data from the scripts and stores it into
external repository in the form of files. The script contains just the programmed code. This
may be needed when the data needs to be varied over the test run.This script need not be
modified if the data undergoes a change. At times the expected data can also be stored in
the data files that have test data.
Keyword driven Scripting: It is an approach in which the control to check and execute
operations is maintained in external data files. The test data and the operations/sequence of
the test is planned in an external data file and an extra library is needed to interpret this
data in addition to the conventional script. It is an extension of data driven testing.
Page 14 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
A Test automation framework is a collection of assumptions, concepts and tools that offer
support for automated software testing. The main benefit of this framework is the low
maintenance cost. Selecting the right framework/scripting technique helps in maintaining
lower costs of software.
Page 15 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
The following are common methodologies used for automate system or functional testing
for the most business applications.
• Linear Scripting ( Record & Playback)
• Functional decomposition Method
• Keyword driven ( table driven)
• Test Plan driven method(Data driven)
• Hybrid (above two are used in a collaborative manner)
For Example: Consider logging into bus ticket Reservation Application and checking the
application has loaded on successful log-on or not. Below are the steps tester will simply
record steps and add validation steps.
1. SystemUtil.Run "BusReservation.exe","","","open"
2. Dialog ("Login"). WinEdit("User Name:").Set "UserName" here
3. Dialog ("Login"). WinEdit("Passcode:").Set "Password" here
4. Dialog ("Login"). WinButton ("OK"). Click
5. 'Check bus ticket Reservation Window has loaded after successful log-on
6. Window("Bus Ticket Reservation").Check CheckPoint ("Bus ticket Reservation
check box")
Page 16 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
The main concept behind this, is to reduce test cases to their most likely fundamental tasks
and to write user defined functions, business functions and utility scripts which perform
their tasks freely from one another.
Table 3: List of scripts and its functionalities
Script Functionality
Page 17 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
A Controller script is needed to identify the keywords and perform the necessary actions.
Table 4: List of scripts and its functionalities
Script Functionality
Page 18 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Note - The "Test Plan Driven" method reserves best of the advantages of the "Functional
Decomposition" method, while removing most of the disadvantages
There are three source components of a Keyword Driven Framework are. Keyword,
Application Map and Component Function.
• Where the keyword means, an act that can be performed on a GUI Components.
For example. In GUI Component Textbox some Keywords (action) would be Input-
text, verify-value, verify-property etc..
• Where an application map provides Named References for UI components. Also
application maps are nothing but "Object Repository"
• Where Component Functions are those functions that aggressively manipulate or
interrogate UI component. Example would be click on web button with all error
handling, enter data in a Web Edit-box with all error handling stuff. Component
functions could be application dependent function or independent function
Page 19 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
NOTE: Even though QTP promotes itself as Keyword Driven Framework, you cannot
achieve complete test tool and application independence using QTP.
Hybrid:
Hybrid framework is the grouping of one or more than one frameworks discussed above
pulling from their strengths and trying to ease their weaknesses. In hybrid test automation
framework most frameworks evolve into overtime and multiple projects.
Truthfully, a number of things are required to extract value from test automation.
• An automation architect to look the software testing requirements against the
capabilities of automation tool suites.
• Automation engineers who prioritize test automation to particular test areas that
save time and improve quality of testing.
• Plans to reuse automation for a string of upcoming projects to profit from the early
investment.
Page 20 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Automating UI testing is another area full with complications and ineffectiveness due to the
fragility while development UIs and the large cost of test maintenance and execution. It is
particularly unproductive when the test target is code beneath the UI.
It will more effective if test automation is implemented immediately after the closer of code
writing. In an agile development environment practicing nonstop delivery, end-to-end
automation does really speed up delivery.
Page 21 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Selenium Selenium is a popular automated web testing tool and helps you Open Source
automate web browsers across different platforms.
Watir Watir is a set of Ruby libraries for automating web browsers and Open Source
allows you to write tests that are easy to read and maintain.
Windmill Windmill is a web testing tool designed to help testers automate Open Source
and debug web applications. It comes with a cross-browser test
recorder, JavaScript integration and an interactive shell to
automate web browsers.
Ranorex Ranorex allows you to automate your web application testing Commercial
among other things and record user interactions and play them
back to execute your tests. Ranorex is one of the more popular
commercial tools to build and run automated UI and Web tests.
SoapUI SoapUI is a cross-platform functional testing tool. It has been Open Source
explicitly designed to help automatically test APIs such as SOAP
and REST interfaces to certify the interoperability of different
applications.
Sahi Sahi is a tool for automation of web application testing. Open Source
Tellurium Tellurium is a web automation tool that allows you to design Commercial
and write your automated tests by plain English without any
scripting or programming hands-on.
Squish Squish is a GUI testing tool for several platforms, including QT, Commercial
native Windows and Mac applications. Squish allows testers and
developers to build automated tests using familiar scripting
languages such as JavaScript, Python, Perl and Ruby.
Page 22 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Ranorex Ranorex allows you to automate your web application testing Commercial
among other things and record user interactions and play them
back to execute your tests. Ranorex is one of the more popular
commercial tools to build and run automated UI and Web tests.
TestComp TestComplete is an automated testing tool for the Win. Platform. It Commercial
lete permits you to record, script and run GUI tests for applications built
using different frameworks and languages, such as .NET or C++.
Test Test Studio is an automated functional and load testing tool that Commercial
Studio helps you test applications on various platforms built using
different frameworks and tools.
eggPlant eggPlant is a GUI test automation tool for professional software Commercial
applications and enterprise teams. It can be used to automate
different application types, such as .NET, Java and Flash
applications.
Table 7 : Unit Testing Frameworks
NUnit NUnit is a unit-testing framework for every.Net languages. It was .NET, Open
primarily ported from JUnut to .NET and has been redesigned to Source
take advantage of many .NET language features.
xUnit xUnit.net is a unit testing tool for the .NET Framework written by .NET, Open
the original inventor of NUnit. . It is community-focused tool. xUnit Source
.net is the latest technology for unit testing C#, F#, VB.NET and
other .NET languages
PyUnit / The Python unit testing framework, sometimes mentioned to as Python,
unittest "PyUnit", is a Python language version of JUnit. It’s a part of the Open
Python framework and its supports test automation, sharing of Source
setup and shutdown code for tests and many other features.
Page 23 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
TestNG TestNG is a Java testing framework encouraged by JUnit and NUnit Java, Open
and introduces some new functionalities that make it powerful and Source
easier to use. TestNG is designed to cover multiple classes of tests,
including unit, functional, end-to-end and integration tests.
PHPUnit PHPUnit is a popular framework for unit testing in PHP projects. It PHP, Open
offers both a framework that makes it easier to write tests as well Source
as the functionality to easily run, execute and analyse tests and
results.
Symfony Lime is a unit and functional testing framework built for the PHP, Open
Lime popular Symfony PHP web application framework. The framework Source
is designed to have readable outcome from tests, including colour
formatting by following the Test Anything Protocol.
Test::Unit Ruby comes with its own usual unit testing framework as part of Ruby, Open
the Test::Unit namespace and can be used to define basic pass/fail Source
tests and group tests. The framework also comes with tools to run
single or whole groups of tests.
RSpec RSpec is a testing tool for the Ruby programming language. Born Ruby, Open
under the banner of behaviour-driven development, it is designed Source
to make test-driven development more fruitful and enjoyable.
Junit JUnit is a unit testing framework to write repeatable tests in Java. Java, Open
JUnit has been important in the development of test-driven Source
development and is one of the standard testing frameworks for
Java developers.
Sauce Labs Selenium & Mobile Cross-browser Selenium & mobile testing
Page 24 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Page 25 of 27
Certificate in Software Testing Skill TCS Business Domain Academy
Summary
• Manual testing may find many number of defects but it is time consuming and
requires lot of personnel
• Test Automation involves automating the manual process which is currently
undergoing testing
• What to automate, when to automate, or even whether one really needs
automation are crucial decisions which the testing (or development) team must
make.
• Linear, Structured, Shared, Data driven and Keyword driven are the scripting
techniques in test automation
• Function decomposition and Test plan driven methodologies are used in the
automation process
• Automation is an effective way of testing only if used in a proper manner. If not it
creates added over head
Page 26 of 27