Quality assurance frameowrk

Automation Based Framework

MARCH 2020
Excellerent solutions
Ethiopia Delivery Center
Contents
Introduction.................................................................................................................................................5
Quality Assurance Framework.....................................................................................................................6
BDD – Behavior Driven Development......................................................................................................7
TDD – Test Driven Development.............................................................................................................8
DDD – Data Driven development.............................................................................................................8
Framework Component.............................................................................................................................10
Continuous Integration Service.............................................................................................................10
Release:.............................................................................................................................................10
Build:..................................................................................................................................................10
Test Run:............................................................................................................................................10
Repository:........................................................................................................................................10
Code Review:.....................................................................................................................................10
Test Coverage:...................................................................................................................................10
Test Management.................................................................................................................................11
Test Case:...........................................................................................................................................11
Test Suites:........................................................................................................................................11
Test Executions:.................................................................................................................................11
Test Reporting:..................................................................................................................................11
Test Strategy..........................................................................................................................................11
Iteration :...........................................................................................................................................12
Construction Iteration:......................................................................................................................12
Release:.............................................................................................................................................12
Production:........................................................................................................................................12
Requirement Management...................................................................................................................12
Backlog Stories:.................................................................................................................................12
Change:..............................................................................................................................................12
Defect/Bug:........................................................................................................................................13
Risk:...................................................................................................................................................13
Collaboration:....................................................................................................................................13
Automation Test....................................................................................................................................13
 Configuration File:.............................................................................................................................14
Input Test Data:.................................................................................................................................14
Object Repository:.............................................................................................................................14
Process......................................................................................................................................................14
PDCA (Plan-Do-Check-Act/Adjust):........................................................................................................14
Automation Tools......................................................................................................................................16
Organization Capability.............................................................................................................................17
Summary...................................................................................................................................................17
Document Control
Document Detail

Title: Quality Assurance Framework

Version: 0.1

Date: March, 2020

Electronic File Name: QA-Framework.docx

Electronic File Location: -

Author:

Contributors:

Change Control

Issue Date Version Details Author

<Issues date> v0.1d Working Draft, not yet for review Abenezer Germande

Referenced Documentation

Ref Document Name Electronic File Location

Introduction
The need for better quality Software products requires more than manual testing, high
number of Test Engineers or extended testing cycles. It requires a systemic approach for
managing the entire Quality Assurance process, development effort and the use of Test
automation technologies.

Software Test automation makes use of specialized tools to control the execution of tests
and compares the actual results against the expected result. Organization who has
embraced test automation enjoy significant competitive advantages, including lower costs,
faster time to market, and better detection of defects. Studies showed the practices of
different development teams that lead to higher software delivery performance.
Automation was a key differentiator.

It found that elite DevOps teams do only about 10% of their testing manually These teams:

 Can produce multiple deploys per day

 Need less than one hour of lead time for changes (between code commit and code
deploy).
 Have a change failure rate of 0–15%
 Can restore an application or service in less than an hour after an unplanned outage
or service incident occurs

They may not always choose to deploy multiple times a day, but the code of a high-
performing team is actually deployable at any time. This is due in part to the fact that
testing happens all the time, so quality is built in.

Low performance teams are the one who have 30-50% of manual test. These teams:

 Deploy between once a week and once a month

 Require one to six months of lead time for changes, on average
 Have a change failure rate of 46–60%
 Need between one week and one month to restore an application or service after an
unplanned
outage or service incident.

Test Automation enabled high performance teams to spend more time doing new work and
less time remediating security issues or defects
These benefits make test automation look really attractive but achieving them in real life
requires plenty of hard work. If automation is not done well it will be abandoned and
benefits will never be realized.

The general problem with test automation seems to be not understanding that test
automation is a software project on its own right. Software projects fail if they do not follow
processes and are not managed adequately, and Quality Assurance and test automation
projects are not different. Excellerent solutions understand how important it is to have a
disciplined approach to software Quality assurance and this document explains our Test
Automation framework as a part of Quality Assurance/ Testing Framework.

Quality Assurance Framework

Our Quality Assurance framework enabled with Test automation framework aims to achieve
quality at speed through

 Design quality
 Process Quality
 Product Quality

The framework enforces built in quality.

Our framework also includes the concerns of different stakeholders, Tools (both on the self
and open source Products) and industry standards in Software Testing Life Cycle. In our
environment, Testers generally fall into three categories, with distinct preferences and
capabilities:

 Developer testers prefer tools that are more API-driven and enjoy working in their
preferred coding environments for test automation
 QA Technical testers are test automation specialists who usually prefer not to use
a coding environment for test writing, opting instead for higher-level tools that use
modeling and hide some of the complexity
 Business testers are non-technical people who do acceptance testing, usability
testing, and even some functional testing Currently, all of this is done manually They
prefer to use natural language to write automation tests

Software testing involves different testing approaches and Frameworks, the following
frameworks are included in this Quality Assurance framework.
 AUTOMATION TEST FRAMEWORKS

BDD TDD DDD

Figure 1: Automation Test Framework Types

BDD – Behavior Driven Development

Is a refinement of TDD and DDD that aims to streamline development through narrowing
communication gaps, creating a better understanding of the customer, and allowing for
continuous communication. Simply put, BDD is a way of combining business requirements
with code and allows you to understand the behavior of the system from a business/end-
user perspective. Tools for Behavior Test like Cucumber and fitness.
Discovery:

Discovery is about getting the appropriate folk together to figure out what we are building.
They think through the business rules and examples that illustrate them

Formulation:

Formulation is writing those examples down in readable statements.

Automation:

A practice to automate the agreed upon statements using tools like cucumber-gherkin

TDD – Test Driven Development

Is a process of developing software, where a test is written prior to writing a code. Once
that is done, developers will work towards writing just enough code to pass the test, and
then begin refactoring.

Unit Test:

A test where individual units or components of software are tested whereby each unit of the
software code performs as expected.

Code:

Set of instructions written by programmer in order to fulfill a given unit test for a given
feature.

Refactor:

Refactoring is a controlled technique for improving the design of an existing code base.
DDD – Data Driven development
An approach to development that connects the implementation to an evolving model;
placing the focus of the project on the core domain (sphere of knowledge), the logic behind
it, and forces collaboration between technical and nontechnical parties to improve the
model.
Framework Component
Continuous Integration Service
Continuous Integration (CI) is a development practice where developers integrate code into
a shared repository frequently, preferably several times a day. Each integration can then be
verified by an automated build and automated tests. After verification and successful build,
there will be a continuous release.

Release:
A testable feature/product automatically pushed to a staging environment for QA team to
apply functional and non-functional tests.

Build:
A practice of merging all developers’ working copies of code to a shared mainline several
times a day, where code review, unit test, integration test and test coverages are applied
when merging.

Test Run:
A process of executing unit and integration test scripts written by development team that
are to be executed at first in the ci/cd pipeline prior to the build process.

Repository:
A shared environment to store files, codes and documents where developers actively
participate to push/merge the changes they made and clone/pull any latest updates from
the server to their workstation.

Code Review:
A practice of applying technology-oriented standards, best practices set forth by
International Standard Organizations, technology providers, IDE, custom practices and
opens communities.

Test Coverage:
An amount measured in percentage used to describe the degree to which the source code of
a program is executed when a particular test suite runs.
Test Management
Test management, process of managing the tests. A test management is also performed
using tools to manage both types of tests, automated and manual, that have been
previously specified by a test procedure.

Test Case:

A TEST CASE is a set of actions executed to verify a particular feature or functionality of

your software application. A Test Case contains test steps, test data, precondition, and post
condition developed for specific test scenario to verify any requirement. The test case
includes specific variables or conditions, using which a testing engineer can compare
expected and actual results to determine whether a software product is functioning as per
the requirements of the customer.

Test Suites:

Test suite is a container that has a set of tests which helps testers in executing and
reporting the test execution status. It can take any of the three states namely Active, in
progress and completed.

Test Executions:

Test execution is the process of executing the code and comparing the expected and actual
results.

Test Reporting:

Test reporting is a process whereby test metrics are reported in summarized format to
update the stakeholders which enables them to take an informed decision.

Test Strategy
An agile team works as a single team towards a common objective of achieving quality
through incremental approach.
Iteration :
A stage where we perform Identifying people for testing, installing chosen testing tools and
scheduling resources are included.

Construction Iteration:
Here the agile team follows the prioritized requirement practice: with each iteration, the
teams take the most essential requirements remaining from the work item stack and
implement them.

Release:
A continuous process wherein a system or feature is deployed to a production environment.
This process also incorporates end-user training, support people and operational people.

Production:
After the release stage, the product will move to the production stage.

Requirement Management
Application development requires constant and effective communication between teams
responsible for business analysis, development, testing and operations, and the goal of
collaboration is to concentrate on core application development activities and reduce
operational overhead.

Backlog Stories:
A prioritized features list, containing short descriptions of all functionalities desired in the
product.

Change:
A practice to entertain a change in specifications, user requirements, design, code or
feature.
Defect/Bug:
Issues that have been identified and being communicated to development team when
testing the product.

Risk:
Is an expectation of loss, a potential problem that may or may not occur in the future.
Generally, it is caused due to lack of information, control or time.

Collaboration:
A process where teams, product owners and project managers come to a discussion table or
using other means of communication channel to work together to achieve the desired
product.

Automation Test
Test automation is the use of software (under a setting of test preconditions) to execute
tests and then determine whether the actual outcomes and the predicted outcomes are the
same. It relies on automated test cases which mimics the actions of a human tester in
terms of creating initial conditions to execute the test, entering the input data to drive the
test, capturing the output, evaluating the result, and finally restoring the system back to its
original state.

A test automation infrastructure, consists of test tools, equipment, test scripts, procedures,
and people needed to make test automation efficient and effective. The creation and
maintenance of a test automation framework are key to the success of any test automation
effort within an organization. The idea behind an automation infrastructure is to ensure
the following:

 Different test tools and equipment are coordinated to work together.

 The library of the existing test case scripts can be reused for different test projects,
thus minimizing the duplication of development effort.
 Nobody creates test scripts in their own ways.
 Consistency is maintained across test scripts.
 The test suite automation process is coordinated such that it is available just in time
for different type of testing.
 People understand their responsibilities in automated testing
Configuration File:
If any information is changed from the properties file, you don’t need to recompile class. In
other words, the advantage of using properties file is we can configure things that are prone
to change over a period of time without need of changing anything in code.

For E.g. We keep application URL, database connection, environment variable in the
properties, xml or JSON file, so in case you want to run test from on other test
environment, just change the URL in a property file and that’s it. You do not require building
the whole project again.

Input Test Data:

It is convenient to keep data for automated tests in special storages that support sequential
access to a set of data, for example, Excel sheets, database tables, arrays, and so on. Often
data is stored either in a text file and are separated by commas or in Excel files and are
presented as a table. If you need to add more data, you simply modify the file either in any
text editor or in Microsoft Excel (in case of hard-coded values, you should modify both data
and code).

Object Repository:

An object repository is a common storage location for all objects. In Selenium Web-Driver
context, objects would typically be the locators used to uniquely identify web elements.

The major advantage of using object repository is the segregation of objects from test
cases. If the locator value of one web-element changes, only the object repository needs to
be changed rather than making changes in all test cases in which the locator has been used.
Maintaining an object repository increases the modularity of framework implementation.

Process
Testing process is methodical process by which the functionality of the product is
determined in terms of certain specific requirements and criteria.

PDCA (Plan-Do-Check-Act/Adjust):
An iterative and continuous method used in software development for the control and
improvement of process and products.
Figure 3: Automation Process
Automation Tools

Figure 4: Automation framework Tools

Organization Capability

Figure 5: Organization capability

Summary
Using architecture, technology and tools will in fact help an organization to have the
following key benefits.

1. Significant improvement in the product quality

2. Improved performance of the software
3. Minimal cost of production
4. Reliable infrastructure managed by coding
5. Improvement in productivity of the organization
6. Faster deployment
7. Promotes fast and reliable problem-solving techniques
8. Improves customer satisfaction
9. Increases revenue
10. Individual improvement and cross-skilling
11. Lower volume of defects
12. Lower cost of a release