Fueling DevOps with a Testing Trifecta: How the New World of Testing is Driving Agile Development

Fueling DevOps with a Testing Trifecta:
How the New World of Testing is Driving
Agile Development
DevOps: Continuous Delivery
@Scott_D_Edwards
#CAWorld
Scott Edwards
CA Technologies
DevTest Portfolio
Session Number

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type of
warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only
Terms of this Presentation

Abstract
In this session, see how CA’s DevTest portfolio arm your testing
teams with three testing solutions built for Agile teams:
1. Test automation to create test cases right from requirements
2. Generating synthetic test data to be used on demand
3. Eliminating testing constraints by virtualizing every environment
you need access to
Make testing the accelerator, not the hurdle, to fast application
delivery with high quality.
Scott
Edwards
CA Technologies
Director of Product
Marketing

Agenda
TESTING IS BROKEN
CUSTOMER SATISFACTION AT THE HEART OF DIGITAL INITIATIVES
WHY QUALITY IS CRITIAL
THE NEED FOR AN AGILE TESTING TRIFECTA
HOW TO ACT NOW
1
2
3
4
5

What is More of a Threat to You?
External Breach Regular Software Update

outage was caused
by a failure during a
daily system refresh“

KNIGHT TRADING …
a “software update accessed outdated
code that suddenly made more than
$440 MILLION
in bad trades in
30 MINUTES
BNY MELLON
problem arose after bank and 3rd party
software provider pushed out an upgrade…
Affecting over
1,200
MUTUAL FUNDS

Any Plan that Defers Testing To The End of a Release …
is BROKEN!
In an Agile /
Continuous
Delivery World …
© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

For Embarking on Digital Initiatives
FreeForm Study, “Exploiting the Software Advantage” 2015Top Drivers
Improved Customer Satisfaction
Better Workforce Productivity
Improved Customer Experience
Improved Competitive Differentiation
Better Agility/ Speed to Market
Opening up New Markets
49%
44%
41%
38%
37%
35%
TESTING CANNOT BE AN AFTERTHOUGHT!

Gatepoint Research Sept 2015
147 IT Executives/Directors/Managers responded
87% Of respondents rate loss of customer satisfaction as potentially the
most critical issue stemming from flubbed application delivery

What is Your Delivery Experience?
What is Your Customer Experience?

12 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Any Focus
On Speed…
Without Quality
Will Result in
CONTINUOUS
DELIVERY SUICIDE

Ignite Continuous Delivery
Eliminate Your Development and Testing Barriers
Focus Area #1
DEVELOPER SPEED
Focus Area #2
AGILE TESTING
TRIFECTA
Focus Area #3
IGNITING DEVOPS
Director of
Development
KEY LAUNCH
ASPECTS
 Service
Virtualization on
Demand in Azure
Marketplace
 SV for Developers
(vapi.ca.com)
KEY LAUNCH
ASPECTS
1. Test automation
2. Test data
management
3. Test constraint
removal
KEY LAUNCH
ASPECTS
 DevOps Simulation
Game
 Hero Kit
 Benefit/ROI
Calculators
© 2015 CA. ALL RIGHTS RESERVED. FOR INTERNAL USE ONLY.
Dir. Of QA / Testing The Influencer

Agile Testing Trifecta … Critical for Continuous Delivery
1
Test Automation
2
Test Data
Management
3
Test Constraint
Removal
TODAY’S TRADITIONAL
TESTING METHODS INHIBIT
DEVELOPMENT SPEED AND
IMPACT QUALITY
A new testing
methodology,
combined with
advanced
technology, is
needed

Test Automation Solutions from CA1

“WHY DO SOFTWARE PROJECTS FAIL?”
Changing
or poorly
documented
requirements*
#1 Reason
48%
*Source: IDC White Paper, sponsored by Application Developers Alliance, Application Developers Alliance Developer Insights Report: A Global Survey of Today's Developer, August 2015.
“…software developers would do well
to invest more time and effort into
understanding the requirements of
the systems they plan to build”
Application Developers Alliance in partnership with IDC
Al Hilwa
IDC Program Director,
Application Development Software

CA Test Case Optimizer
Automatically Create Test Cases… Right from Requirements
REAL WORLD RESULTS
 Cover 100% of requirements in the smallest number of tests
 80% increase in test coverage
 Reduce test cycles by 30% by automatically detecting any
redundant, outdated and duplicate test cases
Define & manage changing Software
Requirements, mapping them to
unambiguous visual flow charts
56%Build Quality into your
Software from the Start
of software defects
can be traced back
to ambiguities in
requirements1
Financial Services Company
Reduced Test cases from
326 to just 17!
1
Test Automation

CA Application Test w/ Mobile Test Automation
Complete test coverage for Agile Practices … Mobile to Mainframe
Beneath
the UI … lies
the complexity
to be tackled
MORE THAN GUI TESTING!
Invoke and verify the behavior of each
layer that makes up an application
Automate Unit, Functional, Load,
Regression & Performance Testing
KEY BENEFITS
 Use for API, Rest API, Mobile, Swift, and SAP testing
 Presentation layer test automation for Web & Mobile UIs
 Reporting and collaboration capabilities that allow team
members to execute, monitor and analyze test results
Financial Service Company
 Run over 400 tests on a nightly basis
 QA runs 460 tests whenever
development releases a build
 83% reduction in defects
1
Test Automation

Test Data Management Solutions from CA2

Why Test Data
Anonymised live
data may only
cover around
30% of a bank’s
test coverage
requirements
£800 million
worth of fines to businesses breaching financial
principles in 2015 (Financial Conduct Authority)
Fines and regulations by
the EU's General Data
Protection Regulation
starting in 2016
Businesses will have to notify customers – in plain language – what
information about them is collected and how it is used
GDPR fines from the
Information
Commissioner’s Office
(ICO) currently planned
for 2 per cent of a
business’s global revenue
(capped at €100million)
FCA planning
unlimited fines

CA Test Data Manager
Generate synthetic test data for maximum coverage
REAL WORLD RESULTS
 Create millions of rows of high quality data in
just minutes
 Reduce infrastructure costs by up to $50k per
database
 Reduce the time needed to generate synthetic
test data by as much as 50%
1Bender RBT
63% of data breaches
caused by human error
in exposing sensitive
records
$5.4 Million = average
cost of data breach
incidents
2
Test Data Mgmt.
Automatic creation of realistic
test data sets required for
testing
Designed to Eliminate
the Risk of Data Breach
Large Credit Card Company
 70% reduction in time and effort for getting access to
Test Data (Months to days)
 80% reduction in time spent manually searching for the
‘right data’
 70% reusability of Test Data Conditions for releases

New Test Data & Test Automation Integrations
TCO w/ Rally
TCO w/ HP ALM
TDM w/ Service Virtualization
TDM w/ Mainframe
Automatically convert user stories into the
smallest set of test cases with 100%
coverage, linked to the right data and
expected results
Import existing requirements and test
cases to an unambiguous, active
flowchart, de-duplicate them, and reduce
test cycles by up to 30%
Generate realistic virtual data which covers
the full range of possible scenarios for
effective service virtualization
High Performance Masking and Test Data
Generation for IMS and DB2 using native
utilities and operations, and automated
data profiling.
2
Test Data Mgmt.

Test Constraint Removal Solutions from CA3

Why Virtual Services?
Today’s Hurdle to Innovation? Constraints across the SDLC
Incomplete
Developmen
t
Data
Constraints
System
Constraints
Cost
Constraints
Development
Constraints
80%
Only
56%
79%
of teams experience
delays in development
and QA due to unavailable
dependencies
of critical dependencies
are unavailable when
dev and test teams
need them
of teams face prohibitive
restrictions, time limits
or access fees on needed
3rd party services
Source: voke Market Snapshot TM Report: Service Virtualization – January 2015
3
Remove Testing
Constraints

Service
Virtualization
the Wind tunnel of
Software
Accelerate
Application
Delivery
Simulate Dependent
Systems
Provide 24/7
Access
Increase Developer
Productivity
1 2 3
Data
Constraints
System
Constraints
Cost
Constraints
Development
Constraints
How Service Virtualization Eliminates Testing Constraints
CA Service
Virtualization
3
Remove Testing
Constraints

“Shift-left” Testing
Without SV
WITH SV
Dev1
Dev2
Dev3
System Test
Integration
Performance Test
UAT
SystemDev
1
Dev
2
Dev
1
Dev
3
Dev
2
Dev
3 Integration Performance UAT…
Uncertain delivery schedule – defects persist until UAT
Typical composite app today
waits for whole assembly to begin
With CA Service Virtualization quality
effort moved earlier in the lifecycle
Reduction in SDLC
Faster Delivery
“Went from finding 40% of our defects in the 3rd
and final stage of testing to often finding ZERO.
Service Virtualization turned this final test phase
into a dress rehearsal for production.”
- AutoTrader http://bit.ly/AutoTrader-SV
3
Remove Testing
Constraints

CA Continuous Application Insight
Regression & API Testing … Reinvented
The CAI Difference
 Provides system of record behind the UI or API
 Identifies dependencies between application
servers, databases and 3rd-party services
Turbocharge Your
Service Virtualization
Global Technology Company
 Automated 3500 regression tests
Large Insurance Company
 Automatically generated 300-600 test cases
a day … w/ 90% coverage
Deep Insight
to Diagnose
Right Issues
Automatic
Creation of Test &
Virtual Services
End-to-End
Transaction
Discovery
3
Remove Testing
Constraints

Test Automation from Requirements to Production
Development
Pre-Production
QA/Test – Goal > 80% Automation
Functional  Regression  Integration  Build  Mobile
CA Test Case Optimizer
Test Less, Cover More
 Test cases optimized
 Deduplicate manual tests
 Quantify costs and time for changes
CA Service Virtualization
Unconstrained Development and Test
 Simulate missing data, functions, and services.
 Simulate APIs and third-party systems
 Turn contracts into assets
Requirements
CA Application Test with Mobile
Any Test, Anywhere
 API functional and non-functional testing
 Create the test once and then run across all targets
 Build level reporting for simplified analysis
 Support for Selenium and Appium
Performance
Load Testing
CA Test Data Manager
The Right Data, in the Right Place at the Right Time
 Eliminate manual data creation and masking
 Minimize security risks by not exposing PII
 Get data to testers and outsourcers fast
Shift Left
Unit Testing
CA Continuous Application Insight
Automate the Automation
 Automatically generate 1000s of test cases
 Provides system of record for what is behind the UI or API
 Follow the transaction across all systems and get detailed error reporting
 Logging and production monitoring
Test cases created and
validated against
requirements.
Review mobile
testing needs.
Mobile and
load testing via
cloud providers.

Your Next Steps

New Benefits Calculators

DevOps Simulation Game Experience…
Experience it @ CA World
• See the Demo in Theater 3 area
Experience ½ Day Simulation Game!
• Santa Clara, CA - December 3
• Boulder, CO - December 10
• Plano, TX - January 12
• Chicago, IL – January 19
• Atlanta, GA – January 26
• New York, NY – February 2
Register Today at http://bit.ly/DevOpsSimXP_RS

Q & A

Recommended Sessions
SESSION # TITLE DATE/TIME
DO3T21S
Case Study: Nordstrom Succeeds in E-Commerce
with Innovative Performance Testing to Reduce
Cycle Times and Improve Quality
DEVOPS THEATER 3
Thursday at 11:30 am
DO4T17S
Case Study: T-Mobile’s DevOps and Continuous
Delivery Journey – Building a Foundation for a
Future Built for Agility
DEVOPS THEATER 4
Thursday at 1:00 Pm
DO3T24S
Business Transformation: Accelerate DevTest with
Microsoft AZURE™ and Service Virtualization on
Demand
DEVOPS THEATER 3
Thursday at 3:00 pm

For More Information
To learn more, please visit:
http://cainc.to/Nv2VOe
CA World ’15

Fueling DevOps with a Testing Trifecta: How the New World of Testing is Driving Agile Development

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Fueling DevOps with a Testing Trifecta: How the New World of Testing is Driving Agile Development (20)

More from CA Technologies (20)

Recently uploaded (20)

Fueling DevOps with a Testing Trifecta: How the New World of Testing is Driving Agile Development