BizDevOps:
Delivering Business Value Quickly at Scale
Our Last Webinar
Watch Now at QASymphony.com/resources
Our Last WebinarUpcoming Webinar Series
Kickstarting BDD: Scaling for the Enterprise
Sign up Today QASymphony.com/resources
About QASymphony
Test Management | Exploratory Testing | Test Analytics | BDD and more...
The Complete Testing Platform
STATE OF THE INDUSTRY
Organizations
practicing agile
development
Organizations have or
are planning a DevOps
initiative in their
company in the next
12 months
The difference in the
number of code
deployments between
high performing and
low performing IT
organizations
94% 71% 46x
The Pace of Software Development is
Moving Faster Than Ever
Source: V1 2016 State of Agile, Puppet 2016 State of DevOps
The Move to DevOps is Clear
1) Quicker time to customer value
2) Better alignment between businesses and
customers
3) Better ability to respond to customer input
DevTestOps
Business
+
Development
+
Operations
“Bringing the business stakeholders
into DevOps in order to confidently
validate that business processes
work end-to-end across single or
multiple applications.”
What is BizDevOps?
Testing in DevOps
Practices like behavior-driven
development (BDD) and test-
driven development (TDD)
are becoming more popular.
Source: V1 2016 and 2009 State of Agile
78%
Increase in Teams
doing BDD
Getting the Biz in DevOps
Dev
Product
Owner
Tester
Living
Documentation
Scenario: Buy Last Coffee
Given There are 1 Coffees left in the Machine
And I have deposited $5
And I previously have $100 in the bank
When I press the coffee button
Then I should be served a coffee
Feature: Serve Coffee
Coffee should not be served until paid for
Coffee should not be served until the button
has been pressed
If there is no coffee left, then the money
should be refunded
Three Amigos in BDD
01
02
03
04
05
5 Major Problems in DevOps Adoption
BizDevOps
Our Presenters
Kevin Dunne
VP of Strategy
Brandon Cipes
VP DevOps,
Bobby Smith
Director QAS Labs
01
02
03
04
05
01
LACK OF TEST
AUTOMATION COVERAGE
5 Major Problems in DevOps Adoption
“Functional testing is one of the most crucial, time-
consuming, and expensive steps in continuous testing —
so it’s necessary to automate this testing, and to
automate it at higher levels than most Agile teams
achieve today.”
– Forrester
Oh, no…
Freaking OutAcceptance
Tests
Integration
Tests
Unit Tests We for sure have this covered.
Super Confident
I think we got this covered
Semi-Confident
QA Risk When Scaling Automation
RISK LEVEL
BDD to the Rescue
Collaborate
Owner, Dev, Test, design acceptance criteria
Record
Document common syntax for all
Automate
Execute test scenarios that validate behavior
Full Traceability
Behavior Driven Development
01
02
03
04
05
02LACK OF VISIBILITY
MAKES TEST
AUTOMATION HARDER
5 Major Problems in DevOps Adoption
We wrote a bunch of
automated tests, but what
exactly do those tests cover?
INSERT PHOTO ON TOP
Festering Problems
Code
Code
Code
Automation Scripts
Code
Code
Code
Code
What Are
We Testing?
Automation Confusion
Automation frameworks without feature visibility create
business confusion around application coverage.
• Automation becomes pure code, without clear understanding
of test coverage
• QA doesn't know if test coverage is sufficient
Communication Breakdowns
Features and requirements change during Agile and DevOps
sprints. If automation is disconnected from these features, we
might be testing a changed feature set without know it.
• Changes are discovered after automation is completed
• Automation is never updated to reflect adequate coverage
01
02
03
04
05
03
MAINTAINING TEST
VERSIONS WITH CODE
5 Major Problems in DevOps Adoption
Are My Tests Connected with Code?
TESTING
VERSION
CONTROL
BUILD & UNIT TEST
AUTOMATED AND
MANUAL TEST
DEPLOY
Check In
TESTS FAIL
Typical CI Workflow
PLAN
DEV
Trigger
Trigger
FEED BACK
Check In
Trigger
Trigger
TEST PASS
PUSH TO PROD
FEEDBACK
01
02
03
04
05
04
MAINTAINING SINGLE
SOURCE OF TRUTH
IS A NIGHTMARE
5 Major Problems in DevOps Adoption
Siloed Teams
Traditional Software Development
Three Amigos
BDD Process
BDD in JIRA
01
02
03
04
05
WHAT IS THE BUSINESS
VALUE IN THE PIPELINE?
05
5 Major Problems in DevOps Adoption
DEVOPS MATURITY MODEL
PROCESS FOCUS
Define Plan Code Build Integrate Test Release Deploy Operate
AUTOMATION FOCUS
AGILE vs DEVOPS
DevOps, TDD/BDD,
Continuous Testing, CI,
CD, Data Driven,
Automated
Deployment
Scrum, Kanban,
Integration Planning,
Retrospective, Small
Integrations, Daily
Standups, Backlog
Planning
*Cloud Bees, Jenkins World 2016
AGILE UPSTREAM AGILE DOWNSTREAM
1) BDD – BDD can help with alignment and building automation
more efficiently
2) Technical Testing is Here – testing is increasingly
technical and should be treated as code
3) Atlassian Stack – provides the system of record for
“BizDevOps” pipeline and should be accessible to everyone
and built to provide visibility to C-Suite.
Key Takeaways
Questions?

BizDevOps – Delivering Business Value Quickly at Scale

  • 1.
  • 2.
    Our Last Webinar WatchNow at QASymphony.com/resources
  • 3.
    Our Last WebinarUpcomingWebinar Series Kickstarting BDD: Scaling for the Enterprise Sign up Today QASymphony.com/resources
  • 4.
    About QASymphony Test Management| Exploratory Testing | Test Analytics | BDD and more... The Complete Testing Platform
  • 5.
    STATE OF THEINDUSTRY
  • 6.
    Organizations practicing agile development Organizations haveor are planning a DevOps initiative in their company in the next 12 months The difference in the number of code deployments between high performing and low performing IT organizations 94% 71% 46x The Pace of Software Development is Moving Faster Than Ever Source: V1 2016 State of Agile, Puppet 2016 State of DevOps
  • 7.
    The Move toDevOps is Clear 1) Quicker time to customer value 2) Better alignment between businesses and customers 3) Better ability to respond to customer input
  • 8.
  • 9.
    Business + Development + Operations “Bringing the businessstakeholders into DevOps in order to confidently validate that business processes work end-to-end across single or multiple applications.” What is BizDevOps?
  • 10.
    Testing in DevOps Practiceslike behavior-driven development (BDD) and test- driven development (TDD) are becoming more popular. Source: V1 2016 and 2009 State of Agile 78% Increase in Teams doing BDD
  • 11.
    Getting the Bizin DevOps Dev Product Owner Tester Living Documentation Scenario: Buy Last Coffee Given There are 1 Coffees left in the Machine And I have deposited $5 And I previously have $100 in the bank When I press the coffee button Then I should be served a coffee Feature: Serve Coffee Coffee should not be served until paid for Coffee should not be served until the button has been pressed If there is no coffee left, then the money should be refunded Three Amigos in BDD
  • 12.
    01 02 03 04 05 5 Major Problemsin DevOps Adoption BizDevOps
  • 13.
    Our Presenters Kevin Dunne VPof Strategy Brandon Cipes VP DevOps, Bobby Smith Director QAS Labs
  • 14.
    01 02 03 04 05 01 LACK OF TEST AUTOMATIONCOVERAGE 5 Major Problems in DevOps Adoption
  • 15.
    “Functional testing isone of the most crucial, time- consuming, and expensive steps in continuous testing — so it’s necessary to automate this testing, and to automate it at higher levels than most Agile teams achieve today.” – Forrester
  • 16.
    Oh, no… Freaking OutAcceptance Tests Integration Tests UnitTests We for sure have this covered. Super Confident I think we got this covered Semi-Confident QA Risk When Scaling Automation RISK LEVEL
  • 17.
    BDD to theRescue Collaborate Owner, Dev, Test, design acceptance criteria Record Document common syntax for all Automate Execute test scenarios that validate behavior Full Traceability Behavior Driven Development
  • 18.
    01 02 03 04 05 02LACK OF VISIBILITY MAKESTEST AUTOMATION HARDER 5 Major Problems in DevOps Adoption
  • 19.
    We wrote abunch of automated tests, but what exactly do those tests cover?
  • 20.
    INSERT PHOTO ONTOP Festering Problems Code Code Code Automation Scripts Code Code Code Code What Are We Testing? Automation Confusion Automation frameworks without feature visibility create business confusion around application coverage. • Automation becomes pure code, without clear understanding of test coverage • QA doesn't know if test coverage is sufficient Communication Breakdowns Features and requirements change during Agile and DevOps sprints. If automation is disconnected from these features, we might be testing a changed feature set without know it. • Changes are discovered after automation is completed • Automation is never updated to reflect adequate coverage
  • 21.
    01 02 03 04 05 03 MAINTAINING TEST VERSIONS WITHCODE 5 Major Problems in DevOps Adoption
  • 22.
    Are My TestsConnected with Code? TESTING
  • 23.
    VERSION CONTROL BUILD & UNITTEST AUTOMATED AND MANUAL TEST DEPLOY Check In TESTS FAIL Typical CI Workflow PLAN DEV Trigger Trigger FEED BACK Check In Trigger Trigger TEST PASS PUSH TO PROD FEEDBACK
  • 24.
    01 02 03 04 05 04 MAINTAINING SINGLE SOURCE OFTRUTH IS A NIGHTMARE 5 Major Problems in DevOps Adoption
  • 25.
  • 26.
  • 27.
  • 28.
    01 02 03 04 05 WHAT IS THEBUSINESS VALUE IN THE PIPELINE? 05 5 Major Problems in DevOps Adoption
  • 29.
    DEVOPS MATURITY MODEL PROCESSFOCUS Define Plan Code Build Integrate Test Release Deploy Operate AUTOMATION FOCUS AGILE vs DEVOPS DevOps, TDD/BDD, Continuous Testing, CI, CD, Data Driven, Automated Deployment Scrum, Kanban, Integration Planning, Retrospective, Small Integrations, Daily Standups, Backlog Planning *Cloud Bees, Jenkins World 2016 AGILE UPSTREAM AGILE DOWNSTREAM
  • 31.
    1) BDD –BDD can help with alignment and building automation more efficiently 2) Technical Testing is Here – testing is increasingly technical and should be treated as code 3) Atlassian Stack – provides the system of record for “BizDevOps” pipeline and should be accessible to everyone and built to provide visibility to C-Suite. Key Takeaways
  • 32.

Editor's Notes

  • #14 Brandon – - Head up devops group, test automation – Good automation strategy for their devops practices - good code coverage - not automated, don’t have chance in hell
  • #17  - Kill Animations -
  • #18 Enter BDD to the rescue. BDD allows the developer, tester, and product owner to collaborate, record, and automate from a living specification. https://cucumber.io/blog/2017/05/15/intro-to-bdd-and-tdd
  • #20 You also run into the trouble of having a bunch of tests that are automated, yet you don’t know what areas, or features these tests case covering….
  • #21  - Flaky Tests - Reliable Tests – Green Builds - What happens when they fail?
  • #24 Static image as well.
  • #26 In support of the bimodal nature of many customers today, QASymphony has expanded the qTest Platform with qTest Pulse, a solution that is tuned and designed for Agile and DevOps teams scaling their Enterprise BDD processes. qTest Pulse integrates with JIRA in real-time to begin their BDD process, and teams to easily share assets and report across projects whether they are using Pulse or Manager.
  • #27 In support of the bimodal nature of many customers today, QASymphony has expanded the qTest Platform with qTest Pulse, a solution that is tuned and designed for Agile and DevOps teams scaling their Enterprise BDD processes. qTest Pulse integrates with JIRA in real-time to begin their BDD process, and teams to easily share assets and report across projects whether they are using Pulse or Manager. Fullfills requirement and testing (DEV)
  • #28 In support of the bimodal nature of many customers today, QASymphony has expanded the qTest Platform with qTest Pulse, a solution that is tuned and designed for Agile and DevOps teams scaling their Enterprise BDD processes. qTest Pulse integrates with JIRA in real-time to begin their BDD process, and teams to easily share assets and report across projects whether they are using Pulse or Manager. Fullfills requirement and testing (DEV)
  • #32 Notes: Instead of struggling to do new things that your tools won’t let you do, you’ve got to have a solution that supports all the latest thinking in development and testing – like DevOps, shift-left, exploratory, and more.