Patterns and practices of building
Gherkin based
living documentations
Building SpecLog with SpecFlow
CHRISTIAN HASSA & GASPAR NAGY
ch@techtalk.at & gn@techtalk.at
19th November 2010
Agenda
Walkthrough: Using SpecFlow in a project
• Gathering requirements & formulating acceptance
criteria
• Development & testing concept
• Build & release process, reports
Special challenges
• User Stories vs. Features
• Synchronization Scenarios
• Tool Widgets
• Automation Helpers
About Us
Project Sponsor SpecFlow
Product Owner, Coach
Twitter: @chr99ha
Gáspár Nagy
TechTalk
Committer to SpecFlow
http://gasparnagy.blogspot.com
Twitter: @gasparnagy
Christian Hassa
TechTalk
TechTalk is a software development and
consulting company with ~60 people
located in Vienna, Budapest and Zürich.
We focus on Scrum and .NET.
The research is partially supported by the
project of Eötvös Loránd University:
TÁMOP-4.2.1.B-09/1/KMR-2010-0003
About You
Who knows SpecFlow?
Who knows Cucumber or jBehave
(Gherkin)?
Process Walkthrough
Using SpecFlow in a project
Our Domain: SpecLog
A product developed by TechTalk
managing flow of requirements in agile projects
integrates with SpecFlow and TFS
Team size: ~ 7 people
WPF SmartClient
• central repository
or stand-alone
• occasionally
connected
Deriving scope from goals … is not easy
Deriving user stories from …
… vision/business goals: too far away
… actor goals: not focused enough
Roadmap helped to focus
First milestone goal:
„use SpecLog for SpecLog“
(even if it hurts …)
vision
Product
Backlog
Epic
User Story
Epic
User Story
User Story
User Story
User Story
business
goals
Roadmap
derive scope from goals
actor
goals
Our first milestone
Some actor-goals for our first milestone
Illustrating user stories with examples:
acceptance criteria
Product
Backlog
Epic
User Story
Epic
User Story
User Story
User Story
User Story
Sprint
Backlog
illustrate
using
examples
UI Scribbles as key examples
Acceptance Criteria
as key examples
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Refining: Formalization in Gherkin
• Formalization in pairs
• Ideal: Developer + Product Owner
• Scenario tags: @specifiedSprint
Backlog
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria Given
When
Then
…
…
…
refine
the
specification
UI Scribbles as key examples
specification with examples
Acceptance Criteria
as key examples
Automating Gherkin scenarios
Automation of Gherkin scenarios
… too costly on UI (custom WPF UI)
… instead on controller
Sprint
Backlog
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria Given
When
Then
…
…
…
SpecFlow
{ … }
{ … }
{ … }
automate
literally
specification with examples
executable
specification
Scenario tags:
@automated
@manual
Development concept
In development we have to find the right balance between…
• High quality – the application will be used by a broad
audience
• Changing directions – the implementation also serves as a
test for the concept
Our goals are
• Develop the application outside-in
• Preferable AC by AC
• Automate AC through controller (view model) layer
• Design concept but no detailed design upfront
• Many things from XP (TDD, Pair programming, common
code ownership)
Demo
Outside-in Development
Task Board
The tasks are aligned to the AC
We use TFS with Excel fronted
Build & Release Process, Reports
We wanted to build a confortable infrastructure that
supports our process
• CI
• Short release cycle through automated build
• Build quality measures
• Reports
Automated Builds - CI
Runs on every check-in (no gated check-in)
No versioning
Builds the application & runs unit tests (*.Tests)
• Fails if any of these fails
Automated Builds – Acceptance
Triggered by check-ins, cumulated (1h)
Verifies build quality
• Executes automated acceptance tests
• Detects version compatibility issues
Produces ready-to-use release
• Increments version number
• Produces an installer
• Generates test execution report (“living
documentation”)
• Publishes the results on a website
Build Result Web
Test Execution Report
Test Execution Report
Executing Acceptance Test
Implementation of an AC can be a longer process
• Might need several days
• Might be developed by more then one person
• Execution of the entire test suite might be long
We cannot expect that all acceptance test passes all the
time
What we rather need is
• Ensure that “done” stories are still working
• Get a feedback about the stories currently
implemented
Executing Acceptance Test
Testing
Developer: Smoke testing of UI
• tag user story with „TestIt“
Tester: exploratory testing of all „TestIt“ user stories
• time boxed, once a week
Limited number of manually executed workflow tests
• extended with new features
Bugs: formulized to Gherkin acceptance criteria
• when automation possible
• grouped to user story on backlog when not fixed
immediately
Special Challenges
in applying BDD
Living documentation
Sprint
Backlog
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria Given
When
Then
…
…
…
F
F F
F F F
F F
AC
AC
AC
AC
AC
Automation
{ … }
{ … }
{ … }
refine
the
specification
validate
continuously
specification with examples
executable
specification
living documentation
Features “done”Current sprint
AC
US
AC
AC
AC
US
AC
AC
merge
when
accepted
validate
continuously
Evolving the living documentation
Product/Sprint Backlog
User Story 1
AccCrit 1
AccCrit 2
User Story 2
AccCrit 3
AccCrit 4
Living Documentation
Feature 1
AccCrit 1
AccCrit 2
Feature n
AccCrit 4
AccCrit m
User Story n
AccCrit 5
AccCrit m
AccCrit 3
AccCrit 5
„Done“
• Next possible increments of
product (units of work)
• Organized /refined according
to priority, value, effort, risk,
actor-goals, …
• Product Documentation
• Organized/refined according to
functionality and overview
• Versioned/maintained with
source code
Synchronization Scenarios
The application should support collaborative editing with
multiple occasionally connected clients
• This is a general cross-cutting concern
• Hard to test manually
• Essential feature of the application
Solution: we concretize this requirement for each story
where it can be applied
• This is part of a check list – so we don’t forget it
Synchronization Scenarios
Synchronization of a user story “value text”
@automated @done
Scenario: should synchronize value text of user story
Given a workspace for a shared repository
And the workspace is active on 'Client1'
And the repository is open on 'Client2'
And a user story is in the repository
When I set the value text of the user story to 'some value' on 'Client1'
And all changes are synchronized
Then the user story in the repository on 'Client2' should have the value
text 'some value'
Synchronization Scenarios
Specifying goal text in “local mode” – no need to mention clients
@automated @done
Scenario: should be able to specify value text for user story through
card
Given an active workspace for a repository
And a user story in the repository
And a card for the user story on the workspace
When I set the value text of the user story to 'some value'
Then the user story in the repository should have the value text 'some
value'
Synchronization Scenarios
Specifying AC headline (title) – no need to mention workspaces even
@automated @done
Scenario: Acceptance criteria can be described with a headline
Given a user story
And an acceptance criterion for the user story
When I set the headline of the acceptance criterion to 'some headline'
Then the acceptance criterion should have the headline 'some headline' in
the repository
[Given("a user story")]
public void GivenAUserStory()
{
EnsureOpenStandaloneRepository();
GivenAUserStoryIsInTheRepository();
}
Synchronization Scenarios
Conflict resolution – for the winner
@automated @done @conflict_resolution
Scenario Outline: first synchronized concurrent modification of requirement goal text should win
Given a workspace for a shared repository
And the workspace is active on 'Client1'
And the workspace is active on 'Client2'
And a <requirement> is in the repository
When I set the goal text of the <requirement> to 'some goal1' on 'Client1'
And I set the goal text of the <requirement> to 'some goal2' on 'Client2'
And all changes are synchronized by 'Client1'
And all changes are synchronized by 'Client2'
And all changes are synchronized by 'Client1'
Then the <requirement> in the repository should have the goal text 'some goal1'
Examples:
| requirement |
| business goal |
| actor-goal |
| user story |
Synchronization Scenarios
Conflict resolution – for the looser
@automated @specified @conflict_resolution
Scenario Outline: concurrent modifications of goal text of requirements that lose should be
captured as conflicts
Given a workspace for a shared repository
And the workspace is active on 'Client1'
And the workspace is active on 'Client2'
And a <requirement> is in the repository
And the <requirement> has no conflicts
When I set the goal text of the <requirement> to 'some goal1' on 'Client1'
And I set the goal text of the <requirement> to 'some goal2' on 'Client2'
And all changes are synchronized by 'Client1'
And all changes are synchronized by 'Client2'
Then the <requirement> in the repository should have the conflict 'conflicting goal: some
goal2' introduced by 'Client2'
Examples:
| requirement |
| business goal |
| actor-goal |
| user story |
Dealing with cross-cutting concepts
Tool-widget
• Floats over workspace
• Can be open/closed using control bar
• Position kept per user session
Formalizing tool widget concept in Gherkin
examples for
tool widget
concept
examples of
tool widgets
(extended over time)
manual test
that there is a
specific tool widget
instance
TestContext, TestClient, TestServer &
Automation Drivers
When automating acceptance criteria, similar tasks have
to be solved from different step bindings
• These should also communicate with each other
Given a user story
And an acceptance criterion for the user story
• TestContext is a class that is injected into the step
binding classes to provide a platform for the
communication
• TestClient, TestServer – entry points for the
automation of a client or a server instance
• Automation Drivers – grouping together the
automation code for automating a class (view model)
Demo
TestContext, TestClient, TestServer & Automation
Drivers
Questions and Answers
Thank you!
More information at
• www.specflow.org
• www.speclog.org
• www.techtalk.at
www.specflow.org

Tech talk specflow_bddx_hassa_nagy

  • 1.
    Patterns and practicesof building Gherkin based living documentations Building SpecLog with SpecFlow CHRISTIAN HASSA & GASPAR NAGY [email protected] & [email protected] 19th November 2010
  • 2.
    Agenda Walkthrough: Using SpecFlowin a project • Gathering requirements & formulating acceptance criteria • Development & testing concept • Build & release process, reports Special challenges • User Stories vs. Features • Synchronization Scenarios • Tool Widgets • Automation Helpers
  • 3.
    About Us Project SponsorSpecFlow Product Owner, Coach Twitter: @chr99ha Gáspár Nagy TechTalk Committer to SpecFlow http://gasparnagy.blogspot.com Twitter: @gasparnagy Christian Hassa TechTalk TechTalk is a software development and consulting company with ~60 people located in Vienna, Budapest and Zürich. We focus on Scrum and .NET. The research is partially supported by the project of Eötvös Loránd University: TÁMOP-4.2.1.B-09/1/KMR-2010-0003
  • 4.
    About You Who knowsSpecFlow? Who knows Cucumber or jBehave (Gherkin)?
  • 5.
  • 6.
    Our Domain: SpecLog Aproduct developed by TechTalk managing flow of requirements in agile projects integrates with SpecFlow and TFS Team size: ~ 7 people WPF SmartClient • central repository or stand-alone • occasionally connected
  • 7.
    Deriving scope fromgoals … is not easy Deriving user stories from … … vision/business goals: too far away … actor goals: not focused enough Roadmap helped to focus First milestone goal: „use SpecLog for SpecLog“ (even if it hurts …) vision Product Backlog Epic User Story Epic User Story User Story User Story User Story business goals Roadmap derive scope from goals actor goals
  • 8.
  • 9.
    Some actor-goals forour first milestone
  • 10.
    Illustrating user storieswith examples: acceptance criteria Product Backlog Epic User Story Epic User Story User Story User Story User Story Sprint Backlog illustrate using examples UI Scribbles as key examples Acceptance Criteria as key examples US US US Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria
  • 11.
    Refining: Formalization inGherkin • Formalization in pairs • Ideal: Developer + Product Owner • Scenario tags: @specifiedSprint Backlog US US US Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Given When Then … … … refine the specification UI Scribbles as key examples specification with examples Acceptance Criteria as key examples
  • 12.
    Automating Gherkin scenarios Automationof Gherkin scenarios … too costly on UI (custom WPF UI) … instead on controller Sprint Backlog US US US Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Given When Then … … … SpecFlow { … } { … } { … } automate literally specification with examples executable specification Scenario tags: @automated @manual
  • 13.
    Development concept In developmentwe have to find the right balance between… • High quality – the application will be used by a broad audience • Changing directions – the implementation also serves as a test for the concept Our goals are • Develop the application outside-in • Preferable AC by AC • Automate AC through controller (view model) layer • Design concept but no detailed design upfront • Many things from XP (TDD, Pair programming, common code ownership)
  • 14.
  • 15.
    Task Board The tasksare aligned to the AC We use TFS with Excel fronted
  • 16.
    Build & ReleaseProcess, Reports We wanted to build a confortable infrastructure that supports our process • CI • Short release cycle through automated build • Build quality measures • Reports
  • 17.
    Automated Builds -CI Runs on every check-in (no gated check-in) No versioning Builds the application & runs unit tests (*.Tests) • Fails if any of these fails
  • 18.
    Automated Builds –Acceptance Triggered by check-ins, cumulated (1h) Verifies build quality • Executes automated acceptance tests • Detects version compatibility issues Produces ready-to-use release • Increments version number • Produces an installer • Generates test execution report (“living documentation”) • Publishes the results on a website
  • 19.
  • 20.
  • 21.
  • 22.
    Executing Acceptance Test Implementationof an AC can be a longer process • Might need several days • Might be developed by more then one person • Execution of the entire test suite might be long We cannot expect that all acceptance test passes all the time What we rather need is • Ensure that “done” stories are still working • Get a feedback about the stories currently implemented
  • 23.
  • 24.
    Testing Developer: Smoke testingof UI • tag user story with „TestIt“ Tester: exploratory testing of all „TestIt“ user stories • time boxed, once a week Limited number of manually executed workflow tests • extended with new features Bugs: formulized to Gherkin acceptance criteria • when automation possible • grouped to user story on backlog when not fixed immediately
  • 25.
  • 26.
    Living documentation Sprint Backlog US US US Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Acceptance Criteria Given When Then … … … F FF F F F F F AC AC AC AC AC Automation { … } { … } { … } refine the specification validate continuously specification with examples executable specification living documentation Features “done”Current sprint AC US AC AC AC US AC AC merge when accepted validate continuously
  • 27.
    Evolving the livingdocumentation Product/Sprint Backlog User Story 1 AccCrit 1 AccCrit 2 User Story 2 AccCrit 3 AccCrit 4 Living Documentation Feature 1 AccCrit 1 AccCrit 2 Feature n AccCrit 4 AccCrit m User Story n AccCrit 5 AccCrit m AccCrit 3 AccCrit 5 „Done“ • Next possible increments of product (units of work) • Organized /refined according to priority, value, effort, risk, actor-goals, … • Product Documentation • Organized/refined according to functionality and overview • Versioned/maintained with source code
  • 28.
    Synchronization Scenarios The applicationshould support collaborative editing with multiple occasionally connected clients • This is a general cross-cutting concern • Hard to test manually • Essential feature of the application Solution: we concretize this requirement for each story where it can be applied • This is part of a check list – so we don’t forget it
  • 29.
    Synchronization Scenarios Synchronization ofa user story “value text” @automated @done Scenario: should synchronize value text of user story Given a workspace for a shared repository And the workspace is active on 'Client1' And the repository is open on 'Client2' And a user story is in the repository When I set the value text of the user story to 'some value' on 'Client1' And all changes are synchronized Then the user story in the repository on 'Client2' should have the value text 'some value'
  • 30.
    Synchronization Scenarios Specifying goaltext in “local mode” – no need to mention clients @automated @done Scenario: should be able to specify value text for user story through card Given an active workspace for a repository And a user story in the repository And a card for the user story on the workspace When I set the value text of the user story to 'some value' Then the user story in the repository should have the value text 'some value'
  • 31.
    Synchronization Scenarios Specifying ACheadline (title) – no need to mention workspaces even @automated @done Scenario: Acceptance criteria can be described with a headline Given a user story And an acceptance criterion for the user story When I set the headline of the acceptance criterion to 'some headline' Then the acceptance criterion should have the headline 'some headline' in the repository [Given("a user story")] public void GivenAUserStory() { EnsureOpenStandaloneRepository(); GivenAUserStoryIsInTheRepository(); }
  • 32.
    Synchronization Scenarios Conflict resolution– for the winner @automated @done @conflict_resolution Scenario Outline: first synchronized concurrent modification of requirement goal text should win Given a workspace for a shared repository And the workspace is active on 'Client1' And the workspace is active on 'Client2' And a <requirement> is in the repository When I set the goal text of the <requirement> to 'some goal1' on 'Client1' And I set the goal text of the <requirement> to 'some goal2' on 'Client2' And all changes are synchronized by 'Client1' And all changes are synchronized by 'Client2' And all changes are synchronized by 'Client1' Then the <requirement> in the repository should have the goal text 'some goal1' Examples: | requirement | | business goal | | actor-goal | | user story |
  • 33.
    Synchronization Scenarios Conflict resolution– for the looser @automated @specified @conflict_resolution Scenario Outline: concurrent modifications of goal text of requirements that lose should be captured as conflicts Given a workspace for a shared repository And the workspace is active on 'Client1' And the workspace is active on 'Client2' And a <requirement> is in the repository And the <requirement> has no conflicts When I set the goal text of the <requirement> to 'some goal1' on 'Client1' And I set the goal text of the <requirement> to 'some goal2' on 'Client2' And all changes are synchronized by 'Client1' And all changes are synchronized by 'Client2' Then the <requirement> in the repository should have the conflict 'conflicting goal: some goal2' introduced by 'Client2' Examples: | requirement | | business goal | | actor-goal | | user story |
  • 34.
    Dealing with cross-cuttingconcepts Tool-widget • Floats over workspace • Can be open/closed using control bar • Position kept per user session
  • 35.
    Formalizing tool widgetconcept in Gherkin examples for tool widget concept examples of tool widgets (extended over time) manual test that there is a specific tool widget instance
  • 36.
    TestContext, TestClient, TestServer& Automation Drivers When automating acceptance criteria, similar tasks have to be solved from different step bindings • These should also communicate with each other Given a user story And an acceptance criterion for the user story • TestContext is a class that is injected into the step binding classes to provide a platform for the communication • TestClient, TestServer – entry points for the automation of a client or a server instance • Automation Drivers – grouping together the automation code for automating a class (view model)
  • 37.
  • 38.
  • 39.
    Thank you! More informationat • www.specflow.org • www.speclog.org • www.techtalk.at www.specflow.org