Functional testing the_good_the_bad_and_the_ugly
27 likes11,173 views
John Ferguson Smart emphasizes the importance of automated functional testing and various technical agile practices, such as test-driven development, in achieving effective agile adoption. He discusses the structure of good automated acceptance tests, the need for living documentation, and the use of various BDD tools, while providing practical examples. The document outlines how to discover, implement, and execute acceptance criteria to enhance testing processes.
Functional testing the_good_the_bad_and_the_ugly
- 3. Technical Agile Practices Test Driven Development Continuous Integration/Delivery Automated Acceptance Criteria Refactoring Pair Programming “Without these technical prac1ces, your agile adop1on is hollow” -‐ Ma6 Wynne
- 4. Technical Agile Practices Test Driven Development Continuous Integration/Delivery Automated Acceptance Criteria Refactoring Pair Programming “Without these technical prac1ces, your agile adop1on is hollow” -‐ Ma6 Wynne
- 5. A good func<onal test tells a story
- 7. QA loses its potential if only done at the end of the project
- 9. Living Documentation - test results that serve everyone
- 10. So show me the tools!
- 12. in theory in practice The Record-Replay Scam
- 13. What do ogres and good functional tests have in common?
- 14. (This test is not like an ogre)
- 15. Remember...good automated acceptance tests have layers
- 16. Start off with the business objectives Business-‐level requirements
- 17. Describe the business flow High-‐level steps (business flow)
- 18. Describe the page interactions Page-‐level interac<ons
- 19. Page implementation details HTML details go here
- 20. Business Rules Business Flow Page/Component interac<ons Page/Component details Good automated acceptance tests have layers
- 21. Come on - can we see the tools now?
- 22. Unit tes<ng BDD tools (39%) Given/When/Then (40%) Tabular (13%)
- 23. WebDriver / Selenium 2 Mature browser automa<on Wide adop<on Cross-‐browser support Cross-‐browser support
- 24. ...or use Selenium Hub
- 25. appium Uses the WebDriver API Can run on SauceLabs (with some limita<ons)
- 26. Watir Web Application Testing in Ruby browser.goto 'http://bit.ly/watir-example' browser.text_field(:name => 'entry.0.single').set 'Watir' browser.checkbox(:value => 'Ruby').set Ruby DSL for web tes<ng WebDriver-‐based Fluent and readable No na<ve support for Page Objects?
- 27. Geb Groovy Browser Automation Browser.drive { go "http://myapp.com/login" assert $("h1").text() == "Please Login" $("form.login").with { username = "admin" password = "password" login().click() } assert $("h1").text() == "Admin Section" } Groovy DSL for web tes<ng WebDriver-‐based Fluent and readable Supports Page Objects
- 28. 28 High level BDD Many tools are available
- 29. 29 Narrative: In order to increase sales of advertised articles As a seller I want buyers to be able to easily find ads for articles they want to buy Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale. search_by_keyword_and_location.story Business-‐friendly specifica<ons An example in JBehave
- 30. 30 Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale. search_by_keyword_and_location.story Scenario: Searching by keyword and location Given Sally wants to buy a <present> for her son When she looks for '<present>' in the '<category>' category Then she should obtain a list of ads for <expected> for sale. Examples: |present |category |expected| |puppy |Pets & Animals | puppies| |kitten |Pets & Animals | kittens| |seiko |Jewellery & Watches| watch | Tabular examples
- 31. 31 Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of puppy ads search_by_keyword_and_location.story public class SearchAdsSteps { @Steps BuyerSteps buyer; @Given("Sally wants to buy a $present for her son") public void buyingAPresent(String present) { buyer.opens_home_page(); } @When("she looks for $keyword in the $category category") public void adSearchByCategoryAndKeyword(String category, String keyword) { buyer.chooses_category_and_keywords(category, keyword); buyer.performs_search(); } @Then("she should obtain a list of $keyword ads") public void shouldOnlySeeAdsContainingKeyword(String keyword) { buyer.should_only_see_results_with_titles_containing(keyword); } } Step implementa<ons in your favorite language
- 32. 32 From web tests to living documentation
- 34. 34 2 Automate your acceptance criteria 1 Discover your acceptance criteria 4 Execute your acceptance tests 3 Implement your acceptance criteria
- 35. 35 1 Discover your acceptance criteria Feature: Browse Catalog In order to find items that I would like to buy As a customer I want to be able to browse through the catalog Story: Browse by category In order to find items more easily As a customer I want to be able to browse through the product categories Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Define acceptance criteria for each story
- 36. 36 1 Discover your acceptance criteria Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes Clarify the acceptance criteria with examples
- 37. 37 2 Automate your acceptance criteria Story: Browse by category In order to find items more easily As a customer I want to be able to browse through the product categories Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes Narrative: In order to find items more easily As a customer I want to be able to see what product categories exist Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes We now have executable specifica<ons
- 38. 38 2 Automate your acceptance criteria ...but they are reported as ‘pending’
- 39. 39 3 Implement your acceptance criteria Narrative: In order to find items more easily As a customer I want to be able to see what product categories exist Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes
- 40. 40 4 Execute your acceptance tests What tests have been run
- 41. 41 What requirements have been tested? 4 Execute your acceptance tests
- 42. 42 4 Execute your acceptance tests How did we show that a par<cular requirement worked?
- 43. 43 4 Execute your acceptance tests How did we show that a par<cular requirement worked?
- 44. Demo time!
- 46. John Ferguson Smart Thank you