Friday, November 5, 2010
Friday, November 5, 2010
London Selenium Meetup
Friday, November 5, 2010
How Mozilla Uses Selenium
Stephen Donner
November 3, 2010
London Selenium Meetup
Friday, November 5, 2010
3
Friday, November 5, 2010
Overview
• Humble Beginnings
• First (Real) Cut at Automation
• Current Architecture (diagram)
• Hudson (Continuous Integration)
• Our Driver / TestRunner Implementation
• Thanks, Python+urllib
• Page Object Model
• The Future
• Questions?
3
Friday, November 5, 2010
Humble Beginnings
• All IDE, all the time
• Necessary evil, but painful
• No scripted runs, inadequate reporting
• Graduated to RC
• Still uncoordinated, ad-hoc running
• We all ran tests from the command-line
• Still no great reporting/build history
4
Friday, November 5, 2010
First (Real) Cut at Automation
• TCparams.py - sort options, applications (still
exists)
• AMOfunctions.py - 2,677 lines
• Really meaty -- see https://wiki.mozilla.org/AMOfunctions
• AMOlocators.py - 1,230 lines
• Flat file of XPath and CSS locators
5
Friday, November 5, 2010
Current Architecture
6
Friday, November 5, 2010
Hudson
• Gives us:
• Reporting / notification
• Email / status page / IRC
• Scheduling
• Build history
7
Friday, November 5, 2010
Our Driver / TestRunner Implementation
• Allows us to abstract code from core (driver)
• tests.py - list of smoke / bft / fft, etc. tests
• consolidates all testfiles in one location, per-project
• ability to tag tests:
8
Friday, November 5, 2010
Our Driver / TestRunner Implementation
• suite.py - driver magic; uses Python’s unittest module
– spawns a new, per-browser process
– runs all tests in parallel among browsers
– only time-bound by the longest-running test (/me glares at IE)
9
Friday, November 5, 2010
Our Driver / TestRunner Implementation
• Running tests for specific features is as simple as:
• python suite.py fft
• python suite.py bft collections (logical AND operation)
• etc...
10
Friday, November 5, 2010
Thanks, Python+urllib
• Staging servers down, much?
• Two ways of dealing with this:
1. From suite.py:
2. From SUMO’s sumo_page.py:
11
Friday, November 5, 2010
Page Object Model
page.py
• Base class for ALL pages; most common functions (click, verify, etc.)
• Individual page classes derive from it
12
Friday, November 5, 2010
Page Object Model
sumo_page.py
• All common elements of a SUMO page: log in/out, My Account, header/
footer, etc.
13
Friday, November 5, 2010
Page Object Model
support_home_page.py
• Page elements are defined as class variables
• Operations that can be performed on the above are defined as functions
14
Friday, November 5, 2010
Page Object Model
test_search_on_home_page.py
15
Friday, November 5, 2010
Page Object Model
• Benefits:
• Easy to read (and write) tests
• Reduces the amount of duplicated code
• Less ramp-up time for new testers + community
• No more giant function files (distributed among pages)
• Caveats:
• All tests can now fail if a common element is broken
• e.g..: AMO footer
• Multiple imports
16
Friday, November 5, 2010
The Future
• Projects in the POM where it makes sense
(typically the case)
• Build-in screen / video captures of failures?
• Selenium 2; update our tests
• Native HTTP header manipulation?
• (Specifically, user-agent/accept-headers)
17
Friday, November 5, 2010
Resources / References
• WebQA homepage: https://wiki.mozilla.org/
QA/Execution/Web_Testing
• Automation page on QMO: http://
quality.mozilla.org/docs/webqa/projects/
automation/
• WebQA Mailing List (public):
mozwebqa@mozilla.org
18
Friday, November 5, 2010
Questions?
19
Friday, November 5, 2010
stephend@mozilla.com
Friday, November 5, 2010
Thanks!
stephend@mozilla.com
Friday, November 5, 2010

"How Mozilla Uses Selenium"

  • 1.
  • 2.
  • 3.
  • 4.
    How Mozilla UsesSelenium Stephen Donner November 3, 2010 London Selenium Meetup Friday, November 5, 2010
  • 5.
  • 6.
    Overview • Humble Beginnings •First (Real) Cut at Automation • Current Architecture (diagram) • Hudson (Continuous Integration) • Our Driver / TestRunner Implementation • Thanks, Python+urllib • Page Object Model • The Future • Questions? 3 Friday, November 5, 2010
  • 7.
    Humble Beginnings • AllIDE, all the time • Necessary evil, but painful • No scripted runs, inadequate reporting • Graduated to RC • Still uncoordinated, ad-hoc running • We all ran tests from the command-line • Still no great reporting/build history 4 Friday, November 5, 2010
  • 8.
    First (Real) Cutat Automation • TCparams.py - sort options, applications (still exists) • AMOfunctions.py - 2,677 lines • Really meaty -- see https://wiki.mozilla.org/AMOfunctions • AMOlocators.py - 1,230 lines • Flat file of XPath and CSS locators 5 Friday, November 5, 2010
  • 9.
  • 10.
    Hudson • Gives us: •Reporting / notification • Email / status page / IRC • Scheduling • Build history 7 Friday, November 5, 2010
  • 11.
    Our Driver /TestRunner Implementation • Allows us to abstract code from core (driver) • tests.py - list of smoke / bft / fft, etc. tests • consolidates all testfiles in one location, per-project • ability to tag tests: 8 Friday, November 5, 2010
  • 12.
    Our Driver /TestRunner Implementation • suite.py - driver magic; uses Python’s unittest module – spawns a new, per-browser process – runs all tests in parallel among browsers – only time-bound by the longest-running test (/me glares at IE) 9 Friday, November 5, 2010
  • 13.
    Our Driver /TestRunner Implementation • Running tests for specific features is as simple as: • python suite.py fft • python suite.py bft collections (logical AND operation) • etc... 10 Friday, November 5, 2010
  • 14.
    Thanks, Python+urllib • Stagingservers down, much? • Two ways of dealing with this: 1. From suite.py: 2. From SUMO’s sumo_page.py: 11 Friday, November 5, 2010
  • 15.
    Page Object Model page.py •Base class for ALL pages; most common functions (click, verify, etc.) • Individual page classes derive from it 12 Friday, November 5, 2010
  • 16.
    Page Object Model sumo_page.py •All common elements of a SUMO page: log in/out, My Account, header/ footer, etc. 13 Friday, November 5, 2010
  • 17.
    Page Object Model support_home_page.py •Page elements are defined as class variables • Operations that can be performed on the above are defined as functions 14 Friday, November 5, 2010
  • 18.
  • 19.
    Page Object Model •Benefits: • Easy to read (and write) tests • Reduces the amount of duplicated code • Less ramp-up time for new testers + community • No more giant function files (distributed among pages) • Caveats: • All tests can now fail if a common element is broken • e.g..: AMO footer • Multiple imports 16 Friday, November 5, 2010
  • 20.
    The Future • Projectsin the POM where it makes sense (typically the case) • Build-in screen / video captures of failures? • Selenium 2; update our tests • Native HTTP header manipulation? • (Specifically, user-agent/accept-headers) 17 Friday, November 5, 2010
  • 21.
    Resources / References •WebQA homepage: https://wiki.mozilla.org/ QA/Execution/Web_Testing • Automation page on QMO: http:// quality.mozilla.org/docs/webqa/projects/ automation/ • WebQA Mailing List (public): [email protected] 18 Friday, November 5, 2010
  • 22.
  • 23.
  • 24.