Writing better
Behat tests
Tim Hunt
Senior Developer The Open University
17 May 2023 #MootIEUK23
2
Writing better Behat tests
1 Behat overview
2 Good tests in general
3 Common mistakes in Moodle Behat tests
4 Good practices
5 Summary
2
Behat
overview
4
A good example
Adapted from question/format/xml/tests/behat/import_export.feature
@qformat @qformat_xml
Feature: Test importing questions from Moodle XML format.
In order to reuse questions
As a teacher
I need to be able to import them in XML format.
@javascript @_file_upload
Scenario: import some multiple choice questions from Moodle XML format
Given the following "courses" exist:
| fullname | shortname |
| Test Course | TC100 |
And the following "users" exist:
| username |
| teacher |
And the following "course enrolments" exist:
| user | course | role |
| teacher | TC100 | editingteacher |
When I am on the "Course 1" "core_question > course question import" page logged in as "teacher"
And I set the field "id_format_xml" to "1"
And I upload "question/format/xml/tests/fixtures/multichoice.xml" file to "Import" filemanager
And I press "Import"
Then I should see "Parsing questions from import file."
And I should see "Importing 1 questions from file"
And I should see "What language is being spoken?"
5
About Behat
Follows a simple script
- written in 'English’
Interacts with Moodle like a user would
- most like a screen-reader
Works in a separate Moodle install
- which starts empty for each scenario
Checks functionality, not visuals
How Behat works?
Let’s not go there!
6
The test pyramid
Human
testing
UI tests (Behat)
Unit tests (PHPunit)
Slower
Faster
More integrated
More isolated
Good tests in
general
1
8
4-phase test pattern
See, for example, http://xunitpatterns.com/Four%20Phase%20Test.html
Setup
Execute
Verify
Clean-up  not needed in Moodle
9
A good example
Adapted from question/format/xml/tests/behat/import_export.feature
@qformat @qformat_xml
Feature: Test importing questions from Moodle XML format.
In order to reuse questions
As a teacher
I need to be able to import them in XML format.
@javascript @_file_upload
Scenario: import some multiple choice questions from Moodle XML format
Given the following "courses" exist:
| fullname | shortname |
| Test Course | TC100 |
And the following "users" exist:
| username |
| teacher |
And the following "course enrolments" exist:
| user | course | role |
| teacher | TC100 | editingteacher |
When I am on the "Course 1" "core_question > course question import" page logged in as "teacher"
And I set the field "id_format_xml" to "1"
And I upload "question/format/xml/tests/fixtures/multichoice.xml" file to "Import" filemanager
And I press "Import"
Then I should see "Parsing questions from import file."
And I should see "Importing 1 questions from file"
And I should see "What language is being spoken?"
10
Good tests
Test one thing
Make it obvious what is being tested
Make it obvious what the expected behaviour is
Are well organised
Plugins -> Features -> Scenarios
11
Bad tests
Are opaque
Are fragile
Are slow
12
A bad example
Adapted from mod/openstudio/tests/behat/content_block_socical.feature
Feature: Open Studio notifications
In order to track activity on content I am interested in
As a student
I want recive notifications about my posts and comments
Background:
Given the following "users" exist:
| username | firstname | lastname | email |
| teacher1 | Teacher | 1 | teacher1@asd.com |
| teacher2 | Teacher | 1 | teacher1@asd.com |
| student1 | Student | 1 | student1@asd.com |
| student2 | Student | 2 | student2@asd.com |
And the following "courses" exist:
| fullname | shortname | category |
| Course 1 | C1 | 0 |
And the following "course enrolments" exist:
| user | course | role |
| teacher1 | C1 | editingteacher |
| teacher2 | C1 | editingteacher |
| student1 | C1 | student |
| student2 | C1 | student |
And the following open studio "instances" exist:
| course | name | description | grouping | groupmode | pinboard | idnumber | tutorroles |
| C1 | Demo Open Studio | Notifification description | GI1 | 1 | 99 | OS1 | editingteacher |
And all users have accepted the plagarism statement for "OS1" openstudio
Given I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1"
And I follow "Add new content"
And I set the following fields to these values:
| id_visibility_3 | 1 |
| Title | Module post |
| Description | Module post |
And I press "Save"
And I follow "My Content"
And I follow "Add new content"
And I set the following fields to these values:
| id_visibility_3 | 1 |
| Title | Module post 1 |
| Description | Module post 1 |
And I press "Save"
13
A bad example continued
Adapted from mod/openstudio/tests/behat/content_block_socical.feature
Scenario: Interactive emoticons in content block social
When I am on the "Demo Open Studio" "openstudio activity" page logged in as "teacher1"
And I wait until the page is ready
# The emoticons should be the gray icon when user doesn't react it.
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//img[contains(@src, 'inspiration_grey_rgb_32px')]" "xpath_element" should
exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//img[contains(@src, 'participation_grey_rgb_32px')]" "xpath_element" should
exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//img[contains(@src, 'favourite_grey_rgb_32px')]" "xpath_element" should
exist
Then I click on "Module post 1" "link"
And I wait until the page is ready
And I click on "0 Favourites" "link"
And I click on "0 Smiles" "link"
And I click on "0 Inspired" "link"
And I wait until the page is ready
And I am on the "Demo Open Studio" "openstudio activity" page
# The emoticons should be the blue icon when user reacts it.
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//img[contains(@src, 'inspiration_rgb_32px')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//img[contains(@src, 'participation_rgb_32px')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//img[contains(@src, 'favourite_rgb_32px')]" "xpath_element" should exist
Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1"
And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element"
And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element"
And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element"
Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "teacher1"
And I click on "Module post 1" "link"
And I should see "2 Favourites"
And I should see "2 Smiles"
And I should see "2 Inspired"
3
Common
mistakes
in Moodle Behat tests
15
Irrelevant navigation
Adapted from mod/quiz/tests/behat/editing_add.feature in Moodle 3.2
# ...
And I log in as "teacher1"
And I follow "Course 1"
And I follow "Quiz 1"
And I navigate to "Edit quiz" node in "Quiz administration“
# ...
Slow Fragile Irrelevant
16
Setting things up using the UI
Adapted from mod/openstudio/tests/behat/content_block_socical.feature
Feature: Open Studio notifications
In order to track activity on content I am interested in
As a student
I want recive notifications about my posts and comments
Background:
# ...
Given I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1"
And I follow "Add new content"
And I set the following fields to these values:
| id_visibility_3 | 1 |
| Title | Module post |
| Description | Module post |
And I press "Save"
And I follow "My Content"
And I follow "Add new content"
And I set the following fields to these values:
| id_visibility_3 | 1 |
| Title | Module post 1 |
| Description | Module post 1 |
And I press "Save“
# ...
Slow Fragile Irrelevant
17
Identifying things on-screen using XPath or CSS
Adapted from question/format/xml/tests/behat/import_export.feature
Scenario: Interactive emoticons in content block social
# ...
# The emoticons should be the blue icon when user reacts it.
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//img[contains(@src, 'inspiration_rgb_32px')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//img[contains(@src, 'participation_rgb_32px')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element" should exist
Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//img[contains(@src, 'favourite_rgb_32px')]" "xpath_element" should exist
Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1"
And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element"
And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element"
And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element"
Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "teacher1"
# ...
Fragile Unclear
18
Long rambling tests
question/type/pmatch/tests/behat/basic_test.feature
Scenario: Create, edit then preview a pattern match question.
When I am on the "Course 1" "core_question > course question bank" page logged in as teacher
# Create a new question.
And I add a "Pattern match" question filling the form with:
| Question name | My first pattern match question |
| Question text | Listen, translate and write |
| id_usecase | Yes, case must match |
| id_allowsubscript | Yes |
| id_allowsuperscript | Yes |
| id_forcelength | warn that answer is too long and invite respondee to shorten it |
| id_applydictionarycheck | Do not check spelling of student |
| id_sentencedividers | ?! |
| id_converttospace | ;: |
| id_synonymsdata_0_word | any |
| id_synonymsdata_0_synonyms | "testing|one|two|three|four" |
| Answer 1 | match (testing one two three four) |
| id_fraction_0 | 100% |
| id_feedback_0 | Well done! |
| id_otherfeedback | Sorry, no. |
| Hint 1 | Please try again. |
| Hint 2 | Use a calculator if necessary. |
Then I should see "My first pattern match question"
# Checking that the next new question form displays user preferences settings.
When I press "Create a new question ..."
And I set the field "item_qtype_pmatch" to "1"
And I click on "Add" "button" in the "Choose a question type to add" "dialogue"
Then the following fields match these values:
| id_usecase | Yes, case must match |
| id_allowsubscript | Yes |
| id_allowsuperscript | Yes |
| id_forcelength | warn that answer is too long and invite respondee to shorten it |
| id_applydictionarycheck | Do not check spelling of student |
| id_sentencedividers | ?! |
| id_converttospace | ;: |
And I press "Cancel"
# Preview it. Test correct and incorrect answers.
And I am on the "My first pattern match question" "core_question > preview" page
And I set the following fields to these values:
| How questions behave | Interactive with multiple tries |
| Marked out of | 3 |
| Marks | Show mark and max |
And I press "Start again with these options"
Then I should see "Listen, translate and write"
And the state of "Listen, translate and write" question is shown as "Tries remaining: 3"
When I set the field "Answer:" to "testing"
And I press "Check"
Then I should see "Sorry, no."
And I should see "Please try again."
When I press "Try again"
Then the state of "Listen, translate and write" question is shown as "Tries remaining: 2"
When I set the field "Answer:" to "testing one two three four"
And I press "Check"
Then I should see "Well done!"
Then the state of "Listen, translate and write" question is shown as "Correct"
# Backup the course and restore it.
When I log out
And I log in as "admin"
When I backup "Course 1" course using this options:
| Confirmation | Filename | test_backup.mbz |
When I restore "test_backup.mbz" backup into a new course using this options:
| Schema | Course name | Course 2 |
Then I should see "Course 2"
When I navigate to "Question bank" in current page administration
Then I should see "My first pattern match question"
# Edit the copy and verify the form field contents.
When I choose "Edit question" action for "My first pattern match question" in the question bank
Then the following fields match these values:
| Question name | My first pattern match question |
| Question text | Listen, translate and write |
| id_synonymsdata_0_word | any |
| id_synonymsdata_0_synonyms | "testing|one|two|three|four" |
| Answer 1 | match (testing one two three four) |
| id_fraction_0 | 100% |
| id_feedback_0 | Well done! |
| id_otherfeedback | Sorry, no. |
| Hint 1 | Please try again. |
| Hint 2 | Use a calculator if necessary. |
And I set the following fields to these values:
| Question name | Edited question name |
And I press "id_submitbutton"
Then I should see "Edited question name"
Huge time-waste if the failure is at the end
Good Scenarios each test one thing
- Then the failure message tells you what’s broken
Slow Fragile
19
Messy organisation
question/type/pmatch/tests/behat/
Unclear
4
Good
practices
21
Organise and name things clearly
Naming things is hard
- but getting it right helps everyone – including future you!
Organised
22
Use Given, When and Then correctly
Only test one thing per scenario
Remember the four-phase test pattern
local/monitor/tests/behat/local_monitor.feature
@ou @ou_vle @local @local_monitor
Feature: Check the monitor script, one of two that checks the server is working OK
In order to monitor the server system
As some back-end system component (supposing this script is still used, which we don't know)
I need to have the monitor script run
Scenario: Check monitor script
Given the following "courses" exist:
| fullname | shortname | format |
| Course 1 | C1 | topics |
And the following "activities" exist:
| activity | name | intro | course | idnumber |
| label | L1 | <a href="../local/monitor/index.php?display=1">MonLink</a> | C1 | label1 |
When I log in as "admin"
And I am on "Course 1" course homepage
And I follow "MonLink"
Then I should see "All tests completed successfully"
Clear
23
Important Given steps
.
Given the following "..." exist:
“…” can be
- Standard entities like “user”, “course”, “activity”, “course enrolments”, …
- Full list in lib/behat/classes/behat_core_generator.php
- You can extend this for your plugin
Given the following config values are set as admin:
| showuseridentity | email,profile_field_oucu |
Fast Robust
Extensible
24
class behat_mod_quiz_generator extends behat_generator_base {
protected function get_creatable_entities(): array {
return [
'group overrides' => [
'singular' => 'group override',
'datagenerator' => 'override',
'required' => ['quiz', 'group'],
'switchids' => ['quiz' => 'quiz', 'group' => 'groupid'],
],
'user overrides' => [
'singular' => 'user override',
'datagenerator' => 'override',
'required' => ['quiz', 'user'],
'switchids' => ['quiz' => 'quiz', 'user' => 'userid'],
],
];
}
}
Extending entity generation
From mod/quiz/tests/generator/behat_mod_quiz_generator.php
Given the following "mod_quiz > group overrides" exist:
| quiz | group | attempts |
| Test quiz | G1 | 2 |
25
Important When steps
When I am on the "Quiz 1" "mod_quiz > Grades report" page
When I am on the "Intro" "page activity" page logged in as student
When I follow "Edit profile"
When I press "Save changes"
When I set the field "Question text" to "Edited question text."
.
When I click on "Student" "link" in the "View as" "block"
When I set the following fields to these values:
| Question name | AV question |
| Type of recording | Customised audio/video |
Fast Clear Robust
Extensible
26
Extending navigation
/**
* Convert page names to URLs for steps like 'When I am on the "[identifier]" "[page type]" page'.
*
* Recognised page names are:
* | pagetype | name meaning | description |
* | course question bank | Course name | The question bank for a course |
* | preview | Question name | The screen to preview a question |
*
* @param string $type identifies which type of page this is, e.g. 'Preview'.
* @param string $identifier identifies the particular page, e.g. 'My question'.
* @return moodle_url the corresponding URL.
* @throws Exception with a meaningful error message if the specified page cannot be found.
*/
protected function resolve_page_instance_url(string $type, string $identifier): moodle_url {
switch (strtolower($type)) {
case 'course question bank':
return new moodle_url('/question/edit.php',
['courseid' => $this->get_course_id($identifier)]);
case 'preview':
[$questionid, $otheridtype, $otherid] = $this->find_question_by_name($identifier);
return new moodle_url('/question/bank/previewquestion/preview.php',
['id' => $questionid, $otheridtype => $otherid]);
default:
throw new Exception('Unrecognised core_question page type "' . $type . '."');
}
}
Adapted from question/tests/behat/behat_core_question.php
When I am on the "Course 1" "core_question > course question bank" page
27
Important Then steps
Then I should see "Embedded question progress for Course 1"
Then I should not see "Fill in the Blanks"
Then "Page 3" "link" should exist
Then "Equation editor" "button" should be visible
Then the field "Describe this image" matches value "Awesome!"
Then the following fields match these values:
Then "Student 1" row "Username" column of "generaltable" table should contain "student1“
Fast Clear Robust
28
More Then steps
Then I should see "Frog" in the "activity" "core_course > Activity chooser tab"
Then I should see "Incorrect" in the "student2" "table_row"
Then "Delete" "button" should not exist in the "Confirm" "dialogue"
Then "Unread posts" "link" in the "Forum1" "list_item" should be visible
- various things like link, button, list item, dialogue, block, region, …
- full list in lib/behat/classes/partial_named_selector.php
- can be extened for your plugin.
- a kitten dies any time you use “css_element” or “xpath_element”
Clear Robust
Extensible
29
Extending selectors
From course/tests/behat/behat_course.php
class behat_course extends behat_base {
public static function get_partial_named_selectors(): array {
return [
new behat_component_named_selector(
'Activity chooser screen', [
"%core_course/activityChooser%//*[@data-region=%locator%][contains(concat(' ', @class, ' '), ' carousel-item ')]"
]
),
new behat_component_named_selector(
'Activity chooser tab', [
"%core_course/activityChooser%//*[@data-region=%locator%][contains(concat(' ', @class, ' '), ' tab-pane ')]"
]
),
];
}
}
Clear Robust
… in the "activity" "core_course > Activity chooser tab"
30
/**
* Put the specified questions on the specified pages of a given quiz.
*
* The first row should be column names:
* | question | page | maxmark |
*
* @param string $quizname the name of the quiz to add questions to.
* @param TableNode $data information about the questions to add.
*
* @Given quiz :quizname contains the following questions:
*/
public function quiz_contains_the_following_questions($quizname, TableNode $data) {
// ... Lots of complex code ...
}
Completely custom steps
A great option when appropriate
- Other extensibility makes it less necessary
- Ensure the step text clearly belongs to your plugin
Given quiz "Quiz 1" contains the following questions:
| question | page |
| TF1 | 1 |
Fast Robust
Example adapted from mod/quiz/tests/behat/behat_mod_quiz.php
5
Summary
32
Summary
Remember everything you already know about testing
Learn by doing and looking at examples (and docs)
- but judge what you are looking at before you copy it
Remember the good practices and avoid the bad ones
Get support from other people
- https://moodledev.io/general/channels#developer-chat
There is lots you could learn, but you don’t need to learn it all at once
- you can get a long way with just some basic steps
- simple is good!

Writing better Behat tests

  • 1.
    Writing better Behat tests TimHunt Senior Developer The Open University 17 May 2023 #MootIEUK23
  • 2.
    2 Writing better Behattests 1 Behat overview 2 Good tests in general 3 Common mistakes in Moodle Behat tests 4 Good practices 5 Summary
  • 3.
  • 4.
    4 A good example Adaptedfrom question/format/xml/tests/behat/import_export.feature @qformat @qformat_xml Feature: Test importing questions from Moodle XML format. In order to reuse questions As a teacher I need to be able to import them in XML format. @javascript @_file_upload Scenario: import some multiple choice questions from Moodle XML format Given the following "courses" exist: | fullname | shortname | | Test Course | TC100 | And the following "users" exist: | username | | teacher | And the following "course enrolments" exist: | user | course | role | | teacher | TC100 | editingteacher | When I am on the "Course 1" "core_question > course question import" page logged in as "teacher" And I set the field "id_format_xml" to "1" And I upload "question/format/xml/tests/fixtures/multichoice.xml" file to "Import" filemanager And I press "Import" Then I should see "Parsing questions from import file." And I should see "Importing 1 questions from file" And I should see "What language is being spoken?"
  • 5.
    5 About Behat Follows asimple script - written in 'English’ Interacts with Moodle like a user would - most like a screen-reader Works in a separate Moodle install - which starts empty for each scenario Checks functionality, not visuals How Behat works? Let’s not go there!
  • 6.
    6 The test pyramid Human testing UItests (Behat) Unit tests (PHPunit) Slower Faster More integrated More isolated
  • 7.
  • 8.
    8 4-phase test pattern See,for example, http://xunitpatterns.com/Four%20Phase%20Test.html Setup Execute Verify Clean-up  not needed in Moodle
  • 9.
    9 A good example Adaptedfrom question/format/xml/tests/behat/import_export.feature @qformat @qformat_xml Feature: Test importing questions from Moodle XML format. In order to reuse questions As a teacher I need to be able to import them in XML format. @javascript @_file_upload Scenario: import some multiple choice questions from Moodle XML format Given the following "courses" exist: | fullname | shortname | | Test Course | TC100 | And the following "users" exist: | username | | teacher | And the following "course enrolments" exist: | user | course | role | | teacher | TC100 | editingteacher | When I am on the "Course 1" "core_question > course question import" page logged in as "teacher" And I set the field "id_format_xml" to "1" And I upload "question/format/xml/tests/fixtures/multichoice.xml" file to "Import" filemanager And I press "Import" Then I should see "Parsing questions from import file." And I should see "Importing 1 questions from file" And I should see "What language is being spoken?"
  • 10.
    10 Good tests Test onething Make it obvious what is being tested Make it obvious what the expected behaviour is Are well organised Plugins -> Features -> Scenarios
  • 11.
  • 12.
    12 A bad example Adaptedfrom mod/openstudio/tests/behat/content_block_socical.feature Feature: Open Studio notifications In order to track activity on content I am interested in As a student I want recive notifications about my posts and comments Background: Given the following "users" exist: | username | firstname | lastname | email | | teacher1 | Teacher | 1 | [email protected] | | teacher2 | Teacher | 1 | [email protected] | | student1 | Student | 1 | [email protected] | | student2 | Student | 2 | [email protected] | And the following "courses" exist: | fullname | shortname | category | | Course 1 | C1 | 0 | And the following "course enrolments" exist: | user | course | role | | teacher1 | C1 | editingteacher | | teacher2 | C1 | editingteacher | | student1 | C1 | student | | student2 | C1 | student | And the following open studio "instances" exist: | course | name | description | grouping | groupmode | pinboard | idnumber | tutorroles | | C1 | Demo Open Studio | Notifification description | GI1 | 1 | 99 | OS1 | editingteacher | And all users have accepted the plagarism statement for "OS1" openstudio Given I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1" And I follow "Add new content" And I set the following fields to these values: | id_visibility_3 | 1 | | Title | Module post | | Description | Module post | And I press "Save" And I follow "My Content" And I follow "Add new content" And I set the following fields to these values: | id_visibility_3 | 1 | | Title | Module post 1 | | Description | Module post 1 | And I press "Save"
  • 13.
    13 A bad examplecontinued Adapted from mod/openstudio/tests/behat/content_block_socical.feature Scenario: Interactive emoticons in content block social When I am on the "Demo Open Studio" "openstudio activity" page logged in as "teacher1" And I wait until the page is ready # The emoticons should be the gray icon when user doesn't react it. Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//img[contains(@src, 'inspiration_grey_rgb_32px')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//img[contains(@src, 'participation_grey_rgb_32px')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//img[contains(@src, 'favourite_grey_rgb_32px')]" "xpath_element" should exist Then I click on "Module post 1" "link" And I wait until the page is ready And I click on "0 Favourites" "link" And I click on "0 Smiles" "link" And I click on "0 Inspired" "link" And I wait until the page is ready And I am on the "Demo Open Studio" "openstudio activity" page # The emoticons should be the blue icon when user reacts it. Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//img[contains(@src, 'inspiration_rgb_32px')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//img[contains(@src, 'participation_rgb_32px')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//img[contains(@src, 'favourite_rgb_32px')]" "xpath_element" should exist Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1" And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element" And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element" And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element" Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "teacher1" And I click on "Module post 1" "link" And I should see "2 Favourites" And I should see "2 Smiles" And I should see "2 Inspired"
  • 14.
  • 15.
    15 Irrelevant navigation Adapted frommod/quiz/tests/behat/editing_add.feature in Moodle 3.2 # ... And I log in as "teacher1" And I follow "Course 1" And I follow "Quiz 1" And I navigate to "Edit quiz" node in "Quiz administration“ # ... Slow Fragile Irrelevant
  • 16.
    16 Setting things upusing the UI Adapted from mod/openstudio/tests/behat/content_block_socical.feature Feature: Open Studio notifications In order to track activity on content I am interested in As a student I want recive notifications about my posts and comments Background: # ... Given I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1" And I follow "Add new content" And I set the following fields to these values: | id_visibility_3 | 1 | | Title | Module post | | Description | Module post | And I press "Save" And I follow "My Content" And I follow "Add new content" And I set the following fields to these values: | id_visibility_3 | 1 | | Title | Module post 1 | | Description | Module post 1 | And I press "Save“ # ... Slow Fragile Irrelevant
  • 17.
    17 Identifying things on-screenusing XPath or CSS Adapted from question/format/xml/tests/behat/import_export.feature Scenario: Interactive emoticons in content block social # ... # The emoticons should be the blue icon when user reacts it. Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//img[contains(@src, 'inspiration_rgb_32px')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//img[contains(@src, 'participation_rgb_32px')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element" should exist Then "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//img[contains(@src, 'favourite_rgb_32px')]" "xpath_element" should exist Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "student1" And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_5']//span[contains(., '1')]" "xpath_element" And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_4']//span[contains(., '1')]" "xpath_element" And I click on "//div[@class='openstudio-grid-item'][1]//span[@id='content_view_icon_2']//span[contains(., '1')]" "xpath_element" Then I am on the "Demo Open Studio" "openstudio activity" page logged in as "teacher1" # ... Fragile Unclear
  • 18.
    18 Long rambling tests question/type/pmatch/tests/behat/basic_test.feature Scenario:Create, edit then preview a pattern match question. When I am on the "Course 1" "core_question > course question bank" page logged in as teacher # Create a new question. And I add a "Pattern match" question filling the form with: | Question name | My first pattern match question | | Question text | Listen, translate and write | | id_usecase | Yes, case must match | | id_allowsubscript | Yes | | id_allowsuperscript | Yes | | id_forcelength | warn that answer is too long and invite respondee to shorten it | | id_applydictionarycheck | Do not check spelling of student | | id_sentencedividers | ?! | | id_converttospace | ;: | | id_synonymsdata_0_word | any | | id_synonymsdata_0_synonyms | "testing|one|two|three|four" | | Answer 1 | match (testing one two three four) | | id_fraction_0 | 100% | | id_feedback_0 | Well done! | | id_otherfeedback | Sorry, no. | | Hint 1 | Please try again. | | Hint 2 | Use a calculator if necessary. | Then I should see "My first pattern match question" # Checking that the next new question form displays user preferences settings. When I press "Create a new question ..." And I set the field "item_qtype_pmatch" to "1" And I click on "Add" "button" in the "Choose a question type to add" "dialogue" Then the following fields match these values: | id_usecase | Yes, case must match | | id_allowsubscript | Yes | | id_allowsuperscript | Yes | | id_forcelength | warn that answer is too long and invite respondee to shorten it | | id_applydictionarycheck | Do not check spelling of student | | id_sentencedividers | ?! | | id_converttospace | ;: | And I press "Cancel" # Preview it. Test correct and incorrect answers. And I am on the "My first pattern match question" "core_question > preview" page And I set the following fields to these values: | How questions behave | Interactive with multiple tries | | Marked out of | 3 | | Marks | Show mark and max | And I press "Start again with these options" Then I should see "Listen, translate and write" And the state of "Listen, translate and write" question is shown as "Tries remaining: 3" When I set the field "Answer:" to "testing" And I press "Check" Then I should see "Sorry, no." And I should see "Please try again." When I press "Try again" Then the state of "Listen, translate and write" question is shown as "Tries remaining: 2" When I set the field "Answer:" to "testing one two three four" And I press "Check" Then I should see "Well done!" Then the state of "Listen, translate and write" question is shown as "Correct" # Backup the course and restore it. When I log out And I log in as "admin" When I backup "Course 1" course using this options: | Confirmation | Filename | test_backup.mbz | When I restore "test_backup.mbz" backup into a new course using this options: | Schema | Course name | Course 2 | Then I should see "Course 2" When I navigate to "Question bank" in current page administration Then I should see "My first pattern match question" # Edit the copy and verify the form field contents. When I choose "Edit question" action for "My first pattern match question" in the question bank Then the following fields match these values: | Question name | My first pattern match question | | Question text | Listen, translate and write | | id_synonymsdata_0_word | any | | id_synonymsdata_0_synonyms | "testing|one|two|three|four" | | Answer 1 | match (testing one two three four) | | id_fraction_0 | 100% | | id_feedback_0 | Well done! | | id_otherfeedback | Sorry, no. | | Hint 1 | Please try again. | | Hint 2 | Use a calculator if necessary. | And I set the following fields to these values: | Question name | Edited question name | And I press "id_submitbutton" Then I should see "Edited question name" Huge time-waste if the failure is at the end Good Scenarios each test one thing - Then the failure message tells you what’s broken Slow Fragile
  • 19.
  • 20.
  • 21.
    21 Organise and namethings clearly Naming things is hard - but getting it right helps everyone – including future you! Organised
  • 22.
    22 Use Given, Whenand Then correctly Only test one thing per scenario Remember the four-phase test pattern local/monitor/tests/behat/local_monitor.feature @ou @ou_vle @local @local_monitor Feature: Check the monitor script, one of two that checks the server is working OK In order to monitor the server system As some back-end system component (supposing this script is still used, which we don't know) I need to have the monitor script run Scenario: Check monitor script Given the following "courses" exist: | fullname | shortname | format | | Course 1 | C1 | topics | And the following "activities" exist: | activity | name | intro | course | idnumber | | label | L1 | <a href="../local/monitor/index.php?display=1">MonLink</a> | C1 | label1 | When I log in as "admin" And I am on "Course 1" course homepage And I follow "MonLink" Then I should see "All tests completed successfully" Clear
  • 23.
    23 Important Given steps . Giventhe following "..." exist: “…” can be - Standard entities like “user”, “course”, “activity”, “course enrolments”, … - Full list in lib/behat/classes/behat_core_generator.php - You can extend this for your plugin Given the following config values are set as admin: | showuseridentity | email,profile_field_oucu | Fast Robust Extensible
  • 24.
    24 class behat_mod_quiz_generator extendsbehat_generator_base { protected function get_creatable_entities(): array { return [ 'group overrides' => [ 'singular' => 'group override', 'datagenerator' => 'override', 'required' => ['quiz', 'group'], 'switchids' => ['quiz' => 'quiz', 'group' => 'groupid'], ], 'user overrides' => [ 'singular' => 'user override', 'datagenerator' => 'override', 'required' => ['quiz', 'user'], 'switchids' => ['quiz' => 'quiz', 'user' => 'userid'], ], ]; } } Extending entity generation From mod/quiz/tests/generator/behat_mod_quiz_generator.php Given the following "mod_quiz > group overrides" exist: | quiz | group | attempts | | Test quiz | G1 | 2 |
  • 25.
    25 Important When steps WhenI am on the "Quiz 1" "mod_quiz > Grades report" page When I am on the "Intro" "page activity" page logged in as student When I follow "Edit profile" When I press "Save changes" When I set the field "Question text" to "Edited question text." . When I click on "Student" "link" in the "View as" "block" When I set the following fields to these values: | Question name | AV question | | Type of recording | Customised audio/video | Fast Clear Robust Extensible
  • 26.
    26 Extending navigation /** * Convertpage names to URLs for steps like 'When I am on the "[identifier]" "[page type]" page'. * * Recognised page names are: * | pagetype | name meaning | description | * | course question bank | Course name | The question bank for a course | * | preview | Question name | The screen to preview a question | * * @param string $type identifies which type of page this is, e.g. 'Preview'. * @param string $identifier identifies the particular page, e.g. 'My question'. * @return moodle_url the corresponding URL. * @throws Exception with a meaningful error message if the specified page cannot be found. */ protected function resolve_page_instance_url(string $type, string $identifier): moodle_url { switch (strtolower($type)) { case 'course question bank': return new moodle_url('/question/edit.php', ['courseid' => $this->get_course_id($identifier)]); case 'preview': [$questionid, $otheridtype, $otherid] = $this->find_question_by_name($identifier); return new moodle_url('/question/bank/previewquestion/preview.php', ['id' => $questionid, $otheridtype => $otherid]); default: throw new Exception('Unrecognised core_question page type "' . $type . '."'); } } Adapted from question/tests/behat/behat_core_question.php When I am on the "Course 1" "core_question > course question bank" page
  • 27.
    27 Important Then steps ThenI should see "Embedded question progress for Course 1" Then I should not see "Fill in the Blanks" Then "Page 3" "link" should exist Then "Equation editor" "button" should be visible Then the field "Describe this image" matches value "Awesome!" Then the following fields match these values: Then "Student 1" row "Username" column of "generaltable" table should contain "student1“ Fast Clear Robust
  • 28.
    28 More Then steps ThenI should see "Frog" in the "activity" "core_course > Activity chooser tab" Then I should see "Incorrect" in the "student2" "table_row" Then "Delete" "button" should not exist in the "Confirm" "dialogue" Then "Unread posts" "link" in the "Forum1" "list_item" should be visible - various things like link, button, list item, dialogue, block, region, … - full list in lib/behat/classes/partial_named_selector.php - can be extened for your plugin. - a kitten dies any time you use “css_element” or “xpath_element” Clear Robust Extensible
  • 29.
    29 Extending selectors From course/tests/behat/behat_course.php classbehat_course extends behat_base { public static function get_partial_named_selectors(): array { return [ new behat_component_named_selector( 'Activity chooser screen', [ "%core_course/activityChooser%//*[@data-region=%locator%][contains(concat(' ', @class, ' '), ' carousel-item ')]" ] ), new behat_component_named_selector( 'Activity chooser tab', [ "%core_course/activityChooser%//*[@data-region=%locator%][contains(concat(' ', @class, ' '), ' tab-pane ')]" ] ), ]; } } Clear Robust … in the "activity" "core_course > Activity chooser tab"
  • 30.
    30 /** * Put thespecified questions on the specified pages of a given quiz. * * The first row should be column names: * | question | page | maxmark | * * @param string $quizname the name of the quiz to add questions to. * @param TableNode $data information about the questions to add. * * @Given quiz :quizname contains the following questions: */ public function quiz_contains_the_following_questions($quizname, TableNode $data) { // ... Lots of complex code ... } Completely custom steps A great option when appropriate - Other extensibility makes it less necessary - Ensure the step text clearly belongs to your plugin Given quiz "Quiz 1" contains the following questions: | question | page | | TF1 | 1 | Fast Robust Example adapted from mod/quiz/tests/behat/behat_mod_quiz.php
  • 31.
  • 32.
    32 Summary Remember everything youalready know about testing Learn by doing and looking at examples (and docs) - but judge what you are looking at before you copy it Remember the good practices and avoid the bad ones Get support from other people - https://moodledev.io/general/channels#developer-chat There is lots you could learn, but you don’t need to learn it all at once - you can get a long way with just some basic steps - simple is good!