Test Maker
- 1. 1 Test Maker homepage Language selector
- 2. 2 Create a newtest
- 3. 3 Open existing test:two ways • links to the last opened tests • List of all tests (sortable list)
- 4. 4 Currently opened test:quick info Note all the new tabs which appear after test opening
- 5. 5 Design : the3 panels 1. Object selection: list of object types you can add to the test • Questions (8 different types of question) • Design items • Questions from existing tests or library sections 2. Test tree: the current hierarchical list of objects already in the test 3. Object designer: the designer tool of an object Currently selected item in the tree
- 6. 6 Design : Objectselection 1. New object selection panel (left) A simple click on an object inserts an instance of such an item in the Test tree, just after the currently selected item or at the end of the tree if there is no selection. Currently selected item in the tree. New object inserted after.
- 7. 7 Design : Testtree 3. Test tree panel (right) Allows to remove items from the test and reordering of the items in the tree.
- 8. 8 Design : Objectdesigner 2. Object designer panel (middle) Tool to edit properties and functionnalities of the object. Each type of question has its own designer. Note the three tabs: • Design view : designer • Student view : a raw representation of what the student will see • Answer view : the correct answer
- 9. 9 Design : Objectdesigner Design view : designer Student view : a raw representation of what the student will see. Notice the numbering, disposition for example Answer view : the correct answers
- 10. 10 Question : MultipleChoices (MC) Points: maximum score obtained when answer(s) is (are) correct Difficulty: arbitrary question difficulty indicator Numbering: numbering type displayed before the option buttons Disposition: vertical, horizontal, dropdown Randomize: if checked, the proposed options are shuffled Hints: Hint to display for the question (see option « Show Hints » in the « Presentation attributes ») Feedbacks: displayed after submission (see option « Show Feedbacks » in the « Presentation attributes » ) Question text: don’t forget it ! Image: to illustrate the question
- 11. 11 Question : MultipleChoices (MC) (following) Add as many options as needed and select the correct one
- 12. 12 Question : Multi-Select(MS) Scoring: two ways of scoring the answers 1.All or nothing: the points are attributed if all and only the correct options are checked by the student, else the student gets 0 point. 2.Weighted: each correctly checked option receives partial credit; an equal measure of partial credit is deducted for each incorrect answer. (partial credit = Points / #C where #C is the total number of expected correct answers.)
- 13. 13 Question : Multi-Select(MS) (following) Add as many options as needed and select the correct ones
- 14. 14 Question : True/False(TF) Enter the text displayed for both option and select the correct one.
- 15. 15 Question : Ordering(ORD) Scoring: two ways of scoring the answers 1.All or nothing: the points are attributed if all the items are correctly sorted, else the students receives 0 point 2.Weighted: for each item in correct position, the student receives partial credit; (partial credit of item i = Points * Wi / Wtotal where Wi is the weight of item i and Wtotal is the sum of the weight of all items)
- 16. 16 Question : Ordering(ORD) (following) Add as many items as needed Specify the correct position of each item Eventually, set a relative weight attributed to the item (default = 1) Test interface for Ordering question
- 17. 17 Question : Matching(MAT) Scoring: two ways of scoring the answers 1.All or nothing: the points are attributed if all the items are correctly matched, else the students receives 0 point 2.Weighted: for each correct matching, the student receives partial credit; (partial credit of item i = Points * Wi / Wtotal where Wi is the weight of item i and Wtotal is the sum of the weight of all items)
- 18. 18 Question : Matching(MAT) (following) Add as many matching pairs as needed Eventually, assign a relative weight to the matching pair (default = 1) In order to increase difficulty, you can add so called « distractors », which are non associated items (all distractors must be added in the same list) Test interface for Matching question
- 19. 19 Question : FreeText (FT) A Free text question requires a manual grading. Instructor notes: information displayed to the instructor during the manual grading process.
- 20. 20 Question : ShortAnswers (SA) # required answers: number of expected answers Scoring: two ways of scoring the answers 1.All or nothing: the points are attributed if all the expected answers are entered, else the student receives 0 point. 2.Weighted: for each correct answer entered, the student receives partial credit; (partial credit = Points / #C where #C is the total number of expected answers)
- 21. 21 Question : ShortAnswers (SA) (following) Add as many correct answers as needed (at least # required answers) Indicate if answer is case sensitive (default no) Enter eventual alternate (variant) answers with their relative weight)
- 22. 22 Question : FillIn the Blanks (FIB) Scoring: two ways of scoring the answers 1.All or nothing: the points are attributed if all the answers are entered, else the student receives 0 point. 2.Weighted: for each correct answer entered, the student receives partial credit; (partial credit of answer i = Points * Wi / Wtotal where Wi is the weight of answer i and Wtotal is the sum of the weight of all answers)
- 23. 23 Question : FillIn the Blanks (FIB) (following) Add as many text blocks and blank bocks as needed. Set a relative weight for the answer (default = 1) Indicate if answer is case sensitive (default no) Enter eventual alternate (variant) answers with their relative weight) Test interface for Fill in the blanks question
- 24. 24 Design item: Textblock Allows to add additional text and images Test interface
- 25. 25 Summary: quick viewon the test items Jump to the object designer
- 26. 26 Answers sheet: a« paper » view on the test and answers PDF export
- 27. 27 Preview: See whatthe student will get From this screen, start a test attempt in preview mode. You may ask to track this « Preview » attempt in order to see it in tracking and statistics reports. Note : As soon as attempts are tracked, modifications to the questions in the test are restricted to minor changes. But« Preview » attempts can be deleted (see following point) in order to be able to update the test. Tracked « Preview » attempts can be deleted.
- 28. 28 Preview: what thestudent will get
- 29.
- 30. 30 Settings: default attributesof the test used during « publication »
- 31. 31 Settings: General Name: usedto easily identify the test inside Test Manager. Also used during publication Description: additional description for internal use Starting page: design of the starting page of the test
- 32. 32 Settings: Presentation Questions perpage: best practice : 1 per page (0 means all the questions on the same page) Order of questions: Preset : order set at design time Show submit button on each page: or only when last page has been reached Show Hints: (see next slide) Show Feedbacks: feedbacks will be displayed after submission if «Show corrections» is checked (see Submission tab)
- 33. 33 Settings: Grading Grading isthe process of giving an overall grade (quotation) to the test after its submission. It’s based on the scores obtained for each question and uses one of the 4 grading schemes available. Grading can be automatic or manual. Grading at submit: the test is automatically graded after the submission. If not checked, grading must be done manually (for example if some questions require manual scoring, test is best graded manually) Grading scheme: select the grading scheme to apply and the grade/value/percent to pass, depending on the scheme
- 34. 34 Settings: Grading (following) Numeric:numeric scale (min and max values); value to pass. The global score of the test is transformed according to the scale to determine the pass status. Pass/Fail: names of the two grades can be changed; percentage to pass Letters A-E: limits can be changed; grade to pass Custom: like previous but with custom defined grades (maximum ten); grade to pass
- 35. 35 Settings: Attempts Attempts allowed:Maximun number of attempt allowed by the student; default = 1 (note: CC publications allow only one attempt by Test event. LP publications allow multiple attempts by Trainig Plan but only with overal grading « Best attempt » ) Delay before new attempt: in day(s), or hour(s) Overall grading: if more than 1 attempt allowed, how to compute the overall grading. Warning!: all methods are not necessary available for some publication types since they could lower an already attributed grade.
- 36. 36 Settings: Restrictions Backward allowed:backward navigation allowed or not in the test Maximum Time: maximum time (minutes) allowed to complete the test. Student is warned when maximum time is elapsed. Bonus Time: when maximum time is elapsed, the bonus time allows the student to terminate and submit the test before definitive end. Late submission: fixes the way to handle the late submission (after Max + Bonus times). - Allow normal submission: student may submit the test normally ; it’s only flagged as «late» - Auto submission: the test is automatically submitted, terminated and flagged as «late» - Enforce limit: the test is automatically terminated, not submitted Start and End date & time: not applicable in CC publications
- 37. 37 Settings: Submission What isshown to the student after Test submission
- 38. 38 Publication To allow studentsto enter a Test, it must be published. A Publication defines settings for the particular environment where the test will run. Those attributes are inherited from the Test settings and can be overwritten. Moreover, during publication some actions are automatically done to prepare the specific environment (for example Test Categories are created or updated in CC environment or Catalog entries performed in LP environment)
- 39.
- 40. 40 Publication: CC environment Atest event can now be created with this category The publication triggered the creation of a Test Category in CC environment
- 41. 41 Publication: LP environment Onlyone LP publication can be created for a test Specify the language for which the corresponding LP courses must be generated and set their names . No new LP publication can be created for the test
- 42. 42 Publication: LP environment Thepublication triggered the creation of AICC courses in the LP catalog
- 43. 43 Tracking: all attempts,even not submitted
- 44. 44 Score distribution Score distribution:number of students aggregated by score (in %) for submitted attempts
- 45. 45 Question results Question results: how the students answered each question (for submitted attempts)
- 46. 46 Score answers &Grade attempt Grade: Screen to select Test attempts. Displays the most important informations about grading Jump to modify score of questions in an attempt
- 47. 47 Score answers &Grade attempt Typically, a Free Text question requires manual scoring. The « Notes for instructor » appears here to help to score the answer. Updating the score of an answer automatically (re)grades the attempt.
- 48. 48 Score answers &Grade attempt For a publication with « Grade at submit » attribute not checked, the icon « Grade attempt » appears for an attempt after submission. The attempt can be graded by updating the score of any question in it (see previous slide) or, without modying the scores, by clicking this icon.
- 49. 49 Score answers &Grade attempt For a CC publication : the « blue » line displays the grade and status stored in CC environment. For publication with « Grade at submit » attribute checked, the grade and status are automatically sent to CC environment when test is submitted. If an attempt is (re)graded manually, the new/updated grade and status must/can be sent to CC environment here.
- 50. 50 Score answers &Grade attempt For a LP publication : the « blue » line displays the score and status stored in LP environment. For publication with « Grade at submit » attribute checked, the grade and status are automatically sent to LP environment when test is submitted. If an attempt is (re)graded manually, the new/updated score and status must/can be sent to LP environment here.
- 51. 51 Delete & Copytest A test can be deleted, with all its questions, if there is no result collected for it (including tracked « preview » attempts) A test can be copied, with all its questions, to a create a new one. All questions are duplicated and can be maintained independently.
- 52. 52 Questions Library Test Makersallows to create a repository (library) of questions, organized in sections. A library section can be used as a random section in a test. A fixed number of questions will be randomly picked from the library section at runtime. A section already used as a random section in a test cannot be deleted anymore. Questions of the library can also be individually copied in a test. Click here to add a new section in the library. Give it a name and populate it with questions
- 53. 53 Questions Library (following) Questionsof the library section already used in attempts cannot be deleted anymore Questions of a library section can be « deactivated ». They will not be used anymore in runtime. This is useful when a question has already been used in attempts but is now obsolete.
- 54. 54 Random section Click hereto add a random section in a test. •Give it a name •Select the Library section to attach •Set the number of questions to pick up randomly at runtime •Set the points attributed to each question
- 55. 55 Random section (following) Summaryof a test with a random section Answers sheet of a test with a random section
- 56. 56 Update of atest: restrictions As soon as results have been collected for a test, some updates are not possible any more
- 57. 57 Update of atest: restrictions (following) Any update which does not affect the structure and scoring of the questions. • Update of the question texts • Update of the option texts in MC,MS,T/F questions (but no option can be added or removed) • Update of the item texts in ORD questions (but no item can be added or removed) • Update of the choice and match texts in MAT questions (but no item can be added or removed and no distractors can be added or removed) • Updates of the answer texts in SA and FIB questions (but no answer can be added or removed) • Display aspects (disposition, numbering, randomize) •… Any update which affects the structure and scoring of the questions. • Adding or removing question • Adding or removing options, possible answers, items, … depending on the question type • Changing the type of a question • Changing the points attributed to a question • Changing the eventual scoring mode (“weighted” or “All or nothing”) of a question • Changing the “correct” options, the correct order, the correct matching, … depending on the question type •…
- 58. 58 Update of atest: restrictions (following) Best practice 1.Creating a pilot version 2.Test the pilot version with tracked/or not tracked “Preview” attempts inside Test Maker 3.Tracked preview attempts can always be deleted. The test is then fully updatable once again. 4.“Preview” attempts outside Test Maker, by mailing to a Pilot target group. 5.Tracked preview attempts can always be deleted. The test is then fully updatable once again. 6. Repeat previous steps until all appears to be good. 7.Publish the Test (V1) 8.Even after real results have been collected in LP or CC environment, restricted updates are still possible as seen previously. 9.If major structural changes become necessary, create a new version of the Test, starting eventually from a copy of the obsolete one.