Setting examination paper
2 likes689 views
The document outlines a professional development session on crafting semester examination papers at Paro College of Education, Bhutan, held on May 11, 2016, led by Dr. Gembo Tshering. It detailed activities focused on Bloom's Taxonomy, emphasizing knowledge and cognitive processes, various item formats for assessments, and guidelines for examining and reducing errors in test scores. Additionally, it included instructions for guiding examinees during tests and the appearance and validity aspects of semester examination papers.
Setting examination paper
- 1. Paro College of Education Royal University of Bhutan Paro Professional Development: Setting Semester Examination Paper 11 May 2016 Facilitator: Dr Gembo Tshering Controller of Examination
- 2. Overview Date Session Activity Time 11May2016 1st Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Knowledge Dimension 8.40 AM-9.40 AM 2nd Knowledge Dimension Grid 9.40 AM-10.10 AM Tea Session 10.10 AM-10.25 AM 3rd Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension 10.25 AM-11.25 AM 4th Cognitive Process Dimension Grid 11.25 AM-11.45 AM 5th Item Formats 11.45 AM-12.45PM Official Lunch 12.45 PM-1.45 PM 6th Guiding Examinees during the Test 1.45 PM-2.15 PM 7th Appearance of a Semester Exam Paper 2.15 PM-2.30 PM 8th Reducing Mistakes in Test Scores 2.30 PM-2.50 PM 9th Debriefing 2.50 PM-3.00 PM 5/10/2016 2Dr Gembo Tshering/email: [email protected]
- 3. SESSION ONE Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Knowledge Dimension 5/10/2016 Dr Gembo Tshering 3
- 4. An outline of the Bloom’s Taxonomy of Educational Objectives Knowledge Dimension Cognitive Process Dimension Remerging Comprehending Applying Analyzing Evaluating Creating Factual Conceptual Procedural Meta- cognitive 5/10/2016 Dr Gembo Tshering 4
- 5. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Knowledge Dimension • Factual Knowledge – Knowledge of terminology (alphabet, scientific terms) – Knowledge of specific details and elements (significant names, cultural facts) Definition: The basic elements students must know to be acquainted with a discipline or solve problems in it. 5/10/2016 5Dr Gembo Tshering
- 6. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Knowledge Dimension • Conceptual knowledge – Knowledge of classifications and categories (plants, library) – Knowledge of principles and generalizations (Pythagorean theorem, Four Noble Truths) – Knowledge of theories, models, and structures (boiling point, parliament) Definition: The interrelationships among the basic elements within a larger structure that enable them to function together. 5/10/2016 6Dr Gembo Tshering
- 7. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Knowledge Dimension • Procedural Knowledge – Knowledge of subject-specific skills and algorithms (division algorithms, cooking) – Knowledge of subject-specific techniques and methods (interviewing techniques, research methods) – Knowledge of criteria for determining when to use appropriate procedures (four wheel driving, emergency bell) Definition: How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods. 5/10/2016 7Dr Gembo Tshering
- 8. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Knowledge Dimension • Metacognitive Knowledge – Strategic knowledge (rehearsal, mnemonic strategy) – Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge (testing knowledge, cognitive demands) – Self-knowledge (personal strengths, personal weakness) Definition: Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition. 5/10/2016 8Dr Gembo Tshering
- 9. SESSION TWO Knowledge Dimension Grid
- 10. Knowledge and cognitive process dimensions Subject Matter Knowledge Dimension Factual Conceptual Procedural Meta-cognitive Median X Punctuation rules X Calculating median X Cultural values X 5/10/2016 10Dr Gembo Tshering
- 11. SESSION THREE Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension
- 12. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension • Remember – Recognizing – Recalling Definition: Retrieve relevant knowledge from long-term memory. 5/10/2016 12Dr Gembo Tshering
- 13. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension • Understand – Interpreting – Exemplifying – Classifying – Summarizing – Inferring – Comparing – Explaining Definition: Construct meaning from instructional messages, including oral, written, and graphic communication. 5/10/2016 13Dr Gembo Tshering
- 14. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension • Apply – Executing – Implementing Definition: Carry out or use a procedure in a given situation. 5/10/2016 14Dr Gembo Tshering
- 15. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension • Analyze – Differentiating – Organizing – Attributing Definition: Break material into constituent parts and determine how parts relate to one another and to overall structure or purpose. 5/10/2016 15Dr Gembo Tshering
- 16. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension • Evaluate – Checking – Critiquing Definition: Make judgments based on criteria and standards. 5/10/2016 16Dr Gembo Tshering
- 17. Harvesting the Power of Bloom’s Taxonomy of Educational Objectives: Cognitive Process Dimension • Create – Generating – Planning – Producing Definition: Put elements together to form a coherent or functional whole, reorganize elements into a new pattern or structures. 5/10/2016 17Dr Gembo Tshering
- 18. SESSION FOUR Cognitive Process Dimension Grid
- 19. Sample Test Blueprint Content Cognitive Process Dimension Rem Und App Ana Eva Cre Tot Item/Marks Punctuation Q1, e, 5M; Q5,ii,4M; 2/9 Drama Q2, iii, 3M; 1/3 Tot 0 0 3 0 0 0 3/12 5/10/2016 19Dr Gembo Tshering Question Number Sub question Marks
- 21. Item Formats Two Types of Items • Selected-Response Items • Constructed-Response Items We will focus on some selected-response items 5/10/2016 21Dr Gembo Tshering
- 22. Item Formats • Multiple Choice Item Format • Matching Item Format • Extended Matching Item Format • Alternate Choice Item Format • True-False Item Format • Complex Item Format • Context-Dependent Item Format 5/10/2016 22Dr Gembo Tshering
- 23. Item Formats: MC Item Format A conventional MC item will have a stem, the correct choice, and several wrong answers or distractors. Example: Who is Jigme Y. Thinley? A. The first democratically elected Prime Minister of Bhutan B. The GNH ambassador of Bhutan to the world C. The best public speaker in the Bhutanese parliament D. The last Prime Minister of Bhutan before 2008 5/10/2016 23Dr Gembo Tshering
- 24. Item Formats: MC Item Writing Rules • The question or problem in the stem must be self- contained. • Avoid negatively stated stems. • Each alternative must be grammatically consistent with the item stem. • Make all alternatives plausible, but be sure that one of them is indisputably the correct or the best answer. • Randomly use all answer positions in approximately equal numbers. • Never use all of the above as an answer choice, but use none of the above to make the items more demanding. 5/10/2016 24Dr Gembo Tshering
- 25. Aligning Item Formats: Matching MC Item A Matching MC item format starts with a set of options at the top followed by a set of stems below. The instruction that goes before the options and stems tell test takers how to respond and where to mark answers. Mark your answer on the sheet. For each item select the correct answer from options provided below. A. Trashigang B. Thimphu C. Chukha D. Punakha 1. The capital of Bhutan 2. The largest Dzongkhag 3. The winter residence of the Je Khenpo 4. The mega hydro power 5. The winter capital Options/Premises Stems/Responses 5/10/2016 25Dr Gembo Tshering
- 26. Item Formats: Matching MC Item-Writing Rules • Use fairly brief lists, placing the shorter entries on the right. • Employ homogeneous lists (options and stems should comprise similar sort of things). • Include more responses than premises. • List responses in a logical order (chronologically or alphabetically). • Describe the basis for matching and a number of times responses can be used. • Try to place any premises and responses for any matching item on a single page. 5/10/2016 26Dr Gembo Tshering
- 27. Item Formats: Context-Dependent Item sets A Context-Dependent Item Set has an introductory stimulus and usually two to twelve test items related to this stimulus (suitable for testing reading comprehension, stories, poems). Bhutan became monarchy in 1907. Since then, Bhutan has been developing steadily. When did Bhutan become constitutional democracy? A. 2007 B. 2008 C. 2012 D. 2013 Which one of the following is not a part of the democratic process in Bhutan? A. Formation of GYT B. Formation of DYT C. Formation of constitutional offices D. Formation of ministries Note: All MC Item-Writing Rules are applicable to Context-Dependent Item Sets 5/10/2016 27Dr Gembo Tshering
- 28. Guess What is Correct! I am in school. The underlined sentence is an example of dot dot dot or colon or semi-colon or blank A. A/a question full stop or colon or blank B. A/a statement full stop or colon or blank C. A/an exclamation full stop or colon or blank D. A/an instruction full stop or colon or blank 5/10/2016 28Dr Gembo Tshering
- 29. The Correct One Revealed! I am at the school. The underlined sentence is an example of A. a question. B. a statement. C. an exclamation. D. an instruction. 5/10/2016 29Dr Gembo Tshering
- 30. SESSION SIX Guiding Examinees during the Test
- 31. Guiding Examinees during the Test: Instruction at the Test Level (First Page) 5/10/2016 31Dr Gembo Tshering Spring Semester Examination 2016 Paro College of Education Royal University of Bhutan Paro Module: MAT408 (Descriptive Statistics) Program: B.Ed (S) Level: I Writing Time: Three Hours Full Marks: 100 Read the following directions carefully: 1. Do not write during the first 15 minutes. Use this time for reading the questions. You will get full three hours for answering the questions; 2. Write the answers to all the questions in the answer sheets provided by the college. 3. Read the directions to each section and to each question carefully before answering the questions; 4. Once the writing time begins, you are not allowed to ask questions, speak with others, or move around; 5. Do not leave the examination hall before you are certain that all the questions, as directed in the paper, have been answered.
- 32. Guiding Examinees during the Test: Instruction at the Section Level SECTION A 25 Questions-50 Marks Answer ALL Questions Direction: Each question in this section is followed by four possible choices of answers. Choose the most correct answer and write it down in the answer sheets provided. SECTION B FIVE Questions-25 Marks Answer ALL Questions Direction: Answer the questions given below as directed. All answers must be written in the answer sheets provided. The intended marks for each question is given in brackets. SECTION C FIVE Question-25 Marks Directions: From the FIVE questions, choose any three and write their answers as directed in the answer sheets provided. 5/10/2016 32Dr Gembo Tshering
- 33. For MC item: Each question in this section is followed by four possible choices of answers. Choose the most correct answer and write it down in the answer sheets. For Matching item: Match the following options to their correct responses and write the pairs in terms of their alphabetical and numerical representations in the answer sheets (eg., A=3) For Context-dependent Item Sets: Read the story given below and answer the questions that follow as directed. Read the passage given below and answer the questions that follow as directed. Read the abstract given below and answer the questions that follow as directed. Guiding Examinees during the Test: Instruction at the Item Level 5/10/2016 33Dr Gembo Tshering
- 34. SESSION SEVEN Appearance of a Semester Exam Paper
- 35. Face Validity of a Semester Examination Paper Font Size: Section: 14 (Capital, Times New Romans) Others: 12 (Times New Romans) Page Margins Left: 1 “; Right: 1 “; Top: 1”; Bottom: 1“ First Page Test Level Instruction (Semester Name, Module Code, Module Name, Program Name, Writing Time, Full Marks, Number of Sections, Choice Questions, etc.,) 5/10/2016 35Dr Gembo Tshering
- 36. Face Validity of Semester Examination Paper Second Page Section and Item Level instructions and Items (Note: show item level marks inside brackets and right-align the brackets) Item Numbering First level should be counting number (1,2,3…) Second level should be lower case alphabet letter (a,b,c…) Third level should be Roman number (i, ii,…) 5/10/2016 36Dr Gembo Tshering
- 37. Face Validity of Semester Examination Paper Footer Section a)Left corner will have ©PCE b)Centre will have Spring Exam 2016 c) Right corner will have Page X of Y d)The footer content will have top border line Example as given below: © PCE Spring Exam 2016 Page 1 of 12 5/10/2016 37Dr Gembo Tshering d) a) b) c)
- 38. Item Numbering Use continuous counting numbers to number questions across sections SECTION A Question 1 Question 2 SECTION B Question 3 SECTION C Question 4 Limit item levels (sub division) to three levels Question 1 a) i… ii… iii… iv… b) i… 5/10/2016 38Dr Gembo Tshering 1st level 2nd level 3rd level
- 39. SESSION EIGHT Reducing Mistakes in Test Scores
- 40. Reducing Mistakes in Test Scores: Marking Scheme and Model Answer Question No. Question and Answer Marks 1 What is Newton’s Third Law of Motion? 1 Ans: Every action has equal and opposite reaction. 1 2 Which one of the following is the correct number of faculty members of Paro College of Education? A. 60 B. 65 C. 55 D. 70 1 Ans: A 1 3 Write a letter to your father asking him about your favorite pet. 3 Ans: Format Grammar Vocabulary 1 1 1 5/10/2016 40Dr Gembo Tshering
- 41. SESSION NINE Debrefing 5/10/2016 Dr Gembo Tshering 41
- 42. Bibliography 5/10/2016 Dr Gembo Tshering 42 Aiken, R. L & Groth-Marnat, G. (2006). Psychological testing and assessment (12th ed). New York: Pearson Education, Inc. Anderson, W. L.; Krathwohl, R. D.; Airasian, W. P. et al. (Eds). (2001). A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives. New York: Addison Wesley Longman, Inc. Gronlund, E. N & Waugh K. C. (2009). Assessment of Student Achievement (9th ed). Upper Saddle River, New Jersey: Pearson Education, Inc. Haladyna, M. T. (2004). Developing and validating multiple-choice test items (3rd ed). Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc.
- 43. THANK YOU AND HAPPY SPRING SEMESTER EXAMINATION 5/10/2016 43Dr Gembo Tshering