Downloaded 19 times
![Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative Assessment Denise Whitelock The Open University, Milton Keynes [email_address] DMW 2008 The Open University's Institute for Educational Technology](https://image.slidesharecdn.com/potsdam-conference-august-2008dmw-1218636397141826-8/75/Investigating-the-Pedagogical-Push-and-Technological-Pull-of-Computer-Assisted-Formative-Assessment-1-2048.jpg)
![DMW Learning for Life Workshop March 2008 www.storiesabout.com www.storiesabout.com/creativepdp [email_address]](https://image.slidesharecdn.com/potsdam-conference-august-2008dmw-1218636397141826-8/75/Investigating-the-Pedagogical-Push-and-Technological-Pull-of-Computer-Assisted-Formative-Assessment-4-2048.jpg)
Uploaded byDenise Whitelock
Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative Assessment
The document discusses computer-assisted formative assessment and the development of automated feedback tools. It describes stages of analysis a computer could use to analyze students' free-text responses and provide feedback, including detecting errors, revealing omissions, requesting clarification or further analysis, and checking causality. The feedback is aimed at helping students develop their understanding rather than just finding right or wrong answers. The tools are meant to support open-ended, divergent assessment in arts subjects.
More Related Content
Similar to Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative Assessment
Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative Assessment
- 1. Investigating the PedagogicalPush and Technological Pull of Computer Assisted Formative Assessment Denise Whitelock The Open University, Milton Keynes [email_address] DMW 2008 The Open University's Institute for Educational Technology
- 2. Behaviourist prescriptions forassessment: wolves in sheep's clothing DMW Learning for Life Workshop March 2008
- 3. Don’t forget thehuman reaction DMW Learning for Life Workshop March 2008
- 4. DMW Learning forLife Workshop March 2008 www.storiesabout.com www.storiesabout.com/creativepdp [email_address]
- 5. Formative Assessment Black& Wiliam (1998) improvement 0.4 to 0.7 on standardised tests MacFarlane-Dick (2006) empower students as self regulated learners Feedback and assessment – moving forward DMW 2007 The Open University's Institute for Educational Technology
- 6. Collaborative Formative AssessmentCSCL …. on task talk Predict Look and Explain Collaborative problem solving find a complex problem Tools to support distance students to communicate, find a partner Appropriate simulation DMW 2007 The Open University's Institute for Educational Technology
- 7. BuddySpace Automatic mapconstruction to show locations Automatic roster (‘buddy list’) construction using OU login authentication… so it knows which workgroup or tutorial group I’m in Many views: map, timelines, etc. Plug-ins for future enhancements DMW 2007 The Open University's Institute for Educational Technology
- 8. DMW 2007 TheOpen University's Institute for Educational Technology BuddySpace
- 9. DMW 2007 TheOpen University's Institute for Educational Technology “ State of mind” goals, plans, intentions devices user profiling
- 10. DMW 2007 TheOpen University's Institute for Educational Technology Automatic Map Generation Smart inset chosen, depending on actual data
- 11. DMW 2007 TheOpen University's Institute for Educational Technology Automatic Map Generation Right-click to ‘drill down’into ‘cluster’ nodes (provides scaleability)
- 12. BuddyFinder DMW 2007The Open University's Institute for Educational Technology Who else is facing a similar problem? Where can I find complementary approaches? Has anyone found contrary evidence? Generally: “Find me the person who can really help me now .” More specifically:
- 13. DMW 2007 TheOpen University's Institute for Educational Technology Arbitrary keywords at user’s discretion BuddyFinder
- 14. DMW 2007 TheOpen University's Institute for Educational Technology Easy search
- 15. DMW 2007 TheOpen University's Institute for Educational Technology Matches ‘ light up’
- 16. DMW 2007 TheOpen University's Institute for Educational Technology Invokes Google’s Web Service behind the scenes, if requested, and auto-highlights within user’s own map! BuddyFinder
- 17. Finding an appropriatescience application Complex problem solving with Global Warming 7 variables for manipulation Solar constant, CO 2 , H 2 O, Aerosol content, albedo, ice & snow, cloud cover Immediate effects of manipulation displayed as graphical output DMW 2007 The Open University's Institute for Educational Technology
- 18. Global Warming DMW2007 The Open University's Institute for Educational Technology
- 19. Global Warming DMW2007 The Open University's Institute for Educational Technology
- 20. Next: ‘Yoked’ appsvia BuddySpace DMW 2007 The Open University's Institute for Educational Technology Student A Student B (‘yoked’, but without full screen sharing required!)
- 21. Global Warming: SimlinkPresentation DMW 2007 The Open University's Institute for Educational Technology
- 22. DMW 2008 TheOpen University's Institute for Educational Technology
- 23. Debate with SIMLINKMore questioning Explanation became more important Socio emotive communication more evident Some couples separated for a while and then came back together DMW 2007 The Open University's Institute for Educational Technology
- 24. What is OpenComment? Automated formative assessment tool Free text entry for students Automated feedback and guidance Open questions, divergent assessment For use by Arts Faculty DMW 2008 The Open University's Institute for Educational Technology
- 25. Causal models ofexplanation First step: Identification of question types where students exhibit causal reasoning Looked for questions with: Salient variables Description of these variables Identification of trends Identification of relationship between the variables i.e. causality DMW 2008 The Open University's Institute for Educational Technology
- 26. Three common typesof question types were selected for Open Comment in History Analysis of statistics, usually presented to the student as a table Comprehension of a set text Identifying similarities and differences for a given event DMW 2008 The Open University's Institute for Educational Technology
- 27. Knowledge elicitation/ethnography DWand SW acted as students Answered all three question types C grades Improved only when an “answer model” elicited DMW 2008 The Open University's Institute for Educational Technology
- 28.
- 29.
- 30.
- 31.
- 32. Open Comment componentsA Java-based feedback system A web service shell A graphical interface for testing A Moodle-based question type A forms-based editing tool DMW 2008 The Open University's Institute for Educational Technology
- 33. Pedagogical models: checkingthem out with you now! List the activities that tutors engage with when they are marking a script. What things are they looking for e.g. error detection. Categorise the activities into a scheme or stages from which you could build instructions for a robot to mark the scripts. DMW, IET, Managing Assessment SIG, June 2008
- 34. Stages of analysisby computer of students’ free text entry for Open Comment: advice with respect to content (socio-emotional support stylised example) STAGE 1a: DETECT ERRORS E.g. Incorrect dates, facts. (Incorrect inferences and causality is dealt with below) Instead of concentrating on X, think about Y in order to answer this question Recognise effort (Dweck) and encourage to have another go You have done well to start answering this question but perhaps you misunderstood it. Instead of thinking about X which did not…….. Consider Y DMW 2008 The Open University's Institute for Educational Technology
- 35. Computer analysis continuedSTAGE 2a: REVEAL FIRST OMISSION Consider the role of Z in your answer Praise what is correct and point out what is missing Good but now consider the role X plays in your answer STAGE 2b: REVEAL SECOND OMISSION Consider the role of P in your answer Praise what is correct and point out what is missing Yes but also consider P. Would it have produced the same result if P is neglected? DMW 2008 The Open University's Institute for Educational Technology
- 36. Final stages ofanalysis STAGE 3:REQUEST CLARIFICATION OF KEY POINT 1 STAGE 4:REQUEST FURTHER ANALYSIS OF KEY POINT 1 (Stages 3 and 4 repeated with all the key points) STAGE 5:REQUEST THE INFERENCE FROM THE ANALYSIS OF KEY POINT 1 IF IT IS MISSING STAGE 6:REQUEST THE INFERENCE FROM THE ANALYSIS OF KEY POINT 1 IF IT IS NOT COMPLETE STAGE 7:CHECK THE CAUSALITY STAGE 8:REQUEST ALL THE CAUSAL FACTORS ARE WEIGHTED DMW 2008 The Open University's Institute for Educational Technology
- 37. Feedback 5 maincategories Praise Developing/expanding Presentation and structure essay Misunderstanding Advice – concept linking, analysis DMW 2008 The Open University's Institute for Educational Technology
- 38. What have welearnt? Mixed Methodology Analysis tutor feedback of 200 plus TMAs Ethnographic study Rules of the Game Moving students into the relational zone of proximal development ( Hasse 2001) Knowing what to do helps you play with ideas i.e. Learn All applications based on pedagogical PUSH Open Source Tools New question type for Moodle DMW 2008 The Open University's Institute for Educational Technology
- 39. No longer learningabout X but reflecting about X Students are able to find facts similar to X Know how X might be disputed Are able to make predictions about X Know how to use X in an argument Know how far X can be pushed Supported with tools and strategies DMW 2008 The Open University's Institute for Educational Technology