Downloaded 14 times
![Try It!
• http://adapt2.sis.pitt.edu/kt/
• Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009)
Addictive links: The motivational value of adaptive link annotation.
New Review of Hypermedia and Multimedia 15 (1), 97-118.
• Hsiao, I.-H., Sosnovsky, S., and Brusilovsky, P. (2010) Guiding
students to the right questions: adaptive navigation support in an E-
Learning system for Java programming. Journal of Computer Assisted
Learning 26 (4), 270-283.
• Hsiao, I.-H., Bakalov, F., Brusilovsky, P., and König-Ries, B.
(2013) Progressor: social navigation support through open social
student modeling. New Review of Hypermedia and Multimedia [PDF]
Read About It!](https://image.slidesharecdn.com/addictivelinkswww2014-140710055105-phpapp01/75/Addictive-links-Keynote-talk-at-WWW-2014-workshop-51-2048.jpg)
Uploaded byPeter Brusilovsky
Addictive links, Keynote talk at WWW 2014 workshop
The document discusses the challenges of engaging students in MOOCs despite access to quality content and faculty, highlighting the low usage and success rates of tools like Quizpack. It introduces adaptive navigation support techniques, namely Quizguide, which significantly improve student engagement and performance by tailoring content access based on user models. The findings suggest that combining adaptive and social navigation may enhance motivation and learning outcomes in various educational contexts.
More Related Content
Similar to Addictive links, Keynote talk at WWW 2014 workshop
Addictive links, Keynote talk at WWW 2014 workshop
- 1. Addictive Links: Engaging Studentsthrough Adaptive Navigation Support and Open Social Student Modeling Peter Brusilovsky with: Sergey Sosnovsky, Michael Yudelson, Sharon Hsiao School of Information Sciences, University of Pittsburgh
- 2. MOOC Massive Open OnlineCourse
- 3.
- 4. MOOC Completion Rate Classicloop user modeling - adaptation in adaptive systems http://www.katyjordan.com/MOOCproject.html
- 5. What Else TheseStudents Need? • Top colleges – Stanford, CalTech, Princeton, GATech, Penn, Duke.. • Great faculty – top guns in their fields • Great content • Top online platform – Coursera • FREE!
- 6. The Problem ofEngagement • Great free content and top teachers is not enough to engage students • Peter Norvig: Motivation and Engagement are key problems for MOOCs • The problem is not new • A lot of great advanced content – Works perfectly in lab studies, great gains – Released to students to enhance learning – No impact – students do not use it
- 7. The Case ofQuizPACK • QuizPACK: Quizzes for Parameterized Assessment of C Knowledge • Each question is a pattern of a simple C program. When it is delivered to a student the special parameter is dynamically instantiated by a random value within the pre- assigned borders. • Used mostly as a self- assessment tool in two C- programming courses
- 8. QuizPACK: Value andProblems • Good news: – activity with QuizPACK significantly correlated with student performance in classroom quizzes – Knowledge gain rose from 1.94 to 5.37 • But: – Low success rate - below 40% – The system is under-used (used less than it deserves) • Less than 10 sessions at average • Average Course Coverage below 40%
- 9. Adding Motivation • Studentsneed some better motivation to work with non- mandatory educational content… • Added classroom quizzes: – Five randomly initialized questions out of 20-30 questions assigned each week • Good results - activity, percentage of active questions, course coverage - all increased 2-3 times! But still not as much as we want. Could we do better? • Maybe students bump into wrong questions? Too easy? Too complicated? Discouraging… • Let’s try something that worked in the past adaptive hypermedia that can guide students to the right content
- 10. User Model Collects information aboutindividual user Provides adaptation effect Adaptive System User Modeling side Adaptation side User-Adaptive Systems Classic loop user modeling - adaptation in adaptive systems
- 11. Adaptive Link Annotation:InterBook 1. Concept role 2. Current concept state 3. Current section state 4. Linked sections state 4 3 2 1 √" Metadata-based mechanism
- 12. The Value ofANS • Lower navigation overhead – Access the content at the right time – Find relevant information faster • Better learning outcomes – Achieve the same level of knowledge faster – Better results with fixed time • Encourages non-sequential navigation
- 13. Questions of the current quiz,served by QuizPACK List of annotated links to all quizzes available for a student in the current course Refresh and help icons QuizGuide = QuizPACK+ANS
- 14. Topic-Based Adaptation Concept A Concept B Concept C n Eachtopic is associated with a number of educational activities to learn about this topic n Each activity classified under 1 topic
- 15. QuizGuide: Adaptive Annotations • Target-arrow abstraction: – Number of arrows – level of knowledge for the specific topic (from 0 to 3). Individual, event-based adaptation. – Color Intensity – learning goal (current, prerequisite for current, not-relevant, not-ready). Group, time- based adaptation. n Topic–quiz organization:
- 16. QuizGuide: Success Rate n Itworks! n One-way ANOVA shows that mean success value for QuizGuide is significantly larger then the one for QuizPACK: F(1, 43) = 5.07 (p-value = 0.03).
- 17. QuizGuide: Motivation • Adaptivenavigation support increased student's activity and persistence of using the system Average activity 0 50 100 150 200 250 300 2002 2003 2004 Average num. of sessions 0 5 10 15 20 2002 2003 2004 Average course coverage 0% 10% 20% 30% 40% 50% 60% 2002 2003 2004 Active students 0% 20% 40% 60% 80% 100% 2002 2003 2004 n Within the same class QuizGuide session were much longer than QuizPACK sessions: 24 vs. 14 question attempts at average. n Average Knowledge Gain for the class rose from 5.1 to 6.5
- 18. A new valueof ANS? • The scale of the effect is too large… May be just a good luck? • New effect after 15 years of research? • Maybe the effect could only be discovered in full-scale classroom studies – while past studies were lab- based?
- 19. Round 2: Let’sTry it Again… • Another study with the same system – QuizGuide+QuizPACK vs. QuizPACK • A study with another system using similar kinds of adaptive navigation support – NavEx+WebEx vs. WebEx • NavEx - a value-added ANS front-end for WebEx - interactive example exploration system
- 20. WebEx - CodeExamples
- 21. Concept-based student modeling Example2 Example M Example 1 Problem 1 Problem 2 Problem K Concept 1 Concept 2 Concept 3 Concept 4 Concept 5 Concept N Examples Problems Concepts
- 22. NavEx = WebEx+ ANS
- 23. Does it work? • The increase of the amount of work for the course Clicks - Overall 0 50 100 150 200 250 300 Non-adaptive Adaptive Examples Quizzes Lectures - Overall 0 2 4 6 8 10 12 Non-adaptive Adaptive Examples Quizzes Learning Objects - Overall 0 5 10 15 20 25 30 Non-adaptive Adaptive Examples Quizzes
- 24. Is It ReallyAddictive? • Are they coming more often? Mostly, but there is no stable effect • But when they come, they stay… like with an addictive game Clicks - Per Session 0 5 10 15 20 Non-adaptive Adaptive Examples Quizzes Learning Objects - Per Session 0 1 2 3 4 Non-adaptive Adaptive Examples Quizzes
- 25. Why It IsWorking? • Progress-based annotation – Displays the progress achieved so far – Does it work as a reward mechanism? – Open Student Modeling • State-based annotation – Not useful, ready, not ready – Access activities in the right time – Appropriate difficulty, keep motivation
- 26.
- 27. The Diversity ofWork • C-Ratio: Measures the breadth of exploration • Goal distance: Measures the depth Self-motivated Work - C-Ratio (%) 0 0.2 0.4 0.6 Non-adaptive Adaptive Quizzes Examples Self-motivated Work - Goal Distance (LO's) 0 5 10 15 20 Non-adaptive Adaptive Quizzes Examples
- 28. Round 3: Tryinganother domain… • Is it something relevant to C programming or to simple kind of content? • New changes: – SQL Programming instead of C – Programming problems (code writing) instead of questions (code evaluation) – Comparison of concept-based and topic-based mechanisms in the same domain and with the same kind of content
- 29. • SQL-KnoT deliversonline SQL problems, checks student’s answers and provides a corrective feedback • Every problem is dynamically generated using a template and a set of databases • All problems have been assigned to 1 of the course topics and indexed with concepts from the SQL ontology SQL Knowledge Tester
- 30. • To investigatepossible influence of concept-based adaptation in the present of topic-based adaptation we developed two versions of QuizGuide: Topic-based Topic-based+Concept-Based Concept-based vs Topic-based ANS
- 31. • Two DatabaseCourses (Fall 2007): § Undergraduate (36 students) § Graduate (38 students) • Each course divided into two groups: § Topic-based navigation § Topic-based + Concept-Based Navigation • All students had access to the same set of SQL- KnoT problems available in adaptive (QuizGuide) and in non-adaptive mode (Portal) Study Design
- 32. • Total numberof attempts made by all students: in adaptive mode (4081), in non-adaptive mode (1218) • Students in general were much more willing to access the adaptive version of the system, explored more content with it and to stayed with it longer: Questions 0 25 50 75 100 Quizzes 0 5 10 15 20 25 Topics 0 1 2 3 4 5 6 Sessions 0 1 2 3 4 5 Session Length 0 5 10 15 20 25 Adaptive Non-adaptive It works again! Like magic…
- 33. Round 4: TheIssue of Complexity • Let’s now try it for Java… • What is the research goal? • Java is a more sophisticated domain than C – OOP versus Procedural – Higher complexity • Will it work for complex questions? • Will it work similarly? 0% 20% 40% 60% 80% 100% C Java language complexity Easy Moderate Hard
- 34.
- 36. Naviga&on Area Presenta&on Area JavaGuide
- 37. !! !! JavaGuide (Fall 2008) QuizJET (Spring2008) !! parameters (n=22) (n=31) Overall User Statistics Attempts 125.50 41.71 Success Rate 58.31% 42.63% Distinct Topics 11.77 4.94 Distinct Questions 46.18 17.23 Average User Session Statistics Attempts 30.34 21.50 Distinct Topics 2.85 2.55 Distinct Questions 11.16 8.88 Magic… Here We Go Again!
- 38. Round 5: SocialNavigation • Concept-based and topic-based navigation support work well to increase success and motivation • Knowledge-based approaches require some knowledge engineering – concept/topic models, prerequisites, time schedule • In our past work we learned that social navigation – guidance extracted from the work of a community of learners – might replace knowledge-based guidance • Social wisdom vs. knowledge engineering
- 39. Open Social StudentModeling • Key ideas – Assume simple topic-based design – No prerequsites or concept modeling – Show topic- and content- level knowledge progress of a student in contrast to the same progress of the class • Main challenge – How to design the interface to show student and class progress over topics? – We went through several attempts
- 40.
- 41.
- 42. F(1,32)= 11.303, p<.01 71.35% 42.63% 58.31% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% QuizJET+IVQuizJET+Portal JavaGuide Success Rate Success Rate Results: Success
- 43. Class vs. Peers • Peer progress was important, students frequently accessed content using peer models • The more the students compared to their peers, the higher post-quiz scores they received (r= 0.34 p=0.004) • Parallel IV didn’t allow to recognized good peers before opening the model • Progressor added clear peer progress
- 44.
- 45. The Value ofPeers 205.73 113.05 80.81 125.5 0 50 100 150 200 250 Attempts Progressor QuizJET+IV QuizJET+Portal JavaGuide 68.39% 71.35% 42.63% 58.31% 0.00% 20.00% 40.00% 60.00% 80.00% Success Rate Progressor QuizJET+IV QuizJET+Portal JavaGuide
- 46.
- 47. Take-home messages • Acombination of progress-based and state- based adaptive link annotation increases the amount and the diversity of student work with non-mandatory educational content • The effect is stable and the scale of it is quite large • Properly organized Social Navigation might be at least as successful as the knowledge-based • Requires a long-term classroom study to observe
- 48. Why It IsImportant? • Many systems demonstrated their educational effectiveness in a lab-like settings: once the students are pushed to use it - it benefits their learning • However, once released to real classes, these systems are under-used - most of them offer additional non- mandatory learning opportunities • “Students are only interested in points and grades” • Convert all tools into credit-bearing activities? • Or use alternative approaches to increase motivation
- 49. What we aredoing now? • Exploring new generation of open social modeling tools in wide variety if classes and domains from US to Nigeria – Interested to be a pilot site? • Exploring more advanced guidance and modeling approaches based on large volume of social data • Applying open social modeling to motivate readings
- 50. Acknowledgements • Joint workwith – Sergey Sosnovsky – Michael Yudelson – Sharon Hsiao • Pitt “Innovation in Education” grant • NSF Grants – EHR 0310576 – IIS 0426021 – CAREER 0447083
- 51. Try It! • http://adapt2.sis.pitt.edu/kt/ • Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009) Addictive links: The motivational value of adaptive link annotation. New Review of Hypermedia and Multimedia 15 (1), 97-118. • Hsiao, I.-H., Sosnovsky, S., and Brusilovsky, P. (2010) Guiding students to the right questions: adaptive navigation support in an E- Learning system for Java programming. Journal of Computer Assisted Learning 26 (4), 270-283. • Hsiao, I.-H., Bakalov, F., Brusilovsky, P., and König-Ries, B. (2013) Progressor: social navigation support through open social student modeling. New Review of Hypermedia and Multimedia [PDF] Read About It!