Download to read offline
![Greedy Sequencing: Content Ranking by
Knowledge Maximization
15
amount of known
prerequisites
amount of
unknown outcomes
rank of the content, [0-1]
number of outcomes
np:number of prerequisites
ki: knowledge of concept i
wi: weight of concept i, log(tf-idf value)](https://image.slidesharecdn.com/ec-tel2015versionpeterpresented-151026170521-lva1-app6892/75/EC-TEL-2015-15-2048.jpg)
Uploaded byRoya Hosseini
EC-TEL 2015
This document summarizes research on adaptive sequencing in open social student modeling. It describes how combining knowledge-based guidance with social guidance can encourage non-sequential navigation, increase learning speed for strong students, and positively relate to student performance. A classroom study found that adaptive sequencing increased learning speed and the odds of correct problem solving. Students also provided positive subjective feedback about recommendations. Future work aims to explore alternative visualization and awareness techniques.
More Related Content
EC-TEL 2015
- 1. What Should IDo Next? Adaptive Sequencing in the Context of Open Social Student Modeling Roya Hosseini, I-Han Hsiao, Julio Guerra, Peter Brusilovsky PAWS Lab University of Pittsburgh
- 2. Overview • Motivation –why do we care about guidance? • Past work – how to guide students to the right content? • Current work – adaptive sequencing combined with social guidance – what we learned from the classroom study • Work in progress & future work 2
- 3. Motivation Goal – personalized guidanceto the most appropriate educational content for each learner ! Why personalized guidance? – helps students acquire knowledge faster – improves learning outcomes – reduces navigational overhead – increases student motivation to work with content 3
- 4. Existing Guidance Technologies 1.Knowledge-based approaches • decide the most appropriate content for an individual with respect to the domain model, student model, and course goal • adaptation type: • fine-grained concept-based (ELM-ART, NavEx) • coarse-grained topic-based (QuizGuide) ! 2. Social guidance 4
- 5. Concept-Based Adaptation Example 2Example M Example 1 Problem 1 Problem 2 Problem K Concept 1 Concept 2 Concept 3 Concept 4 Concept 5 Concept N Examples Problems Concepts 5
- 6. ELM-ART: Adaptive LinkAnnotation in LISP 6 green bullet indicates a recommended page red bullet indicates a page user is not ready for G. Weber And P. Brusilovsky, IJAIED 2001. Elm-Art: An Adaptive Versatile System For Web-Based Instruction
- 7. NavEx: Concept-Based AdaptiveNavigation Support bullet is filled based on progress font style denotes the relevance of example a relevant example with no progress an example not ready to be browsed 7 M. Yudelson And P. Brusilovsky, AIED 2005. Navex: Providing Navigation Support For Adaptive Browsing Of Annotated Code Examples.
- 8. Topic-Based Adaptation • eachtopic is associated with a number of educational activities ! • each activity is classified under 1 topic 8 Topic A Topic B Topic C
- 9. QuizGuide : Topic-BasedAdaptive Navigation Support Current quiz number of arrows: knowledge in the topic (0-3) color Intensity: learning goal P. Brusilovsky, S. Sosnovsky And O. Shcherbinina, E-Learn 2004. Quizguide: Increasing The Educational Value Of Individualized Self-Assessment Quizzes With Adaptive Navigation Support. 9 current prerequisite not-relevant not-ready
- 10. Knowledge Maximizer Paradigm 10 Hosseini,R., Brusilovsky, P., & Guerra, J. (AIED 2013, January). Knowledge Maximizer: Concept-based Adaptive Problem Sequencing for Exam Preparation. Learn maximum knowledge from next activity while controlling prerequisites
- 11. Existing Guidance Technologies 1.Knowledge-based approaches 2. Social guidance • uses Open Social Student Modeling (OSSM) • students can view each others’ or class knowledge model • almost as efficient as knowledge-based guidance - higher success rates & engagement - much less knowledge engineering overhead • drawback: make students more conservative in their work ! ! 11
- 12. Mastery Grids: Topic-basedNavigation Support in OSSM Platform anonymized ranked list of peers and their topic-based progress position of current student in class topic-based progress of student topic-based progress of class Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. (EC-TEL 2014). Mastery Grids: An Open Source Social Educational Progress Visualization. 12
- 13. • combines socialguidance with knowledge-based guidance • enhances the approach to maximize student knowledge • implements the guidance in context of Mastery Grids OSSM • reports the results from the classroom study Sequencing + Open Social Student Modeling 13
- 14. Greedy Sequencing (GS) •aims at maximizing student knowledge in domain concepts • concept-based adaptation: - uses prerequisite and outcome concepts in content items 14 User% Modeling% database% Greedy% Sequencing% Knowledge% Report%Service% Rank%C1% Prerequisites% Outcomes% Content%C1:%Concepts%
- 15. Greedy Sequencing: ContentRanking by Knowledge Maximization 15 amount of known prerequisites amount of unknown outcomes rank of the content, [0-1] number of outcomes np:number of prerequisites ki: knowledge of concept i wi: weight of concept i, log(tf-idf value)
- 16. 16 • marked topthree recommendations generated by GS • size of star shows relative rank of content - bigger star —> higher priority
- 17. The Study 143 undergraduatesin ASU (Fall 2014) Java Programming & Data Structure course ‣ 111 problems — 103 examples — 19 topics ! Study had 2 main Parts (1) no sequencing (Aug. 21 – Sep. 25) (2) with sequencing (Sep. 26 – Oct. 21) • 86 subjects logged into the system • we considered 53 subjects with problem attempts >= 30 17
- 18. Navigational Pattern Analysis GSbreaks out the common path of social guidance 0.08 0.08 0.16 0.68 0.06 0.05 0.12 0.78 0.17 0.17 0.2 0.47 Jump−Backward Jump−Forward Next−Topic Within−Topic Part 1 Part 2−N Part 2−R when following GS, “groupthink” stay on the current topic shortens considerably ! students moved to next topic more quickly & expanded their non-sequential navigation
- 19. Value of GSon Amount of Learning & Speed 19 Learning gain: • no significant differences in the learning gain - non-followers (M = 0.50, SD = 0.27) - followers (M=0.44, SD=0.23) ! Learning speed: (learning gain/number of problem attempts)×100 ! • speed of learning was higher among the followers - non-followers (M = 0.54%, SD = 0.27%) - followers (M = 0.97%,SD = 0.88%) speed increased about twice - p = .083, using a Welch t-test
- 20. Value of GSon Learning & Speed: Weak vs. Strong Students 20 0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.40# 1.60# 1.80# 2.00# Weak#students# Strong#students# %#Learning#speed## Non;followers# Followers# 0" 0.1" 0.2" 0.3" 0.4" 0.5" 0.6" 0.7" 0.8" 0.9" Weak"students" Strong"students" Normalized"learning"gain" Non?followers" Followers" • no significant differences in learning gain • followers with high prior knowledge learn faster (p=.039)
- 21. Value of GSon Problem Solving Performance 21 Correctness is more frequent in recommended problems • odds of correct answer in a problem offered by GS was 1.59 (SE = 0.19) times more than a not-recommended problem How: • data collected from part 1 and 2 of study (5760 problem attempts: 5275 not-recommended, 485 offered by GS) • fitted a logistic mixed effects model • fixed effect: attempt type (recommended, not-recommended) • response variable: correctness of attempt (0/1)
- 22. Value of GSon Class Performance 22 An attempt on a GS recommendation was associated with higher grade • attempting a recommended content (problem/example) was associated with 0.56 increase in final grade (SE=0.24, p=.017) ~ 9 times greater than the effect of a not-recommended content How: • data of 40 students (had exam score + used system) • fitted regression model to predict exam grade using number of attempts on contents
- 23. • 6 questions(5-point Likert scale) • data collected from 51 students (answered questionnaire + used the system) M:4.1 M:3.9 M:3.1 M:3.8 M:4.2M:2.4 Subjective Feedback 23 like star useful clear ! reason distractive
- 24. Wrap Up adaptive sequencing+ social guidance: ! ✓encouraged non-sequential navigation patterns ✓increased learning speed of stronger students ‣ more optimal content navigation ✓was positively related to student performance ‣ higher exam score ‣ more success in problems
- 25. Work in Progress& Future Work ๏ running study with over 200 students in ASU - GS vs. probabilistic approach based on FAST ! ๏ what is the best way to visualize student/class data? - alternatives to topic-based guidance (2D content maps ) ! ๏ how to increase students’ awareness of recommendations? - adding annotations, …
- 26. References Knowledge Maximizer: Hosseini,R., Brusilovsky, P., & Guerra, J. (2013, January). Knowledge Maximizer: Concept-based Adaptive Problem Sequencing for Exam Preparation. In Artificial Intelligence in Education (pp. 848-851). Springer Berlin Heidelberg. ! Mastery Grids: Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. (2014). Mastery Grids: An Open Source Social Educational Progress Visualization. In Open Learning and Teaching in Educational Communities (pp. 235-248). Springer International Publishing. ! QuizGuide: P. Brusilovsky, S. Sosnovsky And O. Shcherbinina, 2004. Quizguide: Increasing The Educational Value Of Individualized Self-Assessment Quizzes With Adaptive Navigation Support. In: J. Nall And R. Robson, Eds., World Conference On Elearning, E-Learn 2004 Aace, Washington, Dc, Usa, 1806-1813. ! NavEx: M. Yudelson And P. Brusilovsky, 2005. Navex: Providing Navigation Support For Adaptive Browsing Of Annotated Code Examples. In: C.-K. Looi, G. Mccalla, B. Bredeweg And J. Breuker, Eds., 12Th International Conference On Artificial Intelligence In Education, Ai-Ed'2005 Ios Press, Amsterdam, The Netherlands, 710-717. ! ELM-ART: G. Weber And P. Brusilovsky, 2001. Elm-Art: An Adaptive Versatile System For Web-Based Instruction. International Journal Of Artificial Intelligence In Education, 12 (4), 351-384 26
- 27. Thank You! Intelligent SystemsProgram Roya Hosseini [email protected] Peter Brusilovsky [email protected] I-Han (Sharon) Hsiao [email protected] Julio Guerra [email protected] Try it! adapt2.sis.pitt.edu/kt/mg-gs.html https://www.youtube.com/watch?v=Kak8F2y5GkU