Uploaded bydavinia.hl
Remote Collaborative Multi-User Informal Learning Experiences: Design and Evaluation
The document discusses a study on remote collaborative multi-user informal learning experiences conducted at the 6th European Conference on Technology Enhanced Learning. It evaluates a game designed for collaborative learning in remote museum settings, focusing on inter- and intra-group collaboration through a multi-touch interactive installation and video conferencing. It highlights findings on collaboration dynamics, participant satisfaction, and suggests future improvements for enhancing user interaction and engagement in similar environments.
More Related Content
Similar to Remote Collaborative Multi-User Informal Learning Experiences: Design and Evaluation
Remote Collaborative Multi-User Informal Learning Experiences: Design and Evaluation
- 1. Remote Collaborative Multi-UserInformalLearningExperiences: Design and EvaluationProceedings of 6th European Conference on Technology Enhanced Learning, EC-TEL 2011, Palermo, Italy, September 2011, pp. 43-56Ernesto Arroyo, Valeria Righi, Roger Tarrago, Patricia Santos,Davinia Hernandez-Leo,Josep BlatInteractive Technologies Group GTIhttp://gti.upf.eduUniversitatPompeuFabraBarcelona, Spain
- 2. CONTEXTTwo Remote Museums,one Shared Installation
- 6. A SHARED installationLACOL·LECCIÓ DE COL·LECCIONS DE GUY SELZ
- 7. Large collection oftoys and objectsFrom around the World
- 8.
- 9.
- 10.
- 11.
- 12. ConstrainsReal Stakeholders, RealRequirements
- 13.
- 14.
- 15. Sharing between remotelocations
- 16. Game like interaction(playing at the museum)
- 17.
- 18.
- 19.
- 20. Coordinated Activities inremote museumsObjective
- 21. ObjectiveSupport and promoteinter and intra group collaborative learning on a remote environment
- 22.
- 23. Remote and localcollaboration
- 24. Intra- and Inter-group
- 25. Physical and verbalcollaboration
- 26. Intra- and Inter-group
- 27.
- 28. Balanced or equalparticipationApproach
- 29.
- 30.
- 31.
- 32. APPROACH: Interactive ExperienceMulti-touch Interaction + Videoconferencing + Multi-userRemote Game over a cooperative shared spaceCommon museum installationInternetFigueres MuseumBarcelona Museum
- 33.
- 34.
- 35. Remote and localcollaboration
- 36. Intra- and Inter-group
- 37. Physical and verbalcollaboration
- 38. Intra- and Inter-group
- 39.
- 40. Balanced or equalparticipation
- 41. Meaningful and interestingGoalSolutionXMLrepresentation of game customizable aspects
- 42.
- 43. Simultaneous remote andco-located interaction
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51. Game DescriptionLevel 1: ExplorationCollaborative selectionLevel 2: Geographic categorization QuizSimultaneous remote and co-located interactionLevel 3: Functionality categorization QuizSimultaneous remote and co-located interaction
- 53. LEVEl 0Game Status:Player Selection
- 56. LEVEL 1Game Status:Collaborative Exploration
- 58. LEVEl 2Game Status:Categorization
- 60. Level 3Game Status:Categorization
- 62.
- 63.
- 64.
- 65. ProcedureMethod:9 one-shot experimentalsessions (avg. 5.4 minutes)Participants:18 groups - 39 students (2-3 pl. per group)Context:Set-up in two distant rooms in a university campus
- 66. Quantitative video analysisIntra-and inter- group collaborationHelping and discussing with the members of the same/remote teamPhysical and Verbal collaborationPhysical: helping moving a piece, or touching the surface to indicate the place where to move the pieceVerbal: suggesting, verbally, the place where to move the pieceEquality of inter-group collaborationDid teams equally collaborate with the remote team? Time collaborating?
- 67. Questionnaire: quantitative andqualitative analysisPerceived inter- / intra- communication and collaboration and satisfaction closed+ open commentsOverall Satisfaction“I enjoyed playing this game”Perceived collaboration “I actively worked with the players of my/remote group while I was playing the game”“I played the game mostly on my own”Negative &Positive factors“List 5 aspects of the experience, that you liked/disliked
- 68.
- 69. Overall CollaborationCollaboration decreasedas the game progressed % Total Time CollaboratingGame Levels
- 70. Inter-Intra-group CollaborationParticipants engagedsignificantly more inintra-group collaboration than inter-group % Relative Time Collaborating*p<0.0012 3Game Levelsp>0.05
- 71. Physical – VerbalCollaborationVerbal collaboration is preferred than the physical oneWhen cooperative interaction is required (level 1), participants used more deictic gestures to communicate with the remote partners% Relative Time Collaborating*p<0.001Game Levelsp>0.05
- 72. Equity of inter-groupcollaborationCollaboration was mostly balanced between the two groupsWhen negotiation is required (level 1), one team tended to predominate on the other one, by indicating the piece to select unequalGini CoefficientequalGame Levels
- 73. Follow-up surveyOur participantswere overall satisfied of the experience (M. 3.8 /5)
- 74. Perceived Collaboration: usersperceived to have collaborated more with the members of the same team than with the remote partners
- 75. Positive Aspects: multi-touchinteraction, the collaborative nature of the game, videoconference with the remote groupNegative Aspects: split-screen setup, “easiness” of the game, lack of competition
- 76.
- 77. RecapitulationMulti-touch game toincrease collaborative problem solving between remote museums’ visitors
- 78. Shared workspace +Videoconference = increase remote awareness
- 79. Simultaneous interaction avoidstaking-turn between players and facilitate collaboration
- 80. Players play morewith the co-located partners using verbal communication
- 81. Design of theinstallation?
- 82. Design of thegame?FUTURE WORK
- 83. Future Work -EvaluationEvaluation in-the-wild
- 84.
- 85.
- 86.
- 87. Analysis of educationalbenefitsFuture Work - InteractionProblems, system to adapt game depending:
- 88.
- 89.
- 90.
- 91.
- 92. Generic approach basedon XML configuration of behaviors (e.g., levels) and images to show
- 93.
- 94.
- 95.
- 97. Microsoft concept videoF2Finteraction in immersive and ubiquitious environments
- 98.
- 99. Simultaneous face-to faceinteractionCollaborative interaction over 3D shared spaces
- 100.
- 101.
- 102.
- 103. Occlusion freeCooperative Face-to-FaceInteractionCollaborative interaction over 3D shared spaces
- 104.
- 105.
- 106.
- 107.
- 108. CooperativeCooperative Face-to-Face InteractionInvestigatinginteraction methods on that facilitate cooperation on remotely shared 3D environments
- 109. ICT DepartmentUniversitatPompeuFabraC/Tànger, 122-140,E-08018 BarcelonaTel : (34) 93 542 2500 y (34) 93 542 2201Fax : (34) 93 542 [email protected] upf.eshttp://gti.upf.eduThank you!
Editor's Notes
- #3 Two remote museums, a shared installation.This project was developed within the context of two Spanish museums that exhibitedan art collection simultaneously in Figueres and Barcelona cities.The unique characteristics of this use-case directed the application to become an additional channel for museumvisitors to explore the exhibition and increase the sense of connectedness between twospaces. All the while, allowing museum visitors to interact and collaborate.
- #4 in Figueres and Barcelona cities.Toy museum
- #5 Arts Santa Mònica, housed in an exceptional building on the Ramblain Barcelona, is a space of convergence and crossover between the different disciplines of contemporary artistic creation and science, thought and communication.Attentive to the cultural mutations and social changes accompanying the transition to the knowledge society, and working in conjunction with universities and academic institutions, research institutes and centres for the production and presentation of art, science and communication, Arts Santa Mònica generates ideas, projects, research and materials that stimulate dialogue betweenthe local and all it has to offer and the global dimension of society today.
- #6 Located in Figueres, Spain,It provides an introduction to new points of view and serves as a means to reflect on a very familiar subject matter for both children and adults: toys and games.The toys are grouped together by subject area and also include photographs from different time periods of children and their toys so that visitors can see how children played in years past. Each toy also includes an explanatory note (name, date, manufacturer, origin, etc.) and, in some cases, a note written by well-known authors reflecting on specific toys and games.A visit to our Museum allows visitors to be exposed to numerous considerations, encouraging them to reflect on them during or after their visit:· The past, present and the future based on the evolution of games and toys (before and after industrialisation, with the appearance of new materials, depending on society’s evolution and life conditions, etc.);· Facts and phenomena related to the environment and science, for example, in the “Optical Illusions” section or by seeing the multiple applications of scientific developments to toys; and· The aesthetic traits of the toys and games themselves, bearing in mind the different periods, countries of origin or if the toys and games were aimed at boys or girls.
- #7 The exhibition is fruit of the generous donation by Philippe and DorothéeSelz to the Museu del Joguet de Catalunya of the collection of a series of collections that their parents, Françoise and Guy Selz, put together over half a century (1925-1976) and that they themselves have gone on adding to.
- #9 For examples, some of these objects include: OCELLSAquestgrup de set ocellsestanfets de fustapintadaamb colors vius, ratlles, punts, senefesÖCada un delsocellsest‡ colïlocat a sobredíunapetitaplataformaquadrada, fetqueindicaunacondiciÛest‡ticai, per tant, probablementdecorativa. Elsocellsestanfets a Bombai, a la Õndia.DIABLEEl diableest‡ realitzatambcartrÛi paper maixÈsobrefilferroiest‡ fet a Oaxaca, un dels 31 estats de MËxic. Com jasabeu, un diableÈs un esperitsobrenaturalque, origin‡riament, designava un esperit molt savi, perÚ no pas malËfic. Per influËnciareligiosa (sobretotpeljudaisme, cristianismeiislam), el diableÈsentËs com un ‡ngelqueesvarebelïlar contra DÈuivaseguir a Lluciferi, per aixÚ, va ser expulsatdelscels cap a l'Infern. ElsdiablessÛnesperitsquehanesdevingutmalignesique se suposaquemiren de temptarals humans. A Oaxaca el diableÈsunafiguraben present, jaque la confessiÛ de femajorit‡riaÈs el catolicisme.
- #10 Working with real stake holders, That is working with museum curators, with administrators provide real constrains and requirements
- #11 Open-ended scenarios Stakeholders did not agree on a single scenario. Envisioned scenarios included treasure hunting activities, collaborative puzzles, interactivevideoconferencing and performances, collaborative music composition, collaborative tagging, expertise sharing, collaboratively exploring high definitions images, amongothersHowever, despite this open-endedness, we were restricted by fixed contentFixed content: The toy museum has their main focus on toys, and they wanted to center the installation around that topic. In particular, a collections of toys which is so big, that it is rarely presented entirely and they need to rotate the exhibition. Also having another remote museum showcasing the objects in the installation, helps the reach a bigger exposure.The Toy Museum of Catalonia requested for the experience to be a game (one major design constrain) as they receive frequentguided school visits and students had expressed their interest in playing while at themuseum (expected from a toy museum).Thus, it was necessary to develop an agile flexible system to provide the means for enacting different types of interactive learning games and experiences withdifferent user flows and interface behaviors. Finally, coordinated activities in both museums, groups of students visiting the Arts Santa Monica museum in Barcelona and the Toy Museum of Catalonia in Figueres are invited to play a game with other groups of students also visiting a museum in a different city.All stakeholders agreed that incorporating multi-touch surfaces and videoconferencing could enhance the visitor and learner Experience.
- #13 The goal of this experience is to facilitate collaborative problem solving while working in remote environments.The project also seeks to support one-to-one and many-to-many communication to allow more than one learner to participate andinteract in informal learning experiences between remote and co-located learnersCollaboration and cooperationthe primary goal is to facilitate collaborative activities within a group of co-located students, while also facilitating remote collaboration and coordination between groups of students in different locationsIntra-group collaboration refers to discussion and interaction elicited to solve a problem task, in which co-located members are involved. At the other hand, intergroup collaboration refers to problem solving activities that involve members belonging to remote groups.Physical and verbal collaborationIn collaborative environments individual participation can take place both through verbal communication and through physical actions [8]. Our system supports both physical and verbal intra- and inter-group communication and collaborationEquality of CollaborationSimultaneous interaction can prevent turns-taking behavior among participants and at the same time can promote equitable interaction [22]; however it can also result in parallel working and distribution of tasks among users [22]. In a remote environment parallel working and taking-turn behavior might constrain reciprocity and communication, therefore limiting collaboration among remote members. We adopted several design criteria to avoid limiting collaboration and promote an equal participation among remote partners. A multi-touch table allows for direct manipulation of digital content while
- #19 to facilitate intra- and inter- group collaboration,the system provides students with a shared workspace where simultaneous and multiuserinteractions are allowed for both co-located and remote participants
- #21 Interaction type:
- #35 evaluation consisted of 10 one-shot experimental sessions in which 20 groups (with 2 to 3 persons per group) of 39 first-year university students (14 women, 25 men) played all game levels (lasting on avg. 5.4 min.) in a controlled environment. The stations were set up in two remote rooms at a university campus. Participants in each room were given few minutes to familiarize with the multi-touch table and to introduce themselves to the remote team through videoconference while one researcher explained the main goal of the game to both teams. The researchers remained in the room throughout the session, but did not interact with participants unless in response to specific problems related to technical difficulties. All sessions were video recorded and the actions of each participant were analyzed via videocoding---------------------------------------------------mh...good point! He revisado el documento excel con todos los datos. Las secciones fueron 10 (== 20 grupos), estoy segura de esto! Pero, no me acuerdo por cual razón, tenemos la análisis de solo 9 grupos (tot personas 39!). Quizá seria mejor cambiar la slide y poner 9 experimental sessions y 18 grupos, o si no cambiar el numero de personas a 45..
- #36 explore the efficiency of the design criteria in promoting collaboration and discussion among participants. The user study provided quantitative and qualitative data to assess how the collaboration features in game lead to different collaborative behaviors, and to assess user’s perception and satisfaction with the game.Intra and Inter group collaboration were measured by calculating the time (in seconds) passed by each group in interacting with the co-located or remote partners (e.g. discussing where to move a piece; making suggestions, indicating on the table the correct place etc…).Values represents the score for the entire group, computed as the sum of the contributions of each member.We are interested in exploring three aspects of collaboration: 1) to what extent thegame promoted inter- and intra-group collaboration among the different stages of thegame, 2) the existence of a preferred type of collaboration, i.e. a tendency of users incollaborating more with the co-located members (intra-group collaboration) than withthe remote ones (inter-group collaboration), and 3) to what extent the game promotedequal inter-group participation, i.e. if both groups equally collaborated with the otherteam or if the process of collaboration was mostly initiated by one of the two teams.
- #37 We collected information about overall user satisfaction using a follow-up questionnaire divided into four main categories based on the following criteria: perceived inter- and intra-group communication, perceived inter- and intra-group collaboration, overall satisfaction of the experience. Overall satisfaction was assessed using close and open qualitative questions to elicit positive and negative aspects
- #39 Analysis of collaborative patterns in users’ activities showed that collaborative engagement of participants decreased as the game progressed: on average, participants collaborated for 23.3%, 16.7%, and 13.5% of the total time for level one, two and three respectively. HP: A possible explanation of this phenomenon could be related with the fact that at the beginning of the game users are more willing to collaborate between themselves, moreover they are invited to talk and negotiate in order to select a unique piece. As the game progressed, this initial stimulus decreases and so does the collaboration. **Vertical axis represents the percentage of the time passed collaborating to respect of the level completion time
- #40 This indicates that participants were more engaged in talking and interacting with the members of the same team than in discussing with the remote team, that was true for both level 2 and 3, where inter and intra group collaboration are elicited.**Vertical axis represents the percentage of the time passed collaborating in one of the two modalities (inter- intra-), to respect of the total time passed collaborated.
- #41 During the entire game, users preferred verbal communication as channel for collaboration rather than physical interaction (e.g. deictic gestures) on the surface (p <.001). But in level 1, when coordination and cooperative gestures are required, participants used more deictic gestures (physical collaboration) to communicate with the remote partners with respect to the other levels. **Vertical axis represents the percentage of the time passed collaborating in one of the two modalities (physical, verbal), to respect of the total time passed collaborated.
- #42 Collaboration was mostly balanced between the two groups. However, in level 1, when coordinated gestures are needed, one team had a clear predominating behavior, indicating to the other team the piece to selectWe used the Gini Coefficient to express the equitable nature of collaboration among remote teams. It uses the contribution of each group to compute the deviation with respect of equal participation between the two groups.
- #43 Users perceived to have collaborated more with the members of the same team than with the remote partner;, this validates our observation.Feedback regarding the positive aspects indicates that participants appreciated the collaborative nature of the game and the type of interaction allowed by the multi-touch interface.Analysis of the negative aspects of the game revealed several problems regarding the design of the game, such as the lack of clarity in the goal of the game, the lack of competition, and the game subject, the easiness of the game etcc... Other complains were addressed to the setup of the system, participants were annoyed by the fact that they had to split the attention between the two screens, they mention that sometimes they were not aware of the suggestions displayed on the vertical screen.
- #47 Eduactiional evaluation. Testing with real children in a natrural environment, would allow us to assess the educational impact of the application. We believe that it is necessary to go beyond lab settings and conduct evaluation on real, live environments. This will provide the test fire, and would allow us to really find out if the system satisfies the overall objectives.
- #48 We identified several problems. For instance, the system would have to adapt to situations in which only one user interacts by himeself on the space. As there is only one user the COLLLABOATIVE goals of the application NO LONGER are suitable, so the system should also offer a enjoyable experience to this type of user. Additionally, a similar situation occurs when there is a GROUP of participants interacting with the system, they will no longer have somebody to talk to on the other side In fact, they would not be able to use the app, as it REQUIRES both remote groups to participate. This could be addressed by adding a SOLO mode, or by simulating user interactions on the REMOTE station. Future work includes exploring other scenarios, other installation, or other settings. Specially scenarios where COOPERATIVE INTERACTION ON THE SHARED SURFACE makes sense. The Application is generic ENOUGH as to take an XML configuration file to change the behaviors, levels. All that is required is changing a few images to generate a brand new application. Some other interaction and usability aspects to improve include. Increasing remote awareness- Face to face interactions. and the use of shared immersive workspaces
- #49 Increase awareness support. Such as showing the other participant’s actions so that it is in a clear and interpretable by the end-users. At the moment weonly use highlight to show some other user is interacting with an object. But users have shown that they need further visual feedback.---------Pauchet. Ref aqui: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04384108
- #50 Facilitate face to face interaction. One of the draebacks in thesystem is that people DO NOT REALLY SEE each other. This is not a new problem, as many other similar systems share the same situation. Tele-graffiti LivePaperAgora Others….Commercial video-conferencing systemsThis problem has to be address, however there is a technical limitation in the placement of the camera. A it would need to be behind the screen. DoubleDigitalDesk
- #51 Thus, the future is to have immersive face to face interactive spaces where people can collaborate and communicate seamlessly. For instance, The Microsoft concept video illustrates this point showing their vision for supporting for face to face interaction in immersive and ubiquitous environments.
- #52 We are already tackling some of these problems in our research work.