How to Conduct Usability Studies: A Librarian Primer
2 likes846 views
This document provides an overview of how to conduct usability studies. It discusses planning a study by determining objectives and recruiting participants. Key steps in the process include preparing tasks and test scripts, moderating studies by having participants think aloud, and analyzing results to identify usability issues. Findings should then be communicated through a report and presentation. The goal is to evaluate products and services by testing them with representative users to identify problems and improve user experience. Resources for further information on usability studies are also provided.
How to Conduct Usability Studies: A Librarian Primer
- 1. How to Conduct Usability Studies A Librarian Primer Tao Zhang, PhD Digital User Experience Specialist Purdue University Libraries [email protected] Sponsored by ALA RUSA #RUSAUX Ilana Barnes Business Information Specialist Purdue University Libraries [email protected]
- 2. Learning Objectives 2 • Participants will be able to describe the usability concept and it's applications to libraries. • Participants will be aware of the basic workflows and techniques needed when doing usability tests. • Participants will be aware of resources for usability studies and where to find out more.
- 3. Agenda 3 • What is a usability study? • Usability study process • Conducting the study • Analyzing Results • Communicating • Where to find out more
- 4. About Us Ilana Barnes Tao Zhang 4
- 5. Tell us about you! What type of library do you work in? A) Public B) Academic C) Other How familiar are you with Usability Studies/Tests? A) Very familiar B) Somewhat familiar C) Not familiar but excited to learn! Chat box: Areas you especially want us to talk about 5
- 6. What is User Experience? 6
- 7. Definition Usability study refers to“evaluating a product or service by testing it with representative users.“ - http://www.usability.gov/how-to-and-tools/methods/usability-testing.html 7
- 8. How UX fits into Reference Services • Decisions that user experience research can help you with: • What should our libraries front page look like? • What do our undergraduates expect from the library? • What would we like our undergraduates to expect from the library that they do not know or understand or need? • Understanding your users aims and needs help you help them and show your impact at the same time. 8
- 9. How to plan your usability study topic • What are my objectives? • How much time do I have? • How many teammates do I have? • Who am I presenting this to? • Is this for an existing product? 9
- 10. When to Test • Early or middle stages of development • Types: Exploratory | Assessment | Comparison | Validation • Start preparing at least three weeks before actual test • DO pilot tests 10
- 11. Usability Study Process 11 Recrui'ng Preparing Conduc'ng Analyzing Presen'ng Choosing Features Creating Tasks Preparing Test Script
- 12. Recruiting • Determining the target audience (behavior) • Finding representative members – Behavioral criteria, demographics – Avoid sampling bias • Convincing participants to come – Incentives – Informed content (duration of study, data to collect) • Scheduling – Reminding – Deal with no shows 12
- 13. Recruiting Best people to invite: • People who are going to need the service in the near future • People who have used a competing/previous version service in the recent past Number of participants: • 5-8 • Also consider representativeness and tasks 13
- 14. Choosing Features • Input from: Development team | Stakeholders | Users • Content inventory & task analysis 14
- 15. Creating Tasks Tasks should be: • Reasonable and doable • Described in terms of end goals • Specific Run through tasks to see how long each will take Determine what quantitative data to collect 15
- 16. Discussion You need to do a usability study of your library’s website, because users find it difficult to find information they want on the website. What are some possible tasks you can use in the usability study? 16
- 17. Writing A Study Script • Introduction (5 min) • Warm-up interview, demographics survey (5 min) • Scenarios and tasks (20-30 min) • Interview (10-15 min) • Wrap-up (5 min) 17
- 18. Conducting the Study The physical layout with observer 18
- 19. Conducting the Study The physical layout without observer 19
- 20. Conducting the Study TechSmith Morae www.techsmith.com/morae.html 20 Recorder Observer #1 Participant Test Moderator Observer Manager Observer #2 …
- 22. Conducting the Study Testing on mobile devices 22 IPEVO Ziggi-HD High-Definition USB Document Camera DIY Mobile Usability Testing - SXSW Interactive 2012 http://www.slideshare.net/beleniq/diy- mobile-usability-testing-sxsw- interactive-2012
- 23. Moderation • Follow script but allow some flexibility • Encourage participants to“think aloud” • Ask about expectations • Ask“why”a lot • Suggest solutions when participants are stuck • Discuss with observer right after test 23
- 25. Observation • Watch, Understand • Limit the time users have to perform a task • Encourage users to think aloud with neutral prompts • Elicit detailed information 25
- 26. Interview • Introduction • Warm-up • General issues • Deep focus • Wrap-up 26
- 27. Nondirected Interview Prepare nondirected questions • Focused on the person answering it • Focused on experience, not extrapolation • Concentrate on immediate experience • Avoid judgmental language • Focus questions on a single topic • Keep questions open-ended • Avoid binary questions 27
- 28. Nondirected Interview Examples “Is this a useful feature?” “Is this feature valuable to the work you do right now?” “Do you think this feature would be better if it was also available on smart phones?” 28
- 29. Nondirected Interview Running the interview • Define (technical) terms • Don’t force opinions • Restate answers • Never say the participant is wrong • Work with a partner 29
- 30. Analyzing Results • Collecting observations – Notes from moderator and observer – Quantitative information: task successfulness, navigational path, number of steps, errors, time to complete, subjective ratings (System Usability Scale) – Qualitative feedback from participants • Extracting trends – Repeated concerns – Multiple issues with possible common origin – Identify effects on user experience 30
- 31. Analyzing Results • A list of usability issues – Quantitative and qualitative data as evidence – Possible causes – Effects on user experience • Assign importance/severity • Suggest solutions 31
- 32. Communicating Findings Usability test report • Project overview • Participants • Testing environment • Tasks • Procedure • Key findings – Issues – Recommendations 32
- 33. Communicating Findings Presentation • Prepare the audience • Use supporting evidence (e.g., highlight videos) • Emphasize user perspectives • Use numbers carefully (because of small sample) • Leave enough time for questions 33
- 34. Communicating Findings Common problems: • “This is not statistically significant.” • “This user is our typical user.”“This user does not have relevant experience.” • “User X did Y; therefore, everyone must do Y.” • “More than half of users did not like this design; therefore, we should change it.” 34
- 35. What’s Essential? • Develop a study plan • Find participants • Conduct the sessions • Debrief with observers and decide together • Present results 35
- 36. Discussion Think of an existing design that you want to evaluate for effectiveness • What are my objectives? • What types of resources do I have? • Who am I presenting this to? 36
- 37. Other Types of Usability Studies • Remote usability testing http://www.userzoom.com/ • Eye tracking 37 http://www.normalmodes.com/blog/2009/09/28/eye-tracking- heatmap-gallery-a-preview-discussion-of-ui-considerations/
- 38. Where to Find Out More • 6 Must-See Usability Testing Videos http://blog.usabilla.com/7-must-see-usability-testing-videos/ • Moderating Usability Tests Joe Dumas & Beth Loring (2008) • A Field Guide to Usability Testing Smashing eBook https://shop.smashingmagazine.com/guide-to-usability-testing-1.html • Handbook of Usability Testing Rubin & Chisnell (2008) • Don’t Make me Think by Steve Krug Chapter:“Usability Testing on 10 cents a Day”’ 38
- 39. Learning Objectives 39 • Participants will be able to describe the usability concept and it's applications to libraries. • Participants will be aware of the basic workflows and techniques needed when doing usability tests. • Participants will be aware of resources for usability studies and where to find out more.
- 40. Agenda 40 • What is a usability study? • Usability study process • Conducting the study • Analyzing Results • Communicating • Where to find out more
- 41. Works Cited • 6 Must-See Usability Testing Videos - The Usabilla Blog. (n.d.). Retrieved from http:// blog.usabilla.com/7-must-see-usability-testing-videos/ • Affairs, A. S. for P. (2013). Usability Testing. Retrieved from http://www.usability.gov/how-to-and-tools/methods/usability-testing.html • Clear, Consistent, Capable, NICE» Blog Archive » Eye Tracking Heatmap Gallery: A Preview & Discussion of UI Considerations. (n.d.). Retrieved from http://www.normalmodes.com/blog/ 2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/ • Dumas. Joe, & Loring, B. (2008). Moderating Usability Tests. • DIY Mobile Usability Testing - SXSW Interactive 2012. (n.d.). Retrieved from http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012 • Morae usability testing software from TechSmith. (n.d.). Retrieved from http://www.techsmith.com/morae.html • Krug, S. (2005). Don’t Make Me Think. • Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing. • UserZoom | zooming in on the user experience. (n.d.). Retrieved from http://www.userzoom.com/ • Silverback — guerrilla usability testing. (n.d.). Retrieved from http://silverbackapp.com/ 41
- 42. Final Notes Evaluation of this webinar Questions? Tao Zhang [email protected] Ilana Barnes [email protected] 42