Jane Ruffino

A lot of UX researchers aren’t sure what should actually go into their reports. Too often, findings get reduced to a single vague p-value. The problem? That doesn’t really tell the full story. If we want our work to be trusted and actionable, there are at least three things we should always report: p-values, effect sizes, and confidence intervals. Each one answers a different question. P-value: Is it real or just chance? The p-value is about statistical significance. It tells you how likely it is that your results could have happened by accident if there was no real effect. It’s useful, but by itself it’s limited, it won’t tell you how important the effect is in practice. Effect Size: How big is the difference? Effect size puts the result into context. Was the change large enough to matter for the user experience? A checkout redesign might show a “significant” improvement, but if it only saves half a second, is that really meaningful? Reporting effect size makes sure you’re talking about impact, not just numbers. Confidence Interval: How precise is our estimate? A confidence interval shows the range of values the true effect probably falls into. A narrow range means your estimate is precise, while a wide one means more uncertainty. This helps stakeholders see not just what you found, but also how much confidence they should place in it. When you combine these three, you give a complete picture: the p-value says whether the effect is credible, the effect size says how much it matters, and the confidence interval shows how reliable your estimate is. Together, they turn a UX report from a loose claim into a story stakeholders can actually use to make decisions.

127

17 kommentarer

You asked, I delivered. Quick, honest UX career advice 🫶 This week, I answered your top UX career questions ⬇️ 1. Struggling to find a design job. How to stand out? 2. How is it to be a UX designer in Norway? 3. How to plan a design career? What to learn? 4. How to start working 
on a design portfolio? 5. Should I do conceptual projects if I lack real ones? 6. Should I start freelance 
or do 9 to 5 design job? 7. How to grow 
as a freelance designer? Answers in the carousel. Want more tips like these straight to your inbox? Join my newsletter for weekly insights ➡️ https://shorturl.at/3JiQD Have more questions? Ask me in the comments and I’ll answer them 
in my next posts 😊 ⬇️ #UXDesigner #DesignerCareer

154

78 kommentarer

New Article: Mixed Methods Studies That Mix Many UX teams say they’re doing mixed methods, but they’re really just running 2 separate studies and calling it a day. This week’s article breaks down what real mixed methods looks like in UX research. It covers how to design studies that actually integrate qual and quant, when to use them, and how to deal with conflicting findings. Check it out (especially if you’re a UX researcher or research-adjacent working in the real world). https://lnkd.in/euJtJaJG

39

2 kommentarer

🌱 THOUGHT LEADERSHIP WARNING 🌱 As a design systems leader who’s build high-craft libraries at scale, I knew I had to disrupt my own learning methodology. After months of stakeholder alignment sessions and design token frameworks, I made a bold strategic decision. I would take my expertise to the ultimate testing ground. Nature’s original design system. I’m talking about touching grass. Actual grass. With my hands. The insights were transformational. Here are 5 design system lessons I learned from this unconventional user research: 1️⃣ Consistency doesn’t mean identical Every blade of grass follows the same growth pattern, but no two are exactly alike. Your components can share DNA without being carbon copies. 2️⃣ Scalability happens naturally A lawn doesn’t need a roadmap to grow. It just needs the right conditions. Stop over-engineering your system’s future and focus on healthy foundations. 3️⃣ Maintenance beats perfection Grass doesn’t grow in straight lines, but regular mowing keeps it functional. Your design system needs consistent care, not perfect documentation. 4️⃣ Adaptation is survival Grass grows differently in shade vs sun, wet vs dry. Your components should flex for different contexts instead of breaking. 5️⃣ Root systems matter more than what’s visible The real work happens underground. Your design tokens and naming conventions are more important than your Figma library looking pretty. Side effect: I can now identify 12 different grass species and my Vitamin D levels are no longer concerning my doctor. 10/10 would recommend this UX research method. #DesignSystems #TouchGrass #ModelHopping

143

9 kommentarer

Excited to share a new chapter I co-authored ✨ "Wellbeing-Centered UX: Supporting Content Moderators" In Trust & Safety, we talk a lot about protecting communities. This work reminds us that the people behind the tools matter too. Trust and Safety experts face some of the hardest challenges online, and the way we design their tools directly impacts both their wellbeing and the quality of their decisions. Our chapter explores how applying UX principles — clarity, usability, and a focus on human needs — can make Trust & Safety systems more sustainable and humane. I’m deeply grateful to the Trust and Safety Foundation for the opportunity to contribute to their book "Trust, Safety, and the Internet We Share: Multistakeholder Insights" and for fostering a space where design, research, and policy intersect. I see this as part of an ongoing conversation: how can UX and Trust & Safety continue to learn from each other? I’d love to hear how others are thinking about designing for the people who keep our platforms safe. Leaving the link to the pre-print of the chapter, as well as many others from the book (which will be out later in 2025) in comment below. 

62

4 kommentarer

Why I don’t ask people to think aloud in evaluative research 💭 The ‘think aloud’ method has been a staple of usability testing for decades. But in my experience, the way a lot of researchers use it can introduce some issues: 🚫 Some participants take it too literally, narrating every tiny action. This isn’t how they’d behave at home. 🚫 It makes an already unnatural setting feel even less realistic. 🚫 Once someone starts overanalysing, it’s very hard to bring them back. Instead: ✅ Most participants will talk naturally if you just observe them. ✅ If they’re quiet, you can nudge with simple prompts like “what are you thinking?” I’ve found this leads to more authentic behaviour and better insights than briefing people to ‘think aloud’ from the start. 📨 I cover this in more detail in this week's Desk Notes, my free newsletter. Link in the comments.

60

15 kommentarer

Accessibility isn’t optional - here’s why inclusive design matters. As designers, it’s easy to get caught up in aesthetics, flows, and shiny prototypes. But here’s the truth: if your product isn’t accessible, it’s not complete. Accessibility in design means making sure people of all abilities can use your product, whether that’s someone with low vision navigating a website, a person relying on captions to follow a video, or users with motor impairments who need clear, simple interactions. Inclusive design isn’t just about compliance or ticking WCAG checklists. It’s about empathy. It’s about asking: Can everyone, regardless of ability, truly use and benefit from this product? And here’s why it matters: - It expands your user base (billions of people live with disabilities). - It improves overall user experience - what helps one group often benefits all. - It shows your brand values people, not just profits. Accessibility isn’t an afterthought. It’s good design. Designers, founders, and product teams: let’s stop seeing it as optional and start making it the standard. P.S. What’s one accessibility practice you’ve built into your design process? Your product deserves a designer who listens - I’m here📌 #enoobongakpan

67

10 kommentarer

Many times in UX research, we do not have access to large datasets. We work with 5 participants. Or 8 diary entries. Or 12 responses from a hard-to-reach user group. And that is completely fine - until we start treating small datasets like they behave the same way larger ones do. Here are six mistakes that can quietly derail small-sample studies if we are not careful: Mistake 1: Underestimating model complexity Simple models can still be unstable with low N. Be selective. Use theory to guide your variables, and stick to the leanest model that answers the question. Mistake 2: Ignoring identifiability A model is not identifiable if it cannot produce clear, unique estimates. That means you might be modeling noise and not even know it. Simulating known data is a powerful way to catch this early. Mistake 3: Skipping assumption checks Small samples still require you to check assumptions like linearity and residual behavior. Ignoring these can quietly distort your conclusions. Mistake 4: Avoiding simulation Simulating data is not just for theory papers. It helps you stress-test your model before it meets messy real-world inputs. Mistake 5: Misusing Bayesian methods Bayesian tools can be helpful with small data, but only if you are thoughtful about your priors. In small-N settings, your priors can drive the entire result. Mistake 6: Not reporting decisions Every choice - your model structure, your priors, your checks - should be reported. In small-sample research, transparency is your credibility. If this resonates, I recommend checking out this upcoming workshop led by Mohsen Rafiei, Ph.D., which focuses exactly on these kinds of challenges in UX and behavioral research. You can find all the details and sign-up info here: https://lnkd.in/e4gt88cz

52

3 kommentarer