Fast Human
Insight For
Marketing
VP, Product Marketing
Michael Mace
Summary
● We’re the leading provider of ultra-fast user feedback on websites and apps
○ Feedback from real customers within two hours
○ Usability, navigation, etc.
○ 10 years old, 50k tests a month
● Rising adoption by marketers: ~20% of our volume
○ Quick reactions to messages and material when there’s no time/budget for traditional research
○ Shareable evidence of customer attitudes and reactions
○ Eliminates guessing
○ Far faster than traditional research, national sample, easy for anyone in team to use
● Goal for today: Describe how marketers use us, with real-world examples
○ Tips for your own studies
Agenda
● How UserTesting works
● Real-world examples of how marketing teams use it
○ Validating content and messages
○ Discovering quick insights about attitudes and journeys
○ Improving A/B tests
● How we used it for this webinar
● Tips for your own studies
● Q&A
What We Do
● Human insights at the speed
of digital business
○ In two hours, video and audio
feedback from customers on
almost anything
○ Target your particular customer
○ So easy that anyone in your team
can use it
How it Looks
How it Works
Target
• Panel with demographic
filters, screeners
• Or invite your own
customers, partners
Discover
• Recorded studies: Via
desktop and mobile. List
questions and tasks. Two
hour turnaround.
• Live video interviews:
Schedule and recruit in hours
Understand
• Jump to key video insights
via survey questions and
transcript
• Click and drag to make clips
• Share instantly via email
and Slack
Agenda
● How UserTesting works
● Real-world examples of how marketing teams use it
○ Validating content and messages
○ Discovering quick insights about attitudes and journeys
○ Improving A/B tests
● How we used it for this webinar
● Tips for your own studies
● Q&A
How We’re Used in Marketing
“There is no data for the
short-term decisions I
have to make today. The
competition is not quant
vs. qual. It's qual vs. gut
instinct. The way you
could help me is by
providing data – customer
feedback – where it was
just opinion before.”
--VP Global Marketing, F500
manufacturer
● Validate: Fast reactions to marketing
materials and messages
○ Confirm whether it resonates
○ No more guessing
○ Persuasive video evidence
● Discover: Quick insight on customer
attitudes, needs, and journeys
● Iterate: Improve your A/B tests
○ Increase your win rate by pre-testing
alternates
Recent Studies: Messaging and Material
● Which image is most trustworthy and compelling?
● Which email content resonates best with participants?
● How well do customers understand the vocabulary we use?
● What message did the ad convey?
● How effective are we at communicating who we are and why we’re special?
● Compare the pitches for two products. Which pieces of information resonated
the most?
Example: Videos
● How does Generation Z resonate to ads from Apple, Samsung, and
Google?
FYI, Salt Bae
● Turkish chef, bounces
salt off his elbow
Example: Images
● How do customers feel about images for a travel catalog?
Case Study: Rebranding
• Online education company
• 40+ live video interviews recruited and scheduled
through UserTesting
• Showed copy, images, logos, colors, tagline.
• Probed for reactions and emotional tone.
• Replaced live interviews conducted in their office
• Benefits
• Done in days rather than weeks, scheduling
and recruiting were automated, national sample
instead of local, found their target
demographics easily
“I basically sat
in our phone
booth and
talked”
Recent Studies: Attitudes and Journeys
● What are perceptions of calories shown on a fast food menu?
● Which of these brands would you trust for making a large purchase?
● What brands do people expect to see in our store? Why?
● How do participants research Medicare plans?
● What’s the customer journey for choosing a hotel?
● How do parents initiate the process of finding a toy?
Case Study: Automobile Buyer
Journey
● How they did it in the past: On-site and phone interviews
○ On-site has local bias, Phone interviews are often distracted
○ Slow to schedule, difficult to recruit
● New way: Interviews through UserTesting
○ Everything from retailer impressions to how they researched
○ Easy recruiting: National sample, range of ages and income levels
○ Time needed: Four weeks the old way, a couple of days with UserTesting
● Key finding: Online influencers much less influential than
expected
○ Reworking website to make it search-friendly at any point in the buying
cycle
“I’ve been
telling people
this for so long
now, and to
have you back
this up with
qualitative
data is
fantastic.”
–Head of SEO
Case Study: Selling to Millennials
● Major clothing brand / retailer
● Wanted to understand Millennials:
○ Their clothing shopping process, brand impressions, reactions to online and offline
marketing content, comparison to a key competitor
● Key insights:
○ The company’s email messages were clear and communicated value well
○ Brand awareness was lower than competition
○ Many customers used the shopping area in search engines to browse pictures, and
the brand was not present there
○ Many customers felt the brand’s models were too skinny, meaning the clothes
would not look good on them
○ The competition’s video of people moving around in its clothing was very well
received
Case Study: Starter Offer
● Consumer goods company
○ Products sold through retail and by subscription on its own website
● Wanted to understand online reaction to first-time buyer offer
○ It was free. Could we charge for it?
○ What should we call it?
○ Ran user tests on mockup offers with various wording and prices
● Key insights:
○ Online first-time customers were expecting to pay something
■ The offer didn’t need to be free
○ Show the value (“A $20 value for $10”)
○ Call it a “starter” not a “trial” (trial = free)
Recent Studies: A/B Pre-Screening
● Home page for a major desktop app: Understanding of the offer and pricing
● Store locator page: Customer impressions and preferences
● Video playback page: Understanding of labels, and general preferences
● Webpage for home builder: Understanding of offer
● Webpage for an online communication service: General impressions
Agenda
● How UserTesting works
● Real-world examples of how marketing teams use it
○ Validating content and messages
○ Discovering quick insights about attitudes and journeys
○ Improving A/B tests
● How we used it for this webinar
● Tips for your own studies
● Q&A
User Tests and This Webinar
● How do marketers react to four possible titles?
○ Marketing and the Instant Insight Revolution
○ The Revolution in Agile Marketing
○ Fast Human Insight for Marketing
○ Three Ways Fast Human Insight is Revolutionizing Marketing (winner)
● Do they resonate to the webinar description?
○ Do they understand the message?
○ Are we answering all their questions?
○ What words work best? Validate, agile, resonate…
○ Is our language over the top?
Feedback
● Eight videos, focused on full-time marketers, completed within 2.5 hours
What We Learned
● The core message resonated
● Marketers love bullets (but not numbered)
● Nothing was universally loved, not even “fast”
● The “revolution” rhetoric didn’t irritate people the way we feared
● “Agile marketing” is an attractive subject to some people, but not
as the title for this webinar
● Mention that the webinar is free
Agenda
● How UserTesting works
● Real-world examples of how marketing teams use it
○ Validating content and messages
○ Discovering quick insights about attitudes and journeys
○ Improving A/B tests
● How we used it for this webinar
● Tips for your own studies
● Q&A
Tip (1 of 2)
● Do a website test, and host the material on a sharing service
○ Make sure the document is accessible to anyone with the link
○ Can do this with any file type, not just text
● Tell them it’s not a usability test
● Use a screener that’s hard to guess
Please choose the response that best describes your work status:
•I am not currently employed [Reject]
•I am a full time student [Reject]
•I am employed part time [Reject]
•I am employed full time in an engineering role [Reject]
•I am employed full time in a marketing role [Accept]
•I am employed full time in a sales role [Reject]
•I am employed full time in an administrative role [Reject]
•I am employed full time in a support role [Reject]
•I am employed full time in a management role [Reject]
•I am employed full time in an information technology role [Reject]
•I am employed full time in a role not listed above [Reject]
Tip (2 of 2)
● Get them talking
○ No: “Do you like this?”
○ Yes:
■ “What do you like or dislike about this, and why?”
■ “Would you want to learn more? Why or why not?”
■ “What would you want to do/learn next?”
● Don’t expect consensus
● Don’t turn it into a poll
○ Focus on how people think what are their hot buttons
Summary: No More Guessing
● Quickly validate marketing content
and concepts
○ “Does it resonate with the customer?”
● Quick insight on attitudes and
journeys
○ “Go into your closet and show me your
favorite outfits.”
● A/B alternate testing
● Video evidence to explain decisions
● All at the speed of digital business
“You may think I’m
confused about what
Millennials want, but
you won’t argue with
video of them saying it.”
--VP Marketing, real estate firm
THANK
YOU!
For more information please contact us: webinars@usertesting.com

Three Ways Fast Human Insight is Revolutionizing Marketing

  • 1.
    Fast Human Insight For Marketing VP,Product Marketing Michael Mace
  • 2.
    Summary ● We’re theleading provider of ultra-fast user feedback on websites and apps ○ Feedback from real customers within two hours ○ Usability, navigation, etc. ○ 10 years old, 50k tests a month ● Rising adoption by marketers: ~20% of our volume ○ Quick reactions to messages and material when there’s no time/budget for traditional research ○ Shareable evidence of customer attitudes and reactions ○ Eliminates guessing ○ Far faster than traditional research, national sample, easy for anyone in team to use ● Goal for today: Describe how marketers use us, with real-world examples ○ Tips for your own studies
  • 3.
    Agenda ● How UserTestingworks ● Real-world examples of how marketing teams use it ○ Validating content and messages ○ Discovering quick insights about attitudes and journeys ○ Improving A/B tests ● How we used it for this webinar ● Tips for your own studies ● Q&A
  • 4.
    What We Do ●Human insights at the speed of digital business ○ In two hours, video and audio feedback from customers on almost anything ○ Target your particular customer ○ So easy that anyone in your team can use it
  • 5.
  • 6.
    How it Works Target •Panel with demographic filters, screeners • Or invite your own customers, partners Discover • Recorded studies: Via desktop and mobile. List questions and tasks. Two hour turnaround. • Live video interviews: Schedule and recruit in hours Understand • Jump to key video insights via survey questions and transcript • Click and drag to make clips • Share instantly via email and Slack
  • 7.
    Agenda ● How UserTestingworks ● Real-world examples of how marketing teams use it ○ Validating content and messages ○ Discovering quick insights about attitudes and journeys ○ Improving A/B tests ● How we used it for this webinar ● Tips for your own studies ● Q&A
  • 8.
    How We’re Usedin Marketing “There is no data for the short-term decisions I have to make today. The competition is not quant vs. qual. It's qual vs. gut instinct. The way you could help me is by providing data – customer feedback – where it was just opinion before.” --VP Global Marketing, F500 manufacturer ● Validate: Fast reactions to marketing materials and messages ○ Confirm whether it resonates ○ No more guessing ○ Persuasive video evidence ● Discover: Quick insight on customer attitudes, needs, and journeys ● Iterate: Improve your A/B tests ○ Increase your win rate by pre-testing alternates
  • 9.
    Recent Studies: Messagingand Material ● Which image is most trustworthy and compelling? ● Which email content resonates best with participants? ● How well do customers understand the vocabulary we use? ● What message did the ad convey? ● How effective are we at communicating who we are and why we’re special? ● Compare the pitches for two products. Which pieces of information resonated the most?
  • 10.
    Example: Videos ● Howdoes Generation Z resonate to ads from Apple, Samsung, and Google?
  • 11.
    FYI, Salt Bae ●Turkish chef, bounces salt off his elbow
  • 12.
    Example: Images ● Howdo customers feel about images for a travel catalog?
  • 13.
    Case Study: Rebranding •Online education company • 40+ live video interviews recruited and scheduled through UserTesting • Showed copy, images, logos, colors, tagline. • Probed for reactions and emotional tone. • Replaced live interviews conducted in their office • Benefits • Done in days rather than weeks, scheduling and recruiting were automated, national sample instead of local, found their target demographics easily “I basically sat in our phone booth and talked”
  • 14.
    Recent Studies: Attitudesand Journeys ● What are perceptions of calories shown on a fast food menu? ● Which of these brands would you trust for making a large purchase? ● What brands do people expect to see in our store? Why? ● How do participants research Medicare plans? ● What’s the customer journey for choosing a hotel? ● How do parents initiate the process of finding a toy?
  • 15.
    Case Study: AutomobileBuyer Journey ● How they did it in the past: On-site and phone interviews ○ On-site has local bias, Phone interviews are often distracted ○ Slow to schedule, difficult to recruit ● New way: Interviews through UserTesting ○ Everything from retailer impressions to how they researched ○ Easy recruiting: National sample, range of ages and income levels ○ Time needed: Four weeks the old way, a couple of days with UserTesting ● Key finding: Online influencers much less influential than expected ○ Reworking website to make it search-friendly at any point in the buying cycle “I’ve been telling people this for so long now, and to have you back this up with qualitative data is fantastic.” –Head of SEO
  • 16.
    Case Study: Sellingto Millennials ● Major clothing brand / retailer ● Wanted to understand Millennials: ○ Their clothing shopping process, brand impressions, reactions to online and offline marketing content, comparison to a key competitor ● Key insights: ○ The company’s email messages were clear and communicated value well ○ Brand awareness was lower than competition ○ Many customers used the shopping area in search engines to browse pictures, and the brand was not present there ○ Many customers felt the brand’s models were too skinny, meaning the clothes would not look good on them ○ The competition’s video of people moving around in its clothing was very well received
  • 17.
    Case Study: StarterOffer ● Consumer goods company ○ Products sold through retail and by subscription on its own website ● Wanted to understand online reaction to first-time buyer offer ○ It was free. Could we charge for it? ○ What should we call it? ○ Ran user tests on mockup offers with various wording and prices ● Key insights: ○ Online first-time customers were expecting to pay something ■ The offer didn’t need to be free ○ Show the value (“A $20 value for $10”) ○ Call it a “starter” not a “trial” (trial = free)
  • 18.
    Recent Studies: A/BPre-Screening ● Home page for a major desktop app: Understanding of the offer and pricing ● Store locator page: Customer impressions and preferences ● Video playback page: Understanding of labels, and general preferences ● Webpage for home builder: Understanding of offer ● Webpage for an online communication service: General impressions
  • 19.
    Agenda ● How UserTestingworks ● Real-world examples of how marketing teams use it ○ Validating content and messages ○ Discovering quick insights about attitudes and journeys ○ Improving A/B tests ● How we used it for this webinar ● Tips for your own studies ● Q&A
  • 20.
    User Tests andThis Webinar ● How do marketers react to four possible titles? ○ Marketing and the Instant Insight Revolution ○ The Revolution in Agile Marketing ○ Fast Human Insight for Marketing ○ Three Ways Fast Human Insight is Revolutionizing Marketing (winner) ● Do they resonate to the webinar description? ○ Do they understand the message? ○ Are we answering all their questions? ○ What words work best? Validate, agile, resonate… ○ Is our language over the top?
  • 21.
    Feedback ● Eight videos,focused on full-time marketers, completed within 2.5 hours
  • 22.
    What We Learned ●The core message resonated ● Marketers love bullets (but not numbered) ● Nothing was universally loved, not even “fast” ● The “revolution” rhetoric didn’t irritate people the way we feared ● “Agile marketing” is an attractive subject to some people, but not as the title for this webinar ● Mention that the webinar is free
  • 23.
    Agenda ● How UserTestingworks ● Real-world examples of how marketing teams use it ○ Validating content and messages ○ Discovering quick insights about attitudes and journeys ○ Improving A/B tests ● How we used it for this webinar ● Tips for your own studies ● Q&A
  • 24.
    Tip (1 of2) ● Do a website test, and host the material on a sharing service ○ Make sure the document is accessible to anyone with the link ○ Can do this with any file type, not just text ● Tell them it’s not a usability test ● Use a screener that’s hard to guess Please choose the response that best describes your work status: •I am not currently employed [Reject] •I am a full time student [Reject] •I am employed part time [Reject] •I am employed full time in an engineering role [Reject] •I am employed full time in a marketing role [Accept] •I am employed full time in a sales role [Reject] •I am employed full time in an administrative role [Reject] •I am employed full time in a support role [Reject] •I am employed full time in a management role [Reject] •I am employed full time in an information technology role [Reject] •I am employed full time in a role not listed above [Reject]
  • 25.
    Tip (2 of2) ● Get them talking ○ No: “Do you like this?” ○ Yes: ■ “What do you like or dislike about this, and why?” ■ “Would you want to learn more? Why or why not?” ■ “What would you want to do/learn next?” ● Don’t expect consensus ● Don’t turn it into a poll ○ Focus on how people think what are their hot buttons
  • 26.
    Summary: No MoreGuessing ● Quickly validate marketing content and concepts ○ “Does it resonate with the customer?” ● Quick insight on attitudes and journeys ○ “Go into your closet and show me your favorite outfits.” ● A/B alternate testing ● Video evidence to explain decisions ● All at the speed of digital business “You may think I’m confused about what Millennials want, but you won’t argue with video of them saying it.” --VP Marketing, real estate firm
  • 27.

Editor's Notes

  • #9 Our most common use in marketing is to validate messages and draft deliverables when you don’t have time or money to do formal market research on them. Many marketing teams have to make quick decisions on things like the text and title for an email or the cover image for a catalog. Usually these decisions are made on gut instinct, giving you a high risk of mistakes or missed opportunities. The choices you make are also hard to explain if others question them, because there’s no data to back them up. With user tests, you can get feedback from real customers in hours, increasing your confidence in your decisions and giving you video evidence you can use to explain them. We’re also frequently used to get quick insights on the ways customers think and behave. You can very quickly understand purchasing journeys and customer attitudes on any subject. This lets you move much faster at the start of a project, with confidence that you have a good understanding of the problem to solve. Marketing teams also often use us to improve the quality of their A/B testing and other analytics. The sad truth of A/B testing is that more than half of the tests fail to produce a statistically significant winner. The failure rate is even higher in retail and consumer tech companies. That means a lot of wasted time and frustrated team members. The solution is to pre-screen variants before starting the test via user tests. You can Identify and fix anything that confuses customers, you learn how customers are thinking and adjust variants accordingly, and you can select the most effective variants to test. All of this increases likelihood that you’ll get a statistically significant winner. (Note: The statistic on failure rate of AB studies came from our CX survey, in which we asked about 3,900 respondents what their failure rate is for AB tests. Failure means the AB test did not produce a statistically significant winner. This forces rework while the company tests additional variants. Depending on the volume of traffic to your site, the time loss can be extreme.)
  • #16 Autotrader wanted to understand the car buyer journey, and how its website could best fit into it. In the past, it did this sort of research through a mix of telephone and on-site interviews. But that process was difficult: On-site interviews were subject to local bias because the company could only afford to do interviews at its headquarters in California. The company does have an interview facility at its headquarters, but doesn’t have the staff to fully operate it. That made the process slow and difficult. Phone interviews were problematic because they found that the participants were often distracted, doing interviews while eating or driving. For its latest journey study, the company used UserTesting’s Live Conversation service to interview recent car buyers They interviewed customers about everything from their thoughts on retailers to how they researched a car purchase. Through UserTesting, they were able to recruit nationally, and at a variety of ages and income levels. The process was much faster. They were able to complete the interviews in a few days, as compared to four weeks using traditional interviews. The key finding: The customer journey doesn’t work the way they expected. They thought people started with online experts like Autotrader. Instead, it turns out that most people start the car buying process by researching brands through search engines. Autotrader realized that it needed to restructure its entire site to make the content more search-friendly, so it could intercept customers at any point in the purchasing journey. The quote is from the head of SEO at Autotrader.
  • #17 A major casual clothing brand wanted to explore where it stands with Millennial shoppers: Perceptions of its brand, reactions to marketing content on the web and in email, how the shopping process works, and how it all stacks up against its leading competitor. Through a series of UserTesting sessions, the company learned: There was relatively low top of mind awareness and understanding of its brand compared to the competition. The shopping process for many customers included visiting the shopping area in a search engine, to compare images from multiple brands. The company’s products were not well represented here, interfering with its ability to attract new customers. When people viewed online images of clothing, they were often trying to figure out how the clothing would look on them. Although people found the company’s product images to be attractive, they felt many of the models were unrealistically thin, and worried that the clothing would not look good on themselves. The company’s email messages worked well. They were to the point and communicated product value clearly. One aspect of the competition’s marketing that stood out: It offers videos of normal-looking people wearing and moving around in its clothing. This was very appealing to the customers.
  • #18 A consumer goods company that sells through its own website and through retailers wanted to understand the value of its free starter offer. To new customers online, the company gave a free starter pack. The same pack was selling well at retail, causing the company to ask if it was leaving money on the table online. Could it charge for the starter pack it had been giving away online? The company ran user tests on online mockups showing varying wording and prices for the starter pack. The results showed that: Online customers came to its site expecting to pay something. A free offer wasn’t necessary; a discount would work just as well. It’s important to show the details of the discount (for example, “a $20 value for $10”) It was important to label the offer a “starter” and not a “trial” offer. Trial means free to many customers. videos of normal-looking people wearing and moving around in its clothing. This was very appealing to the customers.