Mobile Research Goes To The Game - Paper

Mobile Research Goes To The Game - Paper

• 1.
  Mobile research goesto the game In the scrimmage moment with mobile? James Burge, Managing Director, Asia-Pacific, Research Now Sally Joubert, CEO, Luma Research Georgia Phillips, Client Service Director, Luma Research Introduction The rapid advances in technology over the past decade have changed the way we interact. Whether it is on a personal level or the interaction between brands and consumers, technology has opened up so many more opportunities to reach out to each other. One of the major advances has been the rise in the use and the capability of mobile phones and this is particularly evident in the Asia Pacific region. APAC has the highest percentage of smartphones globally (76%)1 and more APAC users turn on their mobile device throughout the day than the global norm2 . And this is on the rise with two-thirds (68%) of APAC users agree that compared to 12 months ago, they now use their mobile phone for more activities other than just making calls and texting. Again this is above the global average of 63%3 . These technological advances have had a major impact on the environment for brands. Advertising is everywhere from traditional mass market TV, sport sponsorships, mobile, digital, to fully immersed brand experiences. As consumers we have even shorter attention spans so the competition to really engage is becoming increasingly more difficult. Brands have so many different channels they can use but how do we know what will work best? Businesses are seeking to answer this in order to facilitate advertising and media spend decisions. Importantly they need to know this quickly and ahead of their competitors. Gone are the days when businesses could afford to spend half of the research budget on long, cumbersome tracking studies which might yield some interesting findings on a small number of the channels weeks after the event. 2 http://www.sap.com/bin/sapcom/downloadasset.The%20Mobile%20Consumer%20In sights%20on%20Asia%20Pacific%20Trends%20Impacting%20Mobile%20Momentu m%20and%20Customer%20Engagement-pdf.bypassReg.html 3 http://www.sap.com/bin/sapcom/downloadasset.The%20Mobile%20Consumer%20In sights%20on%20Asia%20Pacific%20Trends%20Impacting%20Mobile%20Momentu m%20and%20Customer%20Engagement-pdf.bypassReg.html
• 2.
  A few yearsago Luma and Research Now saw this as an opportunity for mobile research and started investigating what it could offer in this space. Luma listened to the concerns of their clients and is re-inventing advertising research to meet the needs of business managers in this new environment. Research Now has lead the way in mobile research technology by building and perfecting a best in class mobile platform, enabling researchers and advertisers to better tap into the mobile audience. This paper shares the story of our mobile research evolution from the early stages and challenges through to the current state of play and opportunities. We begin by summarizing our initial early trials and then focus on how we have applied and what we have learnt in two highly competitive advertising spaces. The first is the State of Origin Rugby Series in Australia and the second is the world’s most expensive advertising platform the Super Bowl 2014 in USA. We hope our findings spark questions and ideas within the industry about how mobile research can deliver richer and more relevant insights for the future. Early Testing In November 2011, Luma and Research Now started parallel tests to determine whether the online ad testing methodology add+impact® could be successfully adapted to a mobile platform. The early testing explored the feasibility, practicality, reliability and value of mobile ad testing to find out whether the data collection is as effective and reliable on mobile as it is using an online format. The testing compared the collection of key advertising research effectiveness measures (Attention to the creative and Bonding to the brand) across the two platforms in both Australian and the UK4 . This initial study used static advertising and found remarkable consistency across all the key Brand Bonding measures which evaluate how people feel about the brand after exposure to the advertising campaign (see Figure 1). The response and participation rates in this initial study showed that mobile was a great way to reach people during a variety of activities through the day and night - 56% of the mobile surveys were done out of working hours compared to 21% online. Figure 1 – Brand Bonding Measures Across Platform 4 Research Goes Mobile – Findings from Initial Smartphone Applications Research C. Dubreuil and S Joubert ESOMAR 3D Digital Dimensions Amsterdam 2012
• 3.
  The Attention measureswhich evaluate how people feel about the creative itself revealed some differences in the level of cut-through for the static ads across the two platforms. At the time it was concluded that this could have been due to the different screen sizes, the potential distractions from being ‘on the go’ as opposed to sitting at a computer screen or simply a result of the type of ad itself. To investigate this further we completed a second round of testing in 2012 using video advertising for Kleenex Toilet Tissue and McDonalds. This next round showed consistency across both key ad effectiveness measures suggesting that the differences in the initial tests related to the type of ad rather than the platform itself. See figure 2 below. Figure 2 Attention and Bonding Scores (Phase Two) These results were very encouraging and suggested that utlilising smartphone as a data collection platform does not have a direct impact on key advertising effectiveness measures and can therefore be used alongside traditional online surveys. This opens up enormous opportunity to capitalize on the advantages of mobile as a research tool whilst still ensuring continuity with traditional measures. Potentially mobile offers more cost-effective advertising research because you can reach people more quickly and easily. Next we wanted to explore how to make it even better.
• 4.
  Going to theGame… To do this we have put mobile during two major sporting events where people are typically hard to reach yet the need for quicker, reliable feedback is strong. Global Sponsorship investment is significant (53.2 billion US dollars in 20135 ), as are the opportunities, and businesses need to know quickly whether it is worth it and how to make it better. In 2013 in Australia we chose the State of Origin Rugby Series in Australia – it is the biggest, most watched Australia sports series. As researchers we wanted to see whether mobile could deliver this and importantly what it could offer over traditional pre-post online studies. Specifically: • Could we really interview avid sports fans in the moment? • What could mobile add to traditional research? • Does mobile improve the survey experience? State of Origin 2013 - Australia Method Luma and Research Now undertook a five-part study to conduct our research into mobile sponsorship research in August 2013. We utilised a traditional pre and post online survey and a short mobile survey during each game to track any changes in brand feelings and perceptions. We recruited N=431 respondents who were considering watching or attending the State of Origin series. The composition of the sample surveyed remained reasonably consistent across the three games in terms of location, age, team supporting and passion for Rugby League, meaning there was very minimal sample skew when we moved from the traditional PC-based survey to the mobile survey. What we found Interviewing in the Moment Our study confirmed that we can interview “in the moment” with even the most avid sporting fans. Participation rates and respondent engagement were high. 69% who undertook the initial online survey participated in at least one of the mobile surveys. This is higher than we would expect for a standard repeat survey. What was even more encouraging is that 69% of respondents who participated in Game One also participated in Game Two and 76% in Game Three. 5 http://www.statista.com/statistics/196898/global-sponsorship-spending-by-region- since-2009/
• 5.
  The next interestingfinding related to speed. Across the three games an average of over 60% of interviews were completed by the end of the first half of the game which again is faster than traditional online surveys. We were also encouraged by the level and depth of these mobile respondents’ engagement – 38% uploaded a photo in Game One and 34% in Game Two. Given that these were avid fans watching the game they love, this level of participation was astonishing. Figure 3 shows the variety and quality of the pictures we received: Figure 3 A Cross Section of Photos Uploaded for the State of Origin Mobile Survey Source: State of Origin 2013, Mobile Research Goes to the Game, Research Now & Luma Research 2013 What Can Mobile add to traditional research? The results from the traditional pre-post surveys on their own provided very little clue as to how fans were feeling about the brands and whether or not the sponsorship drove a positive return on investment (see Figure 4).
• 6.
  Figure 4 Brand FeelingsPre and Post State of Origin Source: State of Origin 2013, Mobile Research Goes to the Game, Research Now & Luma Research 2013 With these results alone you could have been excused for thinking that sponsorship does little for the brands, but this was not the whole story. While this traditional pre-post approach was useful in identifying differences in feelings toward the sponsors, for example our strong rugby league fans were much more positive towards all the State of Origin sponsors than the non-fans, we wanted to know more. What else was happening? Were we missing something? The answer was “yes”. Whilst we saw an overall decline in brand feelings when comparing pre and post results, the mobile game data provided additional insight that would have been missed with the standard approach. For instance, we found that feelings toward Holden remained consistent before, during and after the series. However, both GIO and AAMI Insurers were generally lower during play itself (See Figure 5). Our ‘QLD’ Rugby team supporters loved XXXX so much more than the ‘NSW’ supporters during Games 2 & 3, notably when they were winning (See Figure 6). The level of initial data that mobile was able to deliver opened up a wealth of valuable further research that could be explored in any manner of ways to ascertain the exact reasons for these significant findings. For us the key takeout simply put, was we would not have known any of this if we had just stuck the traditional survey method. Figure 5 Brand Feelings Pre, During and after the Games
• 7.
  Source: State ofOrigin 2013, Mobile Research Goes to the Game, Research Now & Luma Research 2013 Figure 6 XXXX Brand Feelings by Team Supporters Source: State of Origin 2013, Mobile Research Goes to the Game, Research Now & Luma Research 2013 Team NSW Team QLD Pre-game Game 1 Game 2 Game 3
• 8.
  Could Mobile addto the respondents’ experience? We know that shorter and more engaging surveys are going to win over the hearts and minds of respondents. So next we looked at whether we could replace wordy questions by utlizing the mobile phones other features like photo uploading. Specifically we looked at whether common ‘feelings’ questions (i.e. Which of the following words best describes how you feel about this State of Origin game?...) with a simple image analysis from the photo uploads. We were interested in this line of questioning as traditionally they have been fairly limited. We know that people often cannot express how they feel or their answers to questions like these reflect only a considered and rational response. Not surprisingly we found that our rugby fans generally claimed they were happy, excited and positive. When we scanned the uploaded images we came to the same conclusions by simply and quickly observing the images (See Figure 7). Given the willingness to participate in this survey and to upload photos this approach is certainly worth considering. Figure 7 Feelings About the Game versus Image Analysis Source: State of Origin 2013, Mobile Research Goes to the Game, Research Now & Luma Research 2013 The video uploads also added another layer of analysis and depth to the findings. We asked respondents to upload a video of themselves talking about sponsorship. The depth and diversity of the clips was incredible. One of the key themes that emerged was the desire to find a sponsor who promoted healthy eating. Many felt that alcohol sponsors would be better balanced or replaced with brands that linked to sport and health, be it healthy food, exercise services or sport itself. Even those drinking a beer shared this same sentiment!
• 9.
  Overall our Stateof Origin Study showed us that mobile can help us deliver not only faster and potentially cheaper but also better research: • You can interview avid sports fans ‘in the moment’ • Mobile can add speed and depth when compared to a traditional approach • Mobile (and all it’s features) can potentially add to respondents positive experience and help to reduce survey length by asking less questions Super Bowl 2014 - USA This year we took this to the next level to see whether mobile could actually replace some of the research we were doing? We chose the Super Bowl in the US as it is arguably the most sort after advertising space being watched by over 111 million on TV and over 500,000 online. Here the stakes are high with marketers shelling out over $4million for a 30-second ad and this is just the starting point. The majority take a multi-channel approach to maximize their investment. Some ads are pre-released, some use teasers to build expectation and others used crowd sourcing and pre-game competitions to maximize engagement. Not surprisingly this event attracts the interest of the global advertising and research industries and as a result the ads are possibly the most researched/talked about ads in the world. Given their Super Bowl 2014 investment advertisers want to know whether their ads are effective and how they can drive post game viewing. We all know about the amazing Super Bowl advertising success stories that have become almost folklore – the apple ad in 1984… but what about the rest? Dozens of polls have tried to track the best and worst performers and interestingly there are some ads which feature on all the tables and others that don’t. We find Super Bowl advertising fascinating and want to find out what mobile can offer in this space. Specifically: • Do the findings from Rugby in Australia translate to Grid Iron in the US i.e. Can we really interview avid sports fans in the moment? • Can we use mobile to do smarter advertising research i.e. can we use quick in the moment mobile surveys to replace some of the cumbersome tracking questions? Method Our Super Bowl 2014 Study had two parts, one mobile and one online. Stage One was a short three minute mobile survey designed to capture ‘in the moment’ feelings about the Super Bowl ads (cut-through, memorability and sharing). We pre-recruited fans who were planning on watching the game and were willing to participate in a short mobile survey. Just as the game finished they were sent the mobile survey and within the two hour window that followed we had received n=500 completes. The majority of the sample were keen sports viewers (47%) or occasional sports viewers (39%).
• 10.
  Using the resultsfrom Stage One we developed a follow up tracking study to measure the effectiveness of 50 of the ads and to look at 20 of the most polarizing ads in detail (Stage Two). A nationally representative sample of n=1000 sourced from the Research Now panel completed the twenty minute online survey. The questionnaire covered traditional advertising research measures recognition and brand link and key add+impact® creative effectiveness measures. What we found Interviewing in the Moment This study again confirmed that mobile gets you to the game. We achieved a 45% response rate in the two hour window after the game which, although based on a pre-recruited sample in this example, is significantly quicker than the typical 24 hour window we would see for online. While the survey timing was different to the State of Origin it was encouraging to see that fans are willing to participate in mobile research when they are ‘in the moment’. Stage One – Our Super Quick Super Bowl Survey With only a two hour window it was amazing how quickly we obtained the ‘in the moment’ feelings about the ads. By Monday morning we had our own Super Quick results to over 50 ads. Firstly we could see which ads were generating the strongest positive feelings (See Figure 8). We know from our previous work that brand feelings are the strongest predictor of purchase intent7 so we are calling this our Top 10. Figure 8 Super Bowl Top 10 Luma’s Super Quick Super Bowl Top 10 1 Budweiser ‘Puppy Love’ 2 Budweiser ‘Hero’ 3 Coke America ‘America the Beautiful’ 4 Doritos ‘Time Machine’ 5 Cheerios ‘Gracie’ 6 Radio Shack ‘Phone Call’ 7 Chrysler ‘America’s Import’ 8 Wonderful Pistachios ‘Wonderful Pistachios’ 9 Bud Light ‘Epic Night’ 10 Dannon Oikos ‘Spill’ Source: Luma Research Now Mobile Study Feb 2014 Which ad made you feel the best? 7 Cramphorn, M and Phillips, G Global Advertising that Sells. ESOMAR Congress Cannes 2005
• 11.
  We also collectedconsumer predictions of the ads that would be most remembered and most likely to be shared (see Figure 9 and 10) Figure 9 Consumer Predictions of the Most Memorable Super Bowl Ads Consumer Predictions of the Most Memorable Super Bowl Ads 1 Budweiser ‘Puppy Love’ 2 Budweiser ‘Hero’ 3 Bud Light ‘Epic Night’ 4 Audi ‘Doberhuahua’ 5 Doritos ‘Time Machine’ 6 Coke ‘America the Beautiful’ 7 VW ‘Wings’ 8 Chrysler ‘America’s Import’ 9 Toyota ‘Epic Night’ 10 Wonderful Pistachios ‘Wonderful Pistachios’ Source: Luma Research Now Mobile Study Feb 2014 Figure 10 Consumer Predictions of the Most Shared Super Bowl Ads Consumer Predictions of the Most Shared Super Bowl Ads 1 Budweiser ‘Puppy Love’ 2 Budweiser ‘Epic Night’ 3 Audi ‘Doberhuahua’ 4 Doritos ‘Time Machine’ 5 Budweiser ‘Hero’ 6 Chrysler ‘America’s Import’ 7 VW ‘Wings’ 8 Coke America ‘America the Beautiful’ 9 Radio Shack ‘Phone call’ 10 Wonderful Pistachios ‘Wonderful Pistachios’ Source: Luma Research Now Mobile Study Feb 2014
• 12.
  From our shortthree minute mobile survey we found out a lot about the ads and here are a few interesting findings: • Budweiser “Puppy Love” was the overall “winner”. It was the strongest ad in terms of generating the most positive feelings and consumers predicted it would be the most memorable and shared ad. • Budweiser “Hero” ad was also a strong performer featuring second on Luma’s Top Ten and also predicted to be remembered and to a less extent shared. • The third Budweiser ad “Epic Night” for Bud Light didn’t perform in quite the same way – it didn’t feature on the Luma’s Top 10 but it was predicted to be remembered and shared. • Interestingly, the ads for Cheerios, Toyota and Dannon Oikos also appeared to work in a different way – generating strong feelings but not necessarily predicted to be remembered or shared. And the list goes on but the main point here is that we were able to capture a lot of feedback in a short space of time. However, it was important for us to know whether these findings are useful and would predict what eventually happened. To do this we looked at how the findings from our mobile study compared to the full detailed online study we conducted post the Super Bowl and to other published polls/surveys conducted by others. Specifically; • How does our Top Ten compare to others? • Could our mobile audience predict what ads were remembered? • Did our audience predictions of sharing actually play out in market? How Does Our Top Ten Perform relative to others? Our initial results are encouraging. With our one simple question we predicted: • Four out of the top five of the Super Bowl Ad Meter Results (published in USA Today) which are based on thousands of voters. This Ad Meter is regarded as the ‘holy grail’ of Super Bowl Polls – many KPIs are won or lost on this basis of this list alone. • Five out of the top six of the Super Bowl add+impact® Commitment Results. This is the key effectiveness score from the add+impact® ad test which is used in over 50 countries globally. The Commitment score is calculated from two key measures attention to the ad and bonding to the brand and is continually validated with in market performance.
• 13.
  Figure 11 Ad Meterand add+impact® Top Ads Tables Could our mobile audience predict what ads were remembered? At this stage it looks like this is not as simple and may not be something that is as easily predicted. • We found almost no relationship between our audience predictions and the recognition scores we obtained from the large scale follow up tracker See Figure 12
• 14.
  Figure 12 add+impact® Recognitionversus Predicted Top Ten Consumer Predictions of the Most Memorable Super Bowl Ads Recognition from follow up online survey 1 Budweiser ‘Puppy Love’ Puppy Love 81% 2 Budweiser ‘Hero’ Geico ‘Pig’ 79% 3 Bud Light ‘Epic Night’ Cheerios ‘Gracie’ 74% 4 Audi ‘Doberhuahua’ Go Daddy ‘Body Builder’ 67% 5 Doritos ‘Time Machine’ VW ‘Wings’ 64% 6 Coke ‘America the Beautiful’ H&M ‘David Beckham’ 64% 7 VW ‘Wings’ Honda ‘Hugfest’ 63% 8 Chrysler ‘America’s Import’ Subway ‘Crunch Time’ 61% 9 Doritos ‘Cowboy’ Doritos ‘Cowboy’ 60% 10 Wonderful Pistachios Toyota ‘Muppets’ 57% However, it is worth noting that there was actually very little relationship between Recognition and the Super Bowl Ad meter score or our other measures of ad effectiveness. Did our audience predictions of sharing play out in market? At the time of writing this paper it looks like this may also not be as straightforward and we are still investigating to see whether consumers can predict this: • According to Unruly Media the top most shared ads of the Super Bowl 2014 were: o Budweiser ‘Puppy Love’ 1,309,403 shares o Budweiser ‘Hero’ 202,556 shares o Axe ‘Make Love, Not War’ 136,282 shares o Jaguar ‘British Villians’ 125, 896 shares o Bud Light ‘Epic Night’ 123,418 shares While consumers predicted the Budweiser ads, AXE and Jaguar were not predicted from our quick mobile survey.
• 15.
  To sum up…. Thankyou for listening to our story. We recognize that this paper is not conclusive and there are many more things we can explore in terms of the impact of mobile on market research. However, we hope these examples inspire you with the doors mobile can open. With sufficient groundwork on the technological capabilities and smartphone penetration, mobile data collection offers a relevant and convenient means of participation for respondents. Furthermore, the affinity people feel with their mobile device demonstrates a clear opportunity for more detailed feedback and involvement in survey responses. People are willing to share even when they are in the middle of a major ‘sporting moment’ and it looks encouraging that mobile will help us to do better research. By daring to combine this with the advertising research process, brands and businesses can benefit from faster, better and cheaper way of harnessing their audiences’ perceptions and motivations.