Download as PDF, PPTX
Uploaded byManuel Da Costa
The sensationalisation of A/B Testing & Why it doesn't work for you - Measurefest October 2014 talk
The document discusses the pitfalls of A/B testing, emphasizing its sensationalization and the need for a structured framework to yield useful results. Key elements include identifying customer insights, defining test variables, and ensuring proper sample size and duration for tests. It highlights the importance of reviewing and iterating on tests to optimize conversions and user behavior.
More Related Content
![[CXL Live 16] How to Utilize Your Test Capacity? by Ton Wesseling](https://cdn.slidesharecdn.com/ss_thumbnails/it6fm2aurnem3mvoaaxu-signature-42db88fde2139044dd6963018b134da7e709739587431e34ce8d4285185c8547-poli-160403200010-thumbnail.jpg?width=640&height=640&fit=bounds)
![[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...](https://cdn.slidesharecdn.com/ss_thumbnails/abt3qhhxq3o7ouyqy5e6-signature-42db88fde2139044dd6963018b134da7e709739587431e34ce8d4285185c8547-poli-160403200010-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to The sensationalisation of A/B Testing & Why it doesn't work for you - Measurefest October 2014 talk
The sensationalisation of A/B Testing & Why it doesn't work for you - Measurefest October 2014 talk
- 1. The Sensationalisation ofA/B Testing And why it doesn’t work for you #MEASUREFEST 8th October 2014
- 2. Manuel da Costa @digital_tonic DIGITAL TONIC
- 3. We have aproblem with A/B testing #MEASUREFEST
- 4. Let’s see howgood you are at A/B testing #MEASUREFEST
- 5.
- 6.
- 7. VARIATION #MEASUREFEST 25%increase in conversions
- 8.
- 9.
- 10. #MEASUREFEST Clickthroughs 606%and Purchases 11%
- 11.
- 14.
- 15. What we don’tsee What was the sample size? How long did they run the test for? Significance of results? What was the target segment for the test? Did they re-test ?
- 16. THROWING crap againstthe wall hoping it sticks #MEASUREFEST
- 17. #MEASUREFEST A B/ TESTING
- 18. You need a Framework #MEASUREFEST
- 19.
- 20. Observe #MEASUREFEST Starthere Your customers
- 21. Identify the leaksin your funnel
- 24.
- 25. Brainstorm We believethat changing X for Y customers will result in Z #MEASUREFEST X = variables you want to test Y= target segment you want to run the test on Z = the outcome i.e. change conversions or user behaviour
- 26. Prioritise your testideas Ease of testing Impact on bottom line
- 27. #MEASUREFEST A B/ TESTING Select your sample size Set your test duration QA your test Integrate with analytics
- 28. Don’t stop yourtest too early
- 29. Review & Iterate #MEASUREFEST Segment your data!!! Statistical significance http://abtestguide.com/calc/ Re-test What did you learn?
- 30. Observe Your customers Review & Iterate Brainstorm A/BTesting #MEASUREFEST
- 31. Need inspiration foryour A/B Testing? LOOK AT YOUR DATA #MEASUREFEST
- 32. Interested in moreConversion Optimisation stuff? Get the slides, resources and more http://bit.ly/ digitaltonicmeasurefest #MEASUREFEST Follow me on twitter : @digital_tonic Drop me a line : [email protected]