2026 Resolution: Be more like a scientist.
A major focus for us this year is increasing experiment velocity for all of our clients.
We have long seen that experiment velocity correlates with business growth & marketing effectiveness. With the added pressure from consumer headwinds, competitive saturation, AI and ✨everything else ✨, it's time to consider an experiment-first approach as table stakes for any brand that wants to thrive.
Velocity is the area most brands fall short. Many brands run zero to a handful of experiments in a year. But velocity without direction or clear bearings is dangerous and can lead to wasted energy or decisions that negatively impact business.
To keep things simple, we use a two bucket framework for planning:
1. Bucket 1: Rapid, Directional Experiments: these are quasi-experiments. Fast, cheap, directional and largely tell you how customers are responding or how a channel is changing. These help you modulate your day-to-day strategy and avoid slow, long term decay.
- What language resonates more with customers right now?
- Which framing gets more attention or engagement?
- Does Feature A outperform Feature B in this context?
These can probably be 80% of your experiments, but deliver 20% of the value.
2. Bucket 2: Structural, Causal Experiments: these are true experiments that tell you what is actually causing outcomes. Slower, statistically rigorous, more expensive and tell you whether your marketing works (and to what degree).
- Is paid media actually driving incremental demand?
- What happens if we remove or add a channel entirely?
- How much of this growth would have happened anyway?
These can probably be 20% of your experiments, but deliver 80% of the value.
A framework like this is essential to avoid the VERY common pitfalls we see:
- Practitioners think these buckets are the same - they treat bucket 2 experiments like the bucket 1. This leads to underinvested tests, false reads, low confidence and bad decisions.
- Practitioners live in one bucket - USUALLY bucket 1. It’s more tangible, it’s more responsive, it's less risky for our egos and we get addicted to the more real-time nature of it. You need BOTH kinds of test to thrive in this environment.
If we can increase testing velocity by 50% for our clients this year, I think that will be a major success. You should challenge yourself to do the same.
New Engen