Matt Lerner - Metrics that matter - BoS USA Online 2020

WWW. STARTUPCORESTRENGTHS .COM

METRICS THAT MATTER You get what you
measure. Choose carefully.
Matt Lerner
22 September, 2020

J A N U A R Y, 2 0 0 9
SAN JOSE, CALIFORNIA

SQUARE: $36B STRIPE: $30B

METRICS
YOU GET WHAT YOU MEASURE ALIGN AND
FOCUS THE TEAM A DOUBLE - EDGED
SWORD
IF YOU MEASURE THE WRONG THING


MATT LERNER – FOUNDER & CEOSilicon Valley
veteran (4 startups: ! ," , # , and $ )
Growth team @ PayPal
Partner @ 500 Startups VC

Lectures at Stanford Business School, Imperial Founder
of Startup Core Strengths – Coaching Program
StartupCoreStrengths.com/playbook (it’s free)

GOOD METRICS
HOW DO TARGETS CAUSE SUCCESS?

90% OF YOUR RESULTS
COME
FROM 10% OF YOUR WORK

THE CRITICAL INSIGHT

100%
80%
60%
40%
20%
0%
Results Work

GOOD METRICS
HELP PEOPLE MAKE BETTER DECISIONS ABOUT WHICH WORK TO DO

GOOD METRICS
Measure customer value in the first
instance, not profit

DO NOT CHASE REVENUE IN THE FIRST INSTANCE
WWW.
STARTUPCORESTRENGTHS .COM

DELIVER VALUE AND REVENUE WILL COME

WWW.
STARTUPCORESTRENGTHS .COM

HOW DO YOU MEASSURE VALUE DELIVERED?

WHAT IS CUSTOMER VALUE DELIVERY ?
You deliver value when you help customers make progress
in their lives.
What measurable customer behavior is the closest proxy for
that “value delivery”?
If customers absolutely loved your product or service, how
would they naturally behave?
How many of them are behaving like this?
(Track in cohorts, improve over time.)


YOUR METRICS


THREE KEY TYPES OF METRICS
1. North Star Metric – Customer behavior that
encompasses the whole funnel or marketplace, and
increments when value is delivered to both customer
and the company.
2. Key Drivers – 1-3 Measured first-order impacts of the

work you do each day to maximise the NSM.
(One of these is your “rate limiting step”)
3. Nuance Metrics – Important numbers to watch,
maintain in a range, but not maximise.

WHAT IS YOUR NORTH STAR METRIC ?
North Star Metric Increments when we deliver value to both
customer and company in a natural way where customers
and company behave as expected.
1. Simple & memorable for the whole company

2. Represents the full funnel including new, engaged and
churned users.
3. Does not change often (e.g. every few years max)
4. Everyone can connect their work to this number

A SIMPLE EXAMPLE


YOUR NORTH STAR
Monthly Recurring Revenue (MRR)
Subscribers * Avg Plan Cost

WAUs (Retained + New Actives) + Lapsed
Convert to Paid Plan
Make More Wireframes, Share, etc.
Make First Wireframe
Trial Account
Qualified Traffic

EXAMPLE NORTH STAR METRICS
If customers absolutely loved your product or service, how would they naturally

behave?
B2B
B2C
Slack – Daily Active Users (DAUs)
Amplitude – Weekly querying users
PayPal – Total Payment Volume (TPV)
Hubspot - # of weekly active teams
MARKETPLACES
AirBnB – Nights booked Facebook – DAUs
Quora – Number of answered Medium –
Minutes Read
Your NSM is typically a transaction where two sides come together.

WHAT IS YOUR NORTH STAR METRIC ?
North Star Metric Increments when we deliver value to both
customer and company in a natural way where customers
and company behave as expected.
1. Simple & memorable for the whole company
2. Represents the full funnel including new, engaged and
churned users.
3. Does not change often (e.g. every few years max)
4. Everyone can connect their work to this number

KEY DRIVERS
Key Drivers are the customer behavior “levers” you pull each day
to maximize the North Star. They:
1. Are controllable customer behaviours.
2. Select only a few (1-3)
3. Often, it’s a ratio (e.g. activation rate, signup rate).
4. Revisit every 3 months.
5. Each one should have an owner and target.


EXAMPLE CUSTOMER VALUE KEY DRIVERS

Mission
“Provide simple,
affordable, secure and
reliable digital payments
to…millions of people
around the world.”
“To offer our customers
the lowest possible prices,
the best available
selection, and the utmost
convenience”
“To give people the power
to build community and
bring the world closer
together.”
North Star
Total Payment Volume Repeat Purchases Daily Active
Users
Key Drivers
Ubiquity
(be available at every
transaction point)
Preference
(be every consumer’s
preferred payment method)
Price
Selection
Convenience
DAUs by product, number
of products
Registered users, user
frequency

Invites, follows, post count,
post engagement, etc.

YOUR RATE LIMITING STEP
Which of these metrics, if improved, makes the whole system
run better?
Find it. Focus on it.


YOUR KEY DRIVERS
Monthly Recurring Revenue (MRR)
Make More Wireframes, Share, etc.

Trial Account
Qualified Traffic

WHAT ARE YOUR KEY DRIVERS ?
Which main levers cause your “value delivery metric” to go up?
Which customer behavior best tracks those levers?

s
U
A
W

e
v
i
t
a
l
u
m
u
C
Monthly Cohorts

RETENTION KEY DRIVERS
Absolute numbers matter less than the shape of the

curve
Key Driver:
How would people naturally behave if they loved
your product?
What early life behavior pattern predicts retention?
That’s the habituation threshold
s
U
A
W

e
v
i
t
a
l
u
m
u
C
Monthly Cohorts
What % of new signups cross that line?
Starts with finding your habituation threshold
How much of your revenue is coming from
habituated customers? (Happy MRR) vs. New
customers vs. At-Risk ones?

YOUR HABITUATION THRESHOLD Monthly
Recurring Revenue (MRR)
Make More Wireframes, Share, etc. å
Trial Account
Qualified Traffic


WHAT ARE YOUR KEY DRIVERS ?
Which main levers cause your “value delivery metric” to go up?
Which customer behavior best tracks those levers?
Which one is your rate limiting step?
Who “owns” each of these key metrics?


WHAT ARE YOUR NUANCE METRICS ?
Nuance Metrics are important “read only” numbers (watch but
not maximise). Make sure they stay within an acceptable range.
Many are drivers, but not “key” drivers
Sometimes they are not directly controllable

Helpful if they explain variations in the North Star
Often a “check” number that should stay in a certain range

DOES THIS CHANGE
IF YOU DO NOT HAVE PRODUCT - MARKET FIT ?


“If your product is broken and people are still using it, if you have high retention with a
broken product, that’s a clear sign you have PMF.”

Elad Gil

BE HONEST WITH YOURSELF
Pre-PMF
Post-PMF Post-PMF
“The Hunt” “The Sprint” “The Marathon”
s
e
c
r
u
o
s
e
r

d
n
a

y
gr
e
n
e

,
e
m
i
t

r
u
o
Y

100%
Find
Product-Market
Fit

t
n
a
t
r
o
p
m
I

r
e
p
u
S
n
o
i
s
ic
e
D
60%
Maximise
Proven Vectors
40%
Seek Adjacent
Vectors
80%
Optimise
Core Business
20%
Find New
Business Lines

FINDING PMF KEY DRIVERS
Deepen fit through iterative learning loops.
Key Drivers incent deepening fit + validated

customer learning:
1. Strength of fit – e.g. conversion rates,
activation rates, easy to sell, referral rates,
retention
2. Pace of learning – experiments run per week
Tips: First validate the need. Look for
“language-market fit”

BRINGING YOUR METRICS TO LIFE


HERE IS THE MAGIC

THE PROCESS
The leverage is in which work you choose to do.
Your goal is to get everyone doing the most impactful work, making
good tradeoffs about how to spend their time and budgets.
Each person must understand the North Star, and be able to connect
their work to it. (OKRs)
And that should empower them to make good “resource allocation”
decisions for how to move that metric, as the expert in their
domain.
Keep asking “Which is the most important work, and how is it going?”

COMMON PITFALLS

Data should help us focus, make good
decisions quickly and keep executing.
X Focusing on revenue, not value delivery X
Chasing too many metrics
X Analysis paralysis (e.g. attribution, retention)
X Obsessed with a local maximum


RECAP
North Star Metric of Value Delivery, endures over time
Focus on 1-3 Key Drivers, including your rate limiting step, revisit
focus every 3 months.
If pre-PMF, key drivers focus you on fast learning and fit-deepening
For retention, figure out your habituation threshold & work backwards
Each key driver gets a single accountable person.
Questions for each employee:
Do they understand how their work affects the NSM?

Do they understand their key driver?
Can they make good tradeoffs re. investments of time and budget, and
focus them on the most impactful work?

YOUR HOMEWORK: 5 QUESTIONS
1. A conversation with your CEO, Head of Product, Head
of Marketing, Finance + Analytics. Figure out:
1. What’s your North Star Metric?
2. What are your 1-3 biggest drivers of that metric,
company-wide?
3. What’s your “rate-limiting step”?
2. Ask each employee: “How does your work each day

impact our North Star?” (Check for alignment &
comprehension)
3. Regularly ask each employee or team: “What’s the
most important work, and how’s it going?”

MEASURE TWICE CUT
ONCE

@MattHLerner
StartupCoreStrengths.com/subscrib
e matt@startupcorestrengths.com

Matt Lerner - Metrics that matter - BoS USA Online 2020

More Related Content

Similar to Matt Lerner - Metrics that matter - BoS USA Online 2020

Recently uploaded

Matt Lerner - Metrics that matter - BoS USA Online 2020