15/11/2022, 00:09 You're a PM at Amazon.

Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

You're a PM at Amazon. You released a new feature in the items search functionality recently and
found out that searches increased by 10% but the results page load time is 2 seconds longer. What
would you do?

27.2k
Problem Solving
Asked at
Amazon
views

Interview Guide

How to answer Problem Solving questions

Answers (12) Popular To answer this question, Log in.

Top Amazon interview
Recent questions
Liked by PM Exercises
Popular

Oldest You are the PM for a streaming

PM Team

P
Platinum PM video service. You come into
P
the office and see that one key
Jan 2, 2020
metric has dropped by 80%.
What will you do? 28 answers |
Start answering this Amazon problem solving question 74.6k views

by asking clarifying questions to fully understand data.

How would you launch (roll
out) Amazon Go? 5 answers |
1. Was this feature launched globally? Ans: Yes
8.7k views
2. Is this performance data for the global launch? Ans:
Show all questions
Yes

3. Is a 10% increase per user increase or in the overall

Top Problem Solving
number of total searches? Ans: Overall number of
interview questions
searches

4. Is this 2s increase impacting any You launched a new signup

consumer/business metric? Ans: Yes, Search exits flow to encourage new users to

add more profile information.
have increased & people are leaving the site more
A/B test results indicate that
often the % of people that added
5. In terms of percentage increase, what do 2 seconds more information increased by
8%. However, 7 day retention
mean? Ans: Almost 100%
decreased by 2%. What do you
6. Is this data for the app or website or both? Ans: It is
do? 20 answers | 18.9k views
for website
Drivers are dropping out of a

Understand the goal of the feature city on Lyft. How do you figure
out what's going on? 15

The goal of the feature was to make sure that people are answers | 10k views

able to get to the product they want faster than before &
Show all questions
hence increase average order size per customer

Let's Look at user funnel

User Lands on to the site  ------x%---->Perform Search-

---y%------>Land on SRP-----z%--->Click on a result

So as per my understanding:-

x has increased by 10% (which is good)

y has decreased because of degradation in performance

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 1/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

z has increased because of improvement in search

relevance

So we will focus on figuring & fixing why 'y' has

decreased

Analysis

If x*y*z has decreased which means that less % of

people are clicking on search results than before, then I

would roll-back the feature since fewer people are

progressing on the funnel. The assumption here is that

nothing else is getting impacted by roll-back. 

But for continuing the analysis, I am assuming that more

people are progressing in funnel since x & z have

increased.

Let's focus on why performance has degraded leading to

a decrease in 'y'.

First, let's develop some hypothesis and then we will get

some more data points to analyze the issue

Search Time = Time for the request to reach back end

search server + Server time to perform search + Time for

response to reach client + Time required by the client to

display the SRP

Each of the above is measurable by tools or internal

logging, so I would measure these and compare it with

the previous implementation.

1. Time for the request to reach back end search

server -  After comparison, assume we find out that

the time has increased and this metric has

degraded. Possible reasons are:-

1. The deployment strategy of the new feature

was changed because of which the time

increased. Action item : Evaluate if we can use

the same deployment strategy. Deployment

strategy here means the underlying hardware,

infrastructure, routes (both internal as

external)  etc for the search service

2. Check if search queries from a region where

the performance is not up to the mark have

increased thus skewing up the average time.

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 2/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Action item : Evaluate if this change in search

queries is permanent & if yes, plan a

deployment catering to the region.

3. Check if the performance of a certain region

has degraded significantly.  Action item : Plan a

deployment catering to the region.

4. Check if underlying infrastructure such as

undersea cables etc has snapped leading

requests to take alternate routes that are

taking longer. The probability of this is less

since the impact we have seen is after we

made the changes that point to something

internal. I wouldn't spend too much time on

this analysis.

2. Server time to perform a search -  This is the time

required by the backend to actually perform the

search

1. Check if the number of items to be searched

increased significantly. (Assumption: No)

2. The throughput of the new algorithm for

search has decreased. Action item: If yes, we

will evaluate optimizations in the algorithm as

well as adding more hardware if needed to

improve throughput. 

3. Time for response to reach client - Similar analysis

as point#1. 

4. Time required by the client to display the SRP

1. Is the UI of SRP changed ?- Action item : If yes,

please evaluate the front-end page-size and

other front-end parameters like page load

time, etc.  Optimize the front-end code.

Chrome Audit tool can be used to evaluate the

same. If the UI is not changed, then this

analysis is not needed but we need to make

sure that we are returning the same number of

results as before, else reduce the number of

search results & implement pagination is not

done already.

36 votes
P   |   5 feedback

Vote Give feedback

5 Feedbacks

Tina Patel
PM
PM

Mar 30, 2020

To add to "Server Time to Perform a Search": You

can get a bit more granular and refer to the BigO

notation which states that Big(log_n) are known

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 3/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

to be efficient algorithms that process quickly

whereas the explicit example would be a binary

tree search.

1 Vote Reply

Amit Aggarwal

P
Platinum PM

Jul 26, 2020

Hi Chetan,

Here is my feedback.

I saw that the clarifying questions that you have

asked have nothing to do with your analysis. For

example - Asking "Is this feature globally

launched?" Then how does its yes and no change

your analysis?

Your analysis and your clarification questions

does not match. So asking unrelated questions

during real-time scenarios could create more

confusion among stakeholders than good to the

team.  

Secondly, Your understanding of the goal of the

feature is kinda skewed. Reducing the search

time does not mean "increase in average

customer order", It could mean "More number of

orders per day" or reduction in "Average time

taken to place an order"

8 Vote Reply

Show more feedback 2 of 5

Norge Hund
Gold PM
G

Apr 27, 2020

First, I’d like to clarify what this data reflects.  Is this

overall data for the platform or is it specific to a certain

channel / geography / browser / device / etc.?

Assuming it is generally reflective of the overall

platform, next I’d like to clarify what was the goal in

releasing the new items search feature.  If we only cared

about increasing the number of searches, then the

analysis could stop here.  But if I were the PM for this

offering, the metrics I care more about are average order

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 4/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

size and search conversion rate.  Increasing the number

of searches are a means to achieve these revenue-

related goals.

Next, I’d move onto a hypothesis regarding the slower

load time.  My initial reaction is one of concern, as I

believe this degraded performance could cause users to

grow frustrated and leave the site or close out of the

search.  Let’s think about this in the context of the user

journey:

A) User lands on homepage

B) User performs search

C) User lands on search results

D) User navigates to product page

E) User adds to cart

F) User purchases item

From my prompt we know we’ve increased B, but we also

potentially decreased C due to degraded

performance.  Let’s check some metrics to see if the

change has impacted C, D, E, and F by looking at: 

% search results exited before vs. after the new feature.

% search product pages viewed before vs after the new

feature.

% search results resulting in add to cart before vs. after

the new feature.

% search results resulting in purchase before vs. after

the new feature.

If we notice declines in these metrics, then we know

we’ve got a problem with our new feature, as it is

negatively impacting users.  We can begin to diagnose

the cause of the degraded performance with our

engineers.

5 votes   |   2 feedback

Vote Give feedback

2 Feedbacks

Amit Aggarwal
Platinum PM
P

Jul 26, 2020

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 5/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Hi Norge, 

Here is my feedback 

You have written "If we only cared about

increasing the number of searches, then the

analysis could stop here."

Maybe you would like to see the increase in

search queries as an issue - May be the newly

launched feature has messed up the relevancy

algorithms and people need to fire multiple

search queries to figure out the relevant product. 

1 Vote Reply

Jagadish Avasarala
PM
PM

Jul 31, 2020

It was mentioned no. of searches increased by

10%. Now it depends on no of users.

If the users have increased by 10%, we can

proceed to further analysis.

But if the users are the same / decreased then it

is implied that users are making more searches

with the new feature.

2 Vote Reply

You might also be interested in these 151 questions

  There is a 15% drop in the open rate of Instagram App.

You are the PM. Tell us what could have happened. 8
answers | 5.4k views

  There is a data point that indicates that there are more

Uber drop-offs at the airport than pick-ups from the
airport. Why is this the case and what would you do
within the product to change that? 6 answers | 15.1k
views

  New patients are not showing up for their first telehealth

appointment. How would you solve this issue? 2 answers
| 1.9k views

 
Check out questions Ask question

ashishkatariya2
Bronze PM
B

A
Jan 15, 2020

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 6/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

* First of all this should have been found in testing this

functionality

* Second is my customer getting impacting with this

delayed response. I will do this by checking the data for

the customers.

* I mean 2 seconds is not too much time ( you just blink

your eye and 2 seconds are over) so this can be ignored.

* If my new feature is not affecting any customer then

 no need to do anything.

* There might also be be some  connection issue with

DB, as  there might be multiple hits on this feature on

the server at the same time so the page will take time to

response.

* I will optimise my server load here and will also have a

load balance in place to handle the request.

3 votes   |   1 feedback

Vote Give feedback

1 Feedback

Amit Aggarwal
Platinum PM
P

Jul 26, 2020

Hi Ashish,

Here is my feedback

An increase fo 2 second page load time is too

much for any eCommerce company of any size.

The bad response time of a page is more than 3

seconds.

While answering you should also focus on the

issue "feature has increased the number of

queries". Is this an issue or benefit to the

company?

0 Vote Reply

Swanidhi Singh
PM
PM

Sep 5, 2020

Clarifying questions -

Increase in searches may not imply that users are finding

it relevant and useful - it could be the opposite.  What

was the goal of the feature that was implemented? Was

it to reduce search time or increase the total number of

orders? - Total number of orders

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 7/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Has the CTR for search results been affected?  No

Have there been any marketing campaigns recently that

could add to increased search? No

Was the feature rolled out across both Web/App

platforms?  Yes

Was the featured rolled out globally or across specific

geographies?  Globally

Has the increase in searches considerably increased in

specific countries?  No.

Does the 2-sec increase in page load negatively impact

business goals?  Yes, user drop rate has increased and

total orders have decreased.

Is the drop rate consistent across both the platforms?

Yes

Is there any specific demographic for which the drop

rates have increased? No

Let's consider the user journey - 

A% - User Lands on the Search Page

B% - Performs the Search

C% - Clicks the search result

D% - Adds the item to cart and places the order

Business Goal is to maximize A% x B% C% x

D%, increasing revenue for the company. For the users

who wait for the search to complete, CTR for search

results has not changed - 10% increase in search (B%)

seems to be a positive indicator. 

Hypothesis - Resurrecting the dropped users by

minimizing search time will lead to more conversions.

Search time is the amount of time that the user waits

after performing search action before the results are

displayed on the screen. 

Typical request journey => 1. Request from client to

server => 2. Server to process the request => 3. Response

from server to client => 4. Response is parsed on the FE.

To identify the specific issue - I'd want to work with the

engineering team to measure how performance across

each of these steps has changed before and after the

feature was released.

1. Request from client to server 

I would validate if the request object size is too high.

Is there a new layer that is introduced between frontend

and backend? Eg. A middleware or third-part API service

Is the issue specific to a browser or platform? Are the

coding standards-compliant and optimal?

2. Server to process the request

Is the algorithm complexity acceptable?

More third party integrations? Can those be optimised?

3. Response from server to client (Same as 1)

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 8/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

4. Response is parsed on the FE

Are the loop iterations optimal?

Is FE waiting for a large response? Can it be lazy-

loaded?

Is there room to optimise how information is delivered to

the user?

3 votes   |   0 feedback

Vote Give feedback

stashbag
Gold PM
G

S
Jan 19, 2021

Approach

1. Understand the feature, its goals, the changes to

the feature  and their impact

2. Understand tradeoffs i.e. is the change net positive

to the goals - Determine whether increased

searches > Search Load time increase by 2

seconds+ by evaluating on different parameters

such as customer satisfaction, Order value

increase, order frequency changes, avg check out

times among others. 

3. Based on the data and our business goals

determine the right course of action

Clarification

Is searches increasing a good thing or bad thing I.e.

are customers searching more because they are

searching for new products or they need multiple

queries for the same product

What product did this impact (web or mobile) —>

cross all platforms

Product Description and Goals

Amazon search is the first point of interaction for most

customers in the Amazon app when they are searching

for things (known or browsing). It is key is helping users

find the right product in the shortest amount of time

based on the search query. To that end, the product

goals would be

User Goals

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 9/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

1. Provide accurate and relevant search results across

the Amazon catalog

2. Minimize the effort customers need to make to find

the right product (number of search queries, time

spent searching, modification to search query)

Business goals

1. Make amazon the de facto leader in Product search

i.e. customer originate product searches on Amazon

and not Google 

2. Grow marketplace transactions

Tradeoffs

I would start by into the impact of the feature on the

following metrics pre/post the feature launch so that I

can get a sense of the impact. 

For that I would collect the following metrics:

1. Number of users joining Amazon —> Did searches

increases because of new users or because of

changes to the product. (Increase is bad as we are

seeing slower load times without any material

impact on search queries from existing users. The

increase could be due to new users.)

2. Number of searches per user for each product —>

Has the number of searches gone per user gone up

(More searches are good if accompanied by more

products)

3. Search to cart conversion rate —> What is the

change in add to carts. (More items added to cart

are good)

4. Search to checkout conversion rate —> What is the

change in single click checkouts  (More checkouts

are good, less checkouts are bad)

5. Avg number of orders per customer —> Has the

frequency of orders being placed gone up / down

(increase in orders is good)

6. # of sessions terminating in search —> Are users not

taking any actions post search i.e. abandoning their

journey (this is bad as users are unable to find what

they are looking for)

7. Customer reviews of the Amazon app --> See how

the Amazon app reviews have changed post

feature. (Drop on ratings might mean feature has

negative impact)

Based which direction the above metrics are trending

pre and post the search enhancement we can decide

whether to persist with the feature i.e. do nothing or

whether to address the page load times. 

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 10/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Summary

If the increases in Search queries is leading to more

marketplace transactions and more frequent product

searches and check-outs the feature is a net positive

and we should not do anything. If the greater search

queries isn’t impacting the metrics that point to more

transactions and better customer experience we should

work on address the slower load times. 

2 votes   |   0 feedback

Vote Give feedback

Nitin Narang
Bronze PM
B

N
Jul 1, 2020

To detmine next steps, I would start with what the new

feature attempted to do. 

Assumption:  

1. Data we have is from a clean A/B test and is not

tainted by other factors external to the feature. 

2. Features goal was to make it easier for the users to

search the products they wish to buy. Thus the

success metrics could be :

Primary metric:  "Average $ value of checkout per

user session"

User session is defined as a period of time

with continuous user activity i.e. no user break

greater than ~15 mintues. 

Secondary metric:  Search performance:  Ratio of - 

number of searches / clicks on the SRP.

As response time goes up, this metric would

get worse because of bounce / abandonment

of search. 

Based on the how the feature performed on the primary

and secondary metrics my actions would be as under:

Primary
Secondary
metric

Metric

Action
(Avg $
(Bounce
per user
rate)
session) 

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 11/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Pass

(Unlikely, Roll out. No further

Pass
based on action needed

info we

are given)

1. Roll out. 

Fail
2. Root cause source

(Most delay and try to fix it. 

likely, 3. If we have an
Pass
based on improvement, make an

info we additional release with

are given) the fix and measure

again.

1. Roll back the feature

Pass 

Fail 2. Restrategise how to

(Unlikely)
improve search 

1. Roll back the feature

2. Root cause source of

delay and try to fix it.

Fail Fail
3. Run the experiment

again if we have a fix

1 vote   |   0 feedback

Vote Give feedback

Surbhis
Bronze PM
B

S
Jun 7, 2022

I will approach this question shortly and succinctly.

Any e-commerce or customer-facing portal has a

standard threshold defined for page load time. As per

Google, it should be <3s or in other words, should in

between 0 to 2s. The API ( external and internal )

response time should not be greater than 300ms. 

To start with, I will ask some clarifying questions.

1) Is it page load time which has increased by 2 more

seconds. What was the load time earlier? expected

Answer:  

2) Is it only page load or API response time as well?

Answer: Only page load time has increased.

Approach Type 1 :

Let's consider the interviewer says the page load time

earlier was within 2s in the 1st clarifying question. That

means now the response time has reached tp

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 12/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

to 4s, which is not recommended for any ecomm or

customer-facing portals. It deteriorates the customer

experience and increases the bounce rate. 

Analysis:

The efficiency of the search feature might have

increased by 10% which is good but the load time of

pages can not be traded off with this 10% improvement

of the search functionality.  Why?

Let's consider-

A simple and ideal e comm user customer journey is 

login--> Search the product-->  add in the carts--->

checkout--> make payment and done.

Searching for the product is at the top of the funnel (

awareness) and later part of the journey till the

conversion is largely attributed to good user experience

and the page load.

Solution: I would quickly involve the dev team to check

the backend, database, APIs calls, code quality and

other root cause analysis to take back the page load

time to at least 2s. As this is the industry standard.

Approach Type 2: 

At times interviewer checks whether Tech PMs

understand Dev monitoring alerts related to code

coverage, page load time etc. 

Let's assume If the interviewer's answer to 1st clarifying

question is 0s. Then adding up 2s is still acceptable as it

is still less than 3s.

Solution: 

Google recommended page load time is always between

0 to 2 s or less than 3s. Since the page load time is still

under the range even if it was exceeded by 2s. As long as

it is within the recommended range, there should not be

a panicking situation.  However, the Dev team must need

to set a monitoring alert for a couple of days and do

enough root cause analysis to rule out any code or DB

related issues.

0 votes   |   0 feedback

Vote Give feedback

Show more answers (5)

PREVIOUS

Welcome to PM Exercises...

NEXT

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 13/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Improve Google Maps...

More product manager interview questions Show all questions

How would you improve Amazon?

Asked at Amazon , Walmart

Product Improvement 9 answers / B

24.6k views S

Evaluate the upsides and downsides of building a

super app — an app having all major B2C features
including entertainment, e-commerce, food
ordering, hotel booking, cab booking, chat, holiday
planning, gaming, med ordering, service booking,
etc.

Asked at Amazon , Google & 3 other companies

Product Strategy 8 answers / M P

10.9k views

How would you determine if a specific block in your

neighborhood is suitable for a new grocery store?

Asked at Amazon , Google & 2 other companies

Product Strategy 12 answers / 10.5k views S P

Companies

Google Product Manager Interview Questions Facebook Product Manager Interview Questions

Amazon Product Manager Interview Questions Microsoft Product Manager Interview Questions

Uber Product Manager Interview Questions Lyft Product Manager Interview Questions

Show More

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 14/15
15/11/2022, 00:09 You're a PM at Amazon. You released a new feature in the items search functionality recently and found out that searches increased by 10% but the results page load time is 2 seconds longer. What would you do? — …

Interview Questions

Product Management Product Design Interview Questions Product Management Product Strategy Interview
Questions

Product Management Product Improvement Interview Product Management Estimation Interview Questions
Questions

Product Management Behavioral Interview Questions Product Management Metrics Interview Questions

Show More

[email protected]
Terms & Conditions
Privacy Policy
Cookies

Copyright © 2021 PM Exercises

https://www.productmanagementexercises.com/2102/released-functionality-recently-searches-increased-results 15/15