1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises Go Back There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. Asked at Facebook over a year ago 3.7k views Request Answer Problem Solving Facebook Instagram Share Interview Guide How to answer Problem Solving questions Save Mark as Read Answers (9) Popular Post Answer Top Facebook interview questions G Gaurav P G Gold PM What is your favorite product? Why? 54 answers | 109k views Nov 20, 2019 How would you design a bicycle renting app for There is a drop of 15% in open rate of Instagram App. You are the tourists? 34 answers | 30.2k PM. Tell us what could have happened? views Build a product to buy and sell antiques. 28 answers | 18.3k Drop in open rate users not even opening once in a day - increase in inactive users users not opening multiple times in a day - reduction in views Show all questions Top Problem Solving interview questions repeat opens You are the PM for a streaming Assumption - This is happening across platform, geography, video service. You come into gender. Also, this is happening gradually over a period of time the office and see that one key (decrease on monthly/weekly basis). metric has dropped by 80%. What will you do? 21 answers | 41.8k views Factors impacting this behavior (metrics to check to see the Drivers are dropping out of a city on Lyft. How do you figure out what's going on? 12 impact) answers | 6.7k views Internal crashes - any app update which lead to crashes Show all questions Metrics to track - Crash Rate, ANR, average load time change in behaviour of users impact of sub metrics (open rate has impact on sub metrics - activities users do post opening the app) complaints - from customer support, play store reviews Play store rating, reviews (1,2,3), sentiment analysis # of complaint tickets External promotions - ran ads which lead to an inorganic boost in open rate (and then sudden drop) https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 1/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises last open date, # of times opened in a day where install source was promotion competitor shift Assumption - Let's assume this drop is due to change in behaviour of use r. Let's deep dive here. To understand the change let's understand how users use the app and what changed. Core activities Feed interaction checking feed DM - chat w/ friends check friends - profile / photos checking friends of friends - profile & follow them posting a photo / video upload / take a photo apply filters adding meta (text, hashtag, location) search & my profile Assumption - Majority of the users are consumers (99%) while only 1% are posters. majority of the time spent by user is on feed interaction activity. rest of the activities are smaller; so there is a change in behaviour on 'feed interaction'. Metrics to check # post scrolled Time spent on feed like a post comments on post stories DM - chat # of user's page opened in a session # of follows / user in a day Possible cases for less open rate Case 1 - takes time to load new post (could be due to bad network) Case 2- post's that are appearing in feed are no more interesting / exciting less leading to lesser open - daily or repeat opens in a day. Moving forward w/ case 2 - which seems to be a possible reason Dig dive on feed composition My friend's post https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 2/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises Influencer's post (celebrities i follow) / stories Assumption - Influencers are the one who drive the open rate a lot as they post more often to have a pull effect. Assuming there is a problem there. Influencer has started posting less engaging content / spam Metrics to check Trend - # of likes, comments on influencer | Time spent on post # of Report spam Sentiment analysis - # of abusive comments (as dislikes) Influencer hasn't been active (posting less content) Metrics to check # of post per day for top influencers (top 1000) Improvements Changes in sorting algo boost post which get more engagement (check growth rate of likes which is increasing as a trend; user you follow and promote it up) demote inactive, less engaging content suggest other influencers to follow Help user generate other interests so that user can follow something else 19 votes | 0 feedback Vote Lokesh Kumar R Give feedback P Platinum PM Jun 27, 2021 That's seems to be a interesting problem and certainly a bigger issue to be sorted as soon as possible. The way I would like to approach the above problem is by first of all by understanding the question properly. Then would like to bucket the probable reasons on a high-level as Internal and External. Depending on my unnderstanding, I would like to start with a bucket an dive-deep into the sub-buckets one by one. So, to start with, My very first question to understand the PS infront of me is: 1. How do we define open rate? Do we consider only the situation where users open the app without any stimulation by the firm through notification, ads or links? Open rate is basically number of app opening in a day. We include all the touchpoints like notification, organic opening, links, and ads. 2. How long are we facing the above issue? We are facing the above issue for quite some-time. The change has been gradual. https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 3/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises 3. Do we see any relationship/connection among different demographics of users like their age, location, device type. No, the drop is pretty consistent across device, location and age factor. Thanks for the above calrity. From the above discussion, we can understand that there could be an influence of external factors causing us the problem as the issue has developed gradually. However, since the issue has been common across worldwide, I would want to check the internal factors, espescially making sure that the data is actually right. Internal Factors Did we make any change to our defintion of metrics? Coz, this might change the number and comparing it with previous data as per the old metric definition might make us feel that there is a drop while in reality, there wouuld have been nothing. No, we did not make any change to the Metric Definition. The other possibility is that there could be a change in how the data is being collected, that is any change in the reporting tools that we use? No, we haven't done any change to the way data is collected. Infact, we are pretty sure that the data is totally right. Oh, Okay. Since I've got such a confirmation the next thing that I would like to ask will be: Did we make any changes in the app? Can you be more specific. So by "any changes", I mean to ask if there was any change in the UI, the algorithm, or some AB test running as a part of some experiment? We haven't done any changes to the Algorithm or are we conducting any AB tests. However there have been some UI changes. What was the UI change that we did? We stopped showing the likes and comments to users to reduce the stress that one undergoes while posting something. However, this is a opt-in option,i.e, that One can bring the likes, comments count tab in settings. Are we sure that the users are aware that they can change this to how it was before? Yeah, we are sure. Because the default setting shows the like and comments count. So If one has turned that off, will certainly know how to turn it on. Wow, that's really a wonderful change. Since it is this way, I doubt this has any influence on what we are facing. This is because, Though likes can cause stress, it certainly helps creators like Meme page owners appreciate themselves and post more. Had this been turned off, creators could have stopped posting actively as they don't have an encouragement to do it. Once, creators posting frequency decreases, the users opening https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 4/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises frequency will decrease as well as they won't have things to see in our app. Okay, with the above discussion, it seems that the internal bucket does not seem to be an issue. Next, I would like to move next to the External bucket. External Bucket: So, in this, the influencing factors could be bad PR, Competition, Government Regulation and Change in User Behaviour. Of this list, I would straight away reject Bad PR as that would more likely cause a Sudden Change and not a gradual one.Similarly, I would reject the Government Regulation as the issue we are facing is worldwide. This seems a reasonable assumption. So what do you want to check next. So next, I would like to check the Competition aspect? Did the competition launch anything new that we know of? We don't have any information on that. Any idea on how we can decide on that? To decide on this and as well sa check if the issue relates to change in User Behaviour, I would first want to know if there are any increase in related metrics like, DAU/MAU, # of uninstallation of apps. This is because, since opening rate has reduced, it could be that active users became more of a casual users or it could be that they stopped being a user of instagram and have uninstalled the app itself. The first would be more related to the change in user behaviour and the latter could be more related to something that the competition did so great that people ignored us totally. By the way the first does not right away rejects that the Competition influence, but I feel it is more reasonable to relate that change to user behaviour. Okay, on checking, we found that the DAU metrics has decreased while MAU has stayed pretty much the same. While there has been only usual noise in the # of uninstallations and nothing significant. Okay, so I would attribute the above issue as a change in User Behaviour as of now. To decide it more accurately, I would like to look into the User Journey aspect and see where do we face a drop apart from the opening rate. User Journey on a higher level Opens the App-> lands on the news feed-> Interacts with the posts-> Session ends Of this, Opening of apps can be through various ways like i listed above. Do we have a difference in any particular way? No. So next, how are the engagement metrics doing with respect to creators? I'm focussing on the creators here because, when opening rate is decreasing and DAU is decreasing, obviously https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 5/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises engagement of users (Consumers) will decrease. Sodiscussing the same will not help us understand the causation. However, If the engagement metrics related to creators are going down, we can understand that the opening rate decreasing is caused by this. On checking, we found that the # of posts generated are decreasing WoW. So, when users don't have new posts to react with, the frequency with which they open the app will reduce. Do you want me to analyse why the creators have reduced postings? Yeah, what you said is the reason.Do share on a high level why creators have decreased their frequency? Ideally speaking I would like to go through their journey and think upon why they are reacting this way. Since you are asking at a higher level, My answer would be: Probably the Steps that the users undergo is more tedious than how the competitors like TikTok have. The reason I'm comparing it with the competition and not the past of IG is that, we got it confirmed that IG team didnot make any change to the app at a technical level. This totally makes sense and you have got it right. The competition has used AI to make the editing process a lot simpler and as well introduced a feature that suggests creators on what they can create next depending on the trend in their location. So as a next step should I also need to talk about what we can do to improvise the situation. No, That is not needed. Thanks for the insights that you brought to the table. 6 votes | 0 feedback Vote R rahul_g Give feedback P Platinum PM Nov 19, 2019 Seek clarity 15% drop over what time frame - is it gradual or drastic? if drastic then figure out the dates when (we'll come back to this later), assuming it's drastic 15% is a drastic drop for a short period for a mature product like IG so there was deifnitely an issue When you say open rate it mean the number of users opening the app or the % of app connect users who end up opening the app? assuming it's the former Rule out errors/ data issues have the metric definition changed recently Is the data updated? https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 6/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises is the drop due to the an artificial spike? example: on new years the number of app open shot up drastically and then normally thus simulating a drastic drop, assume this is not the case Did we make any changes to metric logging frameworks internally? Assuming all of the still suggest there's been a drop Check internal factors Refer to when the drop started - did we do any feature rollout/app release/product releases that day(including front and backend)/experiemnt rollout What other metrics are affected? since this is top of the funnel and data isn't an issue then all metrics down the funnel will be impacted, see which ones are more impacted than other - this should provide a clue What are the customers saying check CS mail/app store/play store review - what are the customers saying Try isolating - is the drop across App/web/mobile web, within app split by OS/OS version/App version/country/ISP/major popular phone models Errors: Any spike in error counts External Check for Major OS release Did playstore or app store make any changes Downtime/outages trends - did the competition launch a new feature/new app We should be able isolate the issue out using the process above 3 votes | 0 feedback Vote S sairah814 Give feedback S Silver PM May 12, 2021 Understand definition of open rate 1. Is this for a specific open rate channel? Push Notifications or organic app opens 2. Is there a specific user group that we’ve seen impacted here: drop in usage overall or drop in return usage over the course of a day 3. What time period have we seen this drop in open rate? Is this a sudden drop or been happening gradually? https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 7/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises Let’s assume open rate = sudden drop in the last week of return usage consistently happening through all channels Understand context 1. Is there a decline in engagement within the app as well or is this limited to users dropping? Has there been a decline with sign ups overall? 2. I’d like to rule out seasonality. Did we see a similar drop this time last year? 3. Do we have any data on the customers that are dropping? I’d like to understand patterns like first time sign up, age groups, and/or typical engagement level. Let’s assume that it’s limited to users dropping (no changes in engagement), not typical from last year and we don’t see any strong patterns on customers dropping although there have been larger declines from the gen x and millennial age group. After comparing this data to other data points, I’d want to focus more deeply on root cause. 1. I think the place I’d want to focus is the types of customers dropping but I’d also want to rule out any feature upgrades that dilute the entry experience into the app. Are you aware of any design changes? 2. Interesting that we see larger declines in the above mentioned groups because they are usually more interested in experimentation and have less loyalty. I’d like to understand if we’ve seen growth of competitors like TikTok or Clubhouse in the market or other new entrants. Let’s assume that we have data that other social media apps have seen increased usage. My hypothesis is that open rates on instagram are dropping because users are spending more time on other social media channels like Tiktok and Clubhouse. Next steps: I would want to explore possible partnerships and features to reengage these users. 1. Review our integration with tiktok and ensure that crossposting hasn’t declined. Investigate further if it has. Otherwise, find new ways to engage users on tiktok to cross post to increase push to instagram. An example might be a special page for user tiktoks or tiktok ratings. 2. Offer an integration with Clubhouse so that users can post which group they’re currently attending or attended in the last week. You could also gamify by scoring channel convos and sharing on instagram 3. Offer a competitive service of “rooms” like clubhouse on instagram directly https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 8/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises 4. Invest in additional video features for Instagram so users can share more videos on Instagram I would first start with how we can increase partnerships with these apps to re-share on instagram. Depending on our relationships, this could be a quick way to re-engage users. 0 votes | 0 feedback Vote Give feedback Leonardo Fernandes G Gold PM Dec 29, 2021 1) Clarification Questions When you say open rate, are people not able to open the app after clicking on it or are they not clicking at all? I am the PM of what team exactly? What is my responsability? Is it specific to iOS or Android or both? Has this drop been gradual or has it been a peak? If it's gradual, for how long has it been droping? Has it happened to a particular geographic region? Has it happened to a particular segment or user group? Let's assume that open rate means people are not clicking on the app; my area does not matter so much; it's not specific to any device; it's a gradual drop over the past 4 weeks; and it's specific to any segment or geographic region. *Well, it's a gradual drop and people are not even clicking on the app, which pops up to my head something related to competitors, user behavior, or cannibalization. a) Internal to the Company Cannibalization Have we launched new products/features that might affect Instagram? Yes, we have launched Shops on Whatsapp. Business Strategies Have we put in place new strategies that justify the drop? For instance, we stopped some notifications. No, we not put in place any relevant business strategy over the last 4 weeks. Marketing Strategies Have we stopped important actions that have helped us get more open rate? No, marketing is still the same. Other Products Have we found similar drops in other products, such as Facebook App, Whatsapp or Messenger? https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 9/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises Not really. On the contrary, Whatsapp downloads and open rate have increased. I'd stop here, since two answers gave me a good direction of what the problem might be -----------------------------More questions to ask--------------------------------------- b) Internal to the Team Bugs Have we noticed weird behaviors of out systems? Do we have any internal report about any problem that might affect the app? Have we experienced loading problems? Partners Are all APIs working properly? Have people found problem in updating the app on google and apple stores? Are these stores working normally? Updates Have we launched new updates recently? Without updating the app, can people still use it? Are we testing new features or products that might have affected it? c) External Competitors Have competitors launched new products? Seasonality Over time, is this drop expected to happen? Public Relations Have we been exposed to bad news on the media? User Behavior Are users shifiting behavior towards the app? ( journey/engagement) ------------------------------------------------------------------------------------------ 2) Hypothesis Well, I believe that the drop is explained by the recently launch of Shops on Whatsapp, the same feature that Instagram has. Since stores on Shops generally have whatsapp channels, people may prefer use it directly on Whatsapp rather than on Instagram. We can research deeper this hypothesis by looking at the following metrics: Avg session and #downloads on Whatsapp of customers who have opened less Instagram Time spent on Shops on Instagram and Whatsapp of those customers Avg number of messages per customer per store on Whatsapp and Instagram of those customers Does it all make sense? 3) Actions https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 10/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises The last thing I wanted to mention is regarding actions towards this problem. I believe that this drop is a trade-off from the launch, so if it was expected, it would not be a big deal. However, we could, for instance, use Whatsapp Shops to directly customers to Instagram to give feedbacks or react to products they bought, something like that. 0 votes | 0 feedback Vote Give feedback Show more answers (4) More product manager interview questions Show all questions If you were the PM for the Save feature at Facebook, what metrics would you use to define the success of this feature? 24 answers / Asked at facebook over a year ago Metrics 42.6k views Facebook How do you say no to feature ideas or requests? 6 answers / Asked at facebook in the past year Leadership And Development Facebook 5.8k views Instacart What should Airbnb's strategy be during the COVID-19 pandemic? Asked at facebook in the past year Product Strategy Facebook P 23Ranswers / 15.8k views Google & 3 other companies Companies Google Product Manager Interview Questions Facebook Product Manager Interview Questions Amazon Product Manager Interview Questions Microsoft Product Manager Interview Questions Uber Product Manager Interview Questions Lyft Product Manager Interview Questions https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 11/12 1/22/22, 1:49 PM There is a 15% drop in the open rate of Instagram App. You are the PM. Tell us what could have happened. — Problem Solving Question | PM Exercises Show More Interview Questions Product Management Product Design Interview Questions Product Management Product Strategy Interview Questions Product Management Product Improvement Interview Product Management Estimation Interview Questions Questions Product Management Behavioral Interview Questions Product Management Metrics Interview Questions Show More hello@productmanagementexercises.com Terms & Conditions Privacy Policy Cookies Copyright © 2021 PM Exercises https://www.productmanagementexercises.com/1937/there-drop-open-rate-instagram-tell-what-could-have-happened 12/12
