@andrewchen

Get the newsletter · Featured · Recent · The Cold Start Problem 📘

Why the worst users come from referral programs, free trials, coupons, and gamification

Above: Many small business figured out the hard way why coupon sites generate worse users

Incentive programs often don’t perform
The people you attract with referral programs, free trials, coupons, and gamification — folks who are “incentivized” as a broad umbrella category — are usually MUCH WORSE than organic ones. Worse LTVs, worse conversion, less engaged, and so on.

In a previous life, I headed up Uber’s $300m+/year referral program (“give $5 and get $5”) and learned a ton. Much of the learnings apply to the next wave of gamified consumer apps, web3 games, etc.

So why are these users worse? Let’s discuss.

When CAC/LTV spreadsheets fail
When a new product comes to market, usually the team will measure a baseline set of metrics around lifetime value, etc. if the numbers look good, they might say OK let’s roll out some incentives and get more users like this. Spreadsheets are built, budgets are planned, growth is forecasted, and the new growth project kicks off.

The problem is, all of these forms of incentives usually end up attracting a different type of marginal user that wouldn’t have signed up earlier. They are less qualified, more discount seeking, and behave differently. There is negative selection.

This is especially true when the product has been out there for a while and the core market has mostly been saturated. You also see significant amounts of fraud as users scheme to profit from the incentives. This could be a simple as creating a new account to grab an incentive or it could be something much more organized and nefarious.

This is why core metrics like LTV and engagement can often be half as good or lower, which is often enough to defeat the mathematics that justified the program in the first place. An additional user at upside down mechanics feels good from a top line basis, but in fact, fewer users would be better for the business model. And all the attention towards a complex referral program might take away attention from innovation elsewhere in the product.

One final issue that’s quite subtle, but very important: Cannibalization. You have an target market and sometimes it takes time for a product to spread through its ideal users — this is magical because word of mouth is free. And when it happens in an organic way, the intent is even higher. But if these ideal users encounter the product via an incentive program, you often “pull forward” these users, thus costing you money, when you would have gotten them anyway.

If this all sounds like I might have suffered some trauma from Uber, it’s because I did! Not only did the rider-side referral programs perform worse over time, and perform worse than other channels, in fact, the users were much worse than even users bought from paid ads. It was millions of dollars of spend that didn’t need to happen.

Why this matters — in the world of web3, gamified apps, etc
The ramifications of this are wide, especially on the world of web3, consumer apps that are gamified, etc.

First, it tells you that if you take a game or an app that does not have inherent engagement and retention, it is not enough to add gaming mechanics. If anything, the new mechanics might make things worse, not better, as they attract a group of users who respond to the mechanics, but wouldn’t otherwise use the underlying product. I think we saw a lot of this in web3, where incentivized attracted speculators early on, but struggled to find fun gameplay to attain actual users. Similarly gamified consumer apps (the trad kind) might attract and sustain a certain type of user who is happy to engage in any gamified app, and who will quickly move on because the underlying app doesn’t engage either.

Second, all of these dynamics create sort of a related dynamic to the Law of Shitty Clickthroughs. Not only do individual marketing channels degrade, but many of the new channels you add over time — because they are incentivized — perform worse than the initial channels. Thus the entire machine gets slower and harder as you go.

Final story on this from Uber, funny enough the referral program on the driver side attracted very positively selected users. Whereas the rider referral program got discount seekers, the drivers were highly money motivated. Because they were so motivated and signed up for larger referral bounties, they actually performed better after sign up. Even though referrals was 15% of sign-up they were well over 30% of first trips.

Incentives are a form of selection and you need to make sure you know what you’re selecting for.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at https://a16z.com/investments/. Excluded from this list are investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see https://a16z.com/disclosures for additional important information.