Andrew Chen Archives

Benedict Evans at a16z recently tweeted the following:

There’s so much truth in this tweet. And it resonates so much, I think it deserves a name:

The Bad Product Fallacy
Your personal use cases and opinion are a shitty predictor of a product’s future success.

I’ve been in the Bay Area for 10 years now, and nothing stings more than whiffing on the prediction of whether a product will be success. Getting this wrong can hurt the ego and sometimes the checkbook too – just ask the dozens of investors who’ve passed on Facebook, Google, Uber, and so on! Personally, I missed completely on Facebook’s potential, and that’s just one of many bad predictions over the years.

The Bad Product Fallacy happens because the trajectory of a product evolves quickly – it’s just software, after all – and a simple set of features can quick grow into a rich, complex platform over time.

Let’s look at some of the comment root causes of the Bad Product Fallacy:

It all starts with a toy
The first and most well-studied root cause of the Bad Product Fallacy is from the theory of disruptive innovation. Many products can look like toys before they become successful. Just take Instagram as an example – it was just a photo filters app at the beginning, and is now one of the largest media properties in the world. Or personal computers, which was initially meant for hobbyists since they were underpowered and weren’t useful for business applications.

This whole phenomenon – widely studied as disruptive innovation theory by Harvard’s Clayton Christensen – is nicely summarized in this blurb:

Disruptive technologies are dismissed as toys because when they are first launched they “undershoot” user needs. The first telephone could only carry voices a mile or two. The leading telco of the time, Western Union, passed on acquiring the phone because they didn’t see how it could possibly be useful to businesses and railroads – their primary customers.

– Chris Dixon, gp at a16z

Here’s now you know you might be falling for this trap: If you use a new product for the first time and say, “huh, is that all there is?” then you may just be whiffing. Or if you complain about a lack of features, even as the underlying technologies are being upgraded extremely rapidly.

Just wait a couple years- by then, the product will have improved so much that you’ll realize you got it all wrong.

Moore’s Law for everything
The inverse of disruptive innovation is that products can start out super premium, but then quickly fall in price to find success in a large, mainstream market. The iPhone is the classic example, but Tesla, Uber, and others are pulling this off too. Sometimes there’s a Moore’s Law kind of effect, where things are getting enormously better and cheaper over time.

Let’s look at the iPhone, the classic example. Steve Ballmer made a very bad prediction – when asked about the new device, he laughed! Not a threat! Instead, he explained why the iPhone would fail:

500 dollars? Fully subsidized? With a plan? I said that is the most expensive phone in the world. And it doesn’t appeal to business customers because it doesn’t have a keyboard. Which makes it not a very good email machine.

-Steve Ballmer, Microsoft on the iPhone

Funny, right? Hindsight is 20/20. Or speaking of phones, here’s another funny example, but about mobile phones in general:

In the early 1980s AT&T asked McKinsey to estimate how many cellular phones would be in use in the world at the turn of the century. The consultancy noted all the problems with the new devices—the handsets were absurdly heavy, the batteries kept running out, the coverage was patchy and the cost per minute was exorbitant—and concluded that the total market would be about 900,000. At the time this persuaded AT&T to pull out of the market, although it changed its mind later.

– The Economist, Oct 1999

But of course, mobile phones as a luxury was quickly fixed. By making the cost per minute cheap and fixing the other technical issues, the mobile phone has become the most ubiquitous computing device in the world.

Here’s how you know you’re about to commit this flavor of the Bad Product Fallacy: If you try a product and ask “Why would anyone pay so much for this?” then you need to think through what happens if the service/product becomes much, much cheaper. Or if it turns out that consumers don’t mind the price. Thinking through these trends can change the game.

I’ve myself missed here when looking at Uber in their early years. When Uber first came out, I thought, wow – why would anyone need an app to call a limo? This is a fancy person’s problem. But of course, if you can get the pricing down from a limo to a taxi, then cheaper to a taxi, and one day cheaper than owning a car – well that’s potentially a trillion dollar company. It turns out there’s some kind of Moore’s Law effect for the cost of transportation over time, and now I’m working there :)

S0me products start by selling stamps, coins, and comic books
Marketplaces have their own flavor of this fallacy because they often start with a vertical niche where buyers/sellers gather, and slowly need to grow to new verticals to be relevant. If these initial niches aren’t your jam, then you may miss on the marketplace’s potential, even if its on a trajectory to ultimately grow into areas that you’ll find useful too.

The classic example of this is eBay: Bessemer Ventures had the chance to invest, but at the time, the marketplace had a lot of collectibles. Here was their evaluation:

“Stamps? Coins? Comic books? You’ve GOT to be kidding,” thought Cowan. “No-brainer pass.”

– Bessemer Venture Partners, Anti-Portfolio page

Of course, eBay went on to add many new verticals, from cars to electronics to much more, eventually returning 700X to their original investors.

The tricky thing here is that you may not want to buy products that are in the marketplace’s initial verticals, which means the product won’t serve your use cases or you won’t love it. However, if you wait a couple years, the marketplace may eventually grow into product categories that you care about.

Social networks and content platforms need density, penetration, to become useful
Finally, let’s look at social/communication/UGC networks which have their own issues. These platforms can be super tricky because similar to marketplaces, they need time to mature as the networks form.

The often cited 1/9/90 rule for digital communities fundamentally drives this dynamic:

The 1% rule states that the number of people who create content on the Internet represents approximately 1% of the people actually viewing that content. For example, for every person who posts on a forum, generally about 99 other people are viewing that forum but not posting. (Wikipedia)

This means that, similar to marketplaces, you need the right balance of content creators and consumers in every vertical of content to have a functioning network. If a social communications product like Snapchat is only useful when you have >5 friends using it, you’ll inherently misunderstand it if the core market is teens and not 40 year old venture capitalists. If you tried using the Internet back in 1990, you may have decided that it’d never work since it’s all academic researchers.

Today, you may be skeptical about VR because it’s mostly games and the apps you’d really like to use haven’t been developed yet. But just wait, it might all click once the right dynamic of content creators, consumers, developers, and other constituents are at the table.

Similar to marketplaces, social networks, communications tools, and user-generated content platforms need critical masses of both creators and consumers to make things work. Sometimes this starts with a niche – like college students or San Francisco techies. But if a product can nail an initial vertical and start hitting up other ones, it may be on its way to mainstream success. Don’t judge too early!

Avoiding the Bad Product Fallacy
In the end, we all love to use our own personal judgement to quickly say yes or no to products. But the Bad Product Fallacy says our own opinions are terrible predictors of success, because tech is changing so quickly.

So instead, I leave you with a couple questions to ask when you are looking at a new product:

• If it looks like a toy, what happens if it’s successful with its initial audience and then starts to add a lot more features?
• If it looks like a luxury, what happens if it becomes much cheaper? Or much better, at the same price?
• If it’s a marketplace that doesn’t sell anything you’d buy, what happens when it starts stocking products and services you find valauble?
• If none of your friends use a social product, what happens when they win a niche and ultimately all your friends are using it too?

It’s hard to ask these questions, since they mostly imply nonlinear trajectories in product innovation. However, technology rarely progresses in a straight line – they grow exponentially, whether in utility, price/performance, or in network effect. Ask yourself the above questions to stay centered, and if you use it to find the next Uber or Facebook, give me (and Ben!) a holler :)

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Leave this field empty if you're human: